Skip to content

feature request: allow configuration of specific title languages for scraping #606

@kibabyte

Description

@kibabyte

Checklist

  • I checked that this feature has not been requested before
  • I checked that this feature is not in the "Not planned" list
  • This feature will benefit the majority of users

Problem Description / Use Case

Currently, the environment variable BUILTIN_SCRAPE_WITH_ALL_TITLES and BUILTIN_SCRAPE_TITLE_LIMIT only allow enabling/disabling or limiting the number of alternative titles that are queried for scraping in AIOStreams. However, there is no way to specify that only certain languages' titles should be considered (for example, only English and German), which can result in unnecessary queries for titles in unwanted languages.

Proposed Solution

Please add a configuration or environment variable that allows users to specify exactly which languages of titles should be used when scraping (e.g., "EN,DE"). This would provide users with better control over the behavior, reduce unnecessary queries for irrelevant language titles, and improve scraping efficiency. The idea is not to set a numeric limit or a full on/off toggle, but to allow for inclusion of specific languages based on user preference.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions