Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

tld: comprehensive rewrite #1939

Merged
merged 3 commits into from
Sep 24, 2020
Merged

tld: comprehensive rewrite #1939

merged 3 commits into from
Sep 24, 2020

Commits on Sep 16, 2020

  1. tld: use TLD list from IANA to validate queried items' existence 

    List is cached in `bot.memory` at runtime, and persisted to a plugin
value in `bot.db` at shutdown. If a cached list exists, it is loaded
into `bot.memory` at startup.

The list is updated automatically once the data is 7 days old.
    @dgw
    dgw committed Sep 16, 2020
    Configuration menu
    Copy the full SHA
    1a0b4ee View commit details
    Browse the repository at this point in the history

  2. tld: completely overhaul data fetching from Wikipedia 

    Replaced those horrid (and ugly) regex patterns with a bare-bones HTML
parser, an in-memory lookup dictionary, and dynamic generation of the
output line depending on what fields are available.

Surely there are some remaining edge cases (though I threw a number of
different TLDs at this and knocked out some already), but this is an
absolutely MASSIVE improvement in maintainability IMO.

Uses the same caching logic as for the IANA TLD list, so any given
Sopel instance will only hit Wikipedia at most once a week or so.
    @dgw
    dgw committed Sep 16, 2020
    Configuration menu
    Copy the full SHA
    816ac86 View commit details
    Browse the repository at this point in the history

  3. tld: add basic formatting support (bold & italic) to TLD data parser 

    Also moved the data-processing step inside the parser object, because
having a helper function for that at the module level didn't make sense.
    @dgw
    dgw committed Sep 16, 2020
    Configuration menu
    Copy the full SHA
    f67737d View commit details
    Browse the repository at this point in the history