Skip to content

Commit

Permalink
create WikiAPI dataclasses
Browse files Browse the repository at this point in the history
- standardize requests url format and ensure parameters no longer in hardcoded url strings
- separate functionality for scraping raw data from the wiki (.scrape) and processing it (.parse) with option to combine (.build)
- in WikiCategory class, combine API calls for retrieving page titles within a category & getting subcategories of a category
- ability to go recursively through a category to get pages from all subcategories as well (`recursive=True`), with the option to create a separate list of pages for each subcategory (`group_pages=True`)
- parse starred lists within a single infobox parameter
- will create a separate dataframe for every type of infobox that appears on the set of pages
- create .json file from WikiInfobox dataframe instead of dictionaries
- added dictionary of namespaces on Fandom Wiki (plus ability to build that dict in WikiAPI base class)
  • Loading branch information
eseiver committed Jan 4, 2022
1 parent ef582e9 commit 06337f5
Showing 1 changed file with 482 additions and 234 deletions.
Loading

0 comments on commit 06337f5

Please sign in to comment.