Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Epic] Polish Core Datasets #268

Closed
7 of 14 tasks
sglavoie opened this issue May 11, 2020 · 1 comment
Closed
7 of 14 tasks

[Epic] Polish Core Datasets #268

sglavoie opened this issue May 11, 2020 · 1 comment

Comments

@sglavoie
Copy link

sglavoie commented May 11, 2020

As a PM, I want to review core datasets and make sure all of them are up to date so that I'm sure in their quality

As a PM, I want to re-run and make sure scrapers/scripts are working fine so that I can update data anytime

As a PM, I want to translate scrapers/scripts into dataflows, where possible, so that we use our tools to get the data, plus we can easily update them

As a PM, I want to review READMEs of the core dataset and update where necessary, so that I (and users) are sure that dataset descriptions are accurate enough

Acceptance Criteria

  • We have the latest data for all core datasets
  • We have all of the non-complex scripts working OK (unless the source is broken)
  • We have the missing sources and complex scripts fixed up
  • READMEs are up to date
  • We use dataflows to get the data
    • Automated by Travis

Tasks

  • List all datasets
  • Find and fix the non-complex scripts that are not currently working and review READMEs
  • Fix scripts that are complex and have non-common errors
  • translate scripts to dataflows
  • Fix/refactor more simple scripts addendum #267
  • Fix the broken source datasets #266
  • Fix scripts that require further analysis and debugging #265
  • Run on schedule by travis

Created by @zelima

@rufuspollock
Copy link
Member

FIXED or DUPLICATE of datasets/awesome-data#376

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants