Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

✨ anomalist: Improve automatic detection of new datasets #3429

Merged

Conversation

pabloarosado
Copy link
Contributor

@pabloarosado pabloarosado commented Oct 18, 2024

  • Simplify the way we detect new datasets, and with this speed up the start of Anomalist.
  • Infer a variable mapping when indicator upgrader has not been yet executed.

@owidbot
Copy link
Contributor

owidbot commented Oct 18, 2024

Quick links (staging server):

Site Admin Wizard

Login: ssh owid@staging-site-improve-new-datasets-detection

chart-diff: ✅ No charts for review.

Edited: 2024-10-18 10:55:06 UTC
Execution time: 3.47 seconds

@pabloarosado pabloarosado requested a review from Marigold October 18, 2024 12:25
@pabloarosado pabloarosado marked this pull request as ready for review October 18, 2024 12:25
Copy link
Collaborator

@Marigold Marigold left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code looks good! I'm a bit confused by the way we get new datasets, but I'll ask on slack.

@pabloarosado pabloarosado merged commit e65060d into try-different-anomaly-detectors Oct 18, 2024
7 of 8 checks passed
@pabloarosado pabloarosado deleted the improve-new-datasets-detection branch October 18, 2024 13:47
pabloarosado added a commit that referenced this pull request Oct 18, 2024
* 🎉 anomalist: Experiment with different anomaly detection methods

* Improve script to visualize anomalies

* Improve visualization of anomalies, and try different methods

* Improve cli

* Some refactoring

* Add useful comment

* ✨ anomalist: Improve automatic detection of new datasets (#3429)

* ✨ anomalist: Improve automatic detection of new datasets

* Create new functions to detect new datasets, and speed up anomalist

* Infer variable mapping

* Use inferred variable mapping in Anomalist

* Move function to get datasets info
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants