✨ anomalist: improve utils #3385
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Quick links (staging server):
Login: chart-diff: ✅No charts for review.Edited: 2024-10-08 14:45:06 UTC |
Marigold
reviewed
Oct 9, 2024
| return dataset_id | ||
|
|
||
|
|
||
| def get_variables_in_dataset( |
There was a problem hiding this comment.
Eventually, I think we should move this as a method to grapher_model.Dataset so that all dataset-related methods are in one place. (Not now though!)
There was a problem hiding this comment.
this makes sense. I've added a @deprecated tag
Marigold
reviewed
Oct 9, 2024
|
|
||
|
|
||
| def get_variables_in_dataset( | ||
| dataset_id: int, only_used_in_charts: bool = False, db_conn: Optional[pymysql.Connection] = None |
There was a problem hiding this comment.
db_conn should eventually also get deprecated. We should only use engine or session and read_sql function from db.py.
There was a problem hiding this comment.
this makes sense. I've added a @deprecated tag
lucasrodes
added a commit
that referenced
this pull request
Oct 9, 2024
* ✨ wizard: anomalies * wip * bump streamlit * wip * wip: chart * wip * todo * plot indicator * re-structure * wip: loading indicators * fix API grapher_chart * deprecate chart_html * chart_html -> grapher_chart * clean * ci/cd * wip * wip * changed module name * custom components module * add methods to get uris * new alias * get dataset uris * update import * update gpt pricing * update import * wip * provide entity-context for anomaly * wip: anomalist v2 * wip * wip * lock * ✨ anomalist: improve utils (#3385) * wip * db -> db_utils * io -> db * move things db_utils -> db * db -> grapher_io * db -> grapher_io, db_utils -> db * docstring * db_utils -> db * wip * remove indicator * add overloads * ci/cd * wip * cicd * wip * deprecation warnings * missing import * hide anomalist in wizard
paarriagadap
pushed a commit
that referenced
this pull request
Oct 9, 2024
* ✨ wizard: anomalies * wip * bump streamlit * wip * wip: chart * wip * todo * plot indicator * re-structure * wip: loading indicators * fix API grapher_chart * deprecate chart_html * chart_html -> grapher_chart * clean * ci/cd * wip * wip * changed module name * custom components module * add methods to get uris * new alias * get dataset uris * update import * update gpt pricing * update import * wip * provide entity-context for anomaly * wip: anomalist v2 * wip * wip * lock * ✨ anomalist: improve utils (#3385) * wip * db -> db_utils * io -> db * move things db_utils -> db * db -> grapher_io * db -> grapher_io, db_utils -> db * docstring * db_utils -> db * wip * remove indicator * add overloads * ci/cd * wip * cicd * wip * deprecation warnings * missing import * hide anomalist in wizard
3 tasks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR re-organizes several grapher/db/s3 utils in our library.
We had some tools spread in multiple modules, within
etl/and inapps/. The idea is to compile these in centralised modules, so that it is easier to improve them, their maintenance, and enhancement.Changes include:
etl.db: This module already existed but included a mixture of database utils and methods to query specific bits of data from our database. From now on, it should only include database utils.etl.grapher_io: This is a new module, which should contain all our tools to access data/metadata (and possibly other bits of content) from our Grapher database/api (mysql/s3). The important bit here, is that it accesses non-etl content. Its content comes from various places: new content, frometl.db, from,apps.backport.datasync.data_metadata, etc.etl.grapher_model: Added some functions to some ORM objects. For instance, classVariablehas now aget_datamethod to access it's data (i.e. it accesses s3 data).