-
Notifications
You must be signed in to change notification settings - Fork 106
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
add DatasetListRecord and DatasetListVersion dataclasses to only extract required fields when listing datasets #634
Conversation
Deploying datachain-documentation with
|
Latest commit: |
1133dab
|
Status: | ✅ Deploy successful! |
Preview URL: | https://867ae8d3.datachain-documentation.pages.dev |
Branch Preview URL: | https://add-dataset-list-dataclass.datachain-documentation.pages.dev |
b909d5b
to
5d96190
Compare
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #634 +/- ##
==========================================
- Coverage 87.74% 87.70% -0.04%
==========================================
Files 112 112
Lines 10605 10672 +67
Branches 1431 1437 +6
==========================================
+ Hits 9305 9360 +55
- Misses 946 954 +8
- Partials 354 358 +4
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. |
9f4ddf0
to
53053f6
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good to me 👍
57edb12
to
b86791f
Compare
b86791f
to
1133dab
Compare
Related to https://github.com/iterative/studio/issues/10849 and replaces #628 (see replaced PR for some details)
This PR adds
DatasetListRecord
andDatasetListVersion
dataclass
es and uses those classes withinCatalog
'sls_datasets
andlist_datasets_versions
methods. Using these new classes means that we can significantly cut down the data being processed when listing datasets. This should give us a performance boost in both the CLI and SaaS and hopefully prevent the slow query alerts in the related issue.Note: There is some duplication added in this PR, it does not seem practical to extend the original
dataclass
es in this instance.