-
Notifications
You must be signed in to change notification settings - Fork 0
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
LitData Refactor PR3: Add custom StreamingDataset #92
LitData Refactor PR3: Add custom StreamingDataset #92
Conversation
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the You can disable this status message by setting the Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media? 🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
CodeRabbit Configuration File (
|
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## divya/add-datachunks-func #92 +/- ##
=============================================================
- Coverage 97.81% 97.49% -0.33%
=============================================================
Files 36 37 +1
Lines 3391 3515 +124
=============================================================
+ Hits 3317 3427 +110
- Misses 74 88 +14 ☔ View full report in Codecov by Sentry. |
* Add flag for augmentation * Modify exception * Fix tests * Add litdata to trainer * Modify test * Add tests for data loaderS * Fix tests * Remove files in trainer * Remove val chunks dir * Remove shutil.rmtree * Remove shutil.rmtree * Skip ubuntu test * fix skip ubuntu test * Fix changes * Save training config before fit
… all model types (#91) * Add function to get data chunks * LitData Refactor PR3: Add custom StreamingDataset (#92) * Add custom streamingdatasets * LitData Refactor PR4: Integrate LitData with ModelTrainer class (#94) * Add flag for augmentation * Modify exception * Fix tests * Add litdata to trainer * Modify test * Add tests for data loaderS * Fix tests * Remove files in trainer * Remove val chunks dir * Remove shutil.rmtree * Remove shutil.rmtree * Skip ubuntu test * fix skip ubuntu test * Fix changes * Save training config before fit
* Add individual datapipe functions * LitData Refactor PR2: Implement a function to get the data chunks for all model types (#91) * Add function to get data chunks * LitData Refactor PR3: Add custom StreamingDataset (#92) * Add custom streamingdatasets * LitData Refactor PR4: Integrate LitData with ModelTrainer class (#94) * Add flag for augmentation * Modify exception * Fix tests * Add litdata to trainer * Modify test * Add tests for data loaderS * Fix tests * Remove files in trainer * Remove val chunks dir * Remove shutil.rmtree * Remove shutil.rmtree * Skip ubuntu test * fix skip ubuntu test * Fix changes * Save training config before fit * Fix type annotations
This is the third PR for #80. Here, we implement a custom
litdata.StreamingDataset
class for each model type. In thelitdata.StreamingDataset.__getitem__()
method, we apply augmentation, resizer, pad_to_stride and generate confidence maps (and part affinity fields for bottom-up model) .