Improve task creation with cloud storage and share data #6074
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
🚫 Workflows has been canceled |
|
/check |
|
✔️ All checks completed successfully |
…for manifest content
<!-- Raise an issue to propose your change (https://github.com/opencv/cvat/issues). It helps to avoid duplication of efforts from multiple independent contributors. Discuss your ideas with maintainers to be sure that changes will be approved and merged. Read the [Contribution guide](https://opencv.github.io/cvat/docs/contributing/). --> <!-- Provide a general summary of your changes in the Title above --> ### Motivation and context <!-- Why is this change required? What problem does it solve? If it fixes an open issue, please link to the issue here. Describe your changes in detail, add screenshots. --> ### How has this been tested? <!-- Please describe in detail how you tested your changes. Include details of your testing environment, and the tests you ran to see how your change affects other areas of the code, etc. --> ### Checklist <!-- Go over all the following points, and put an `x` in all the boxes that apply. If an item isn't applicable for some reason, then ~~explicitly strikethrough~~ the whole line. If you don't do that, GitHub will show incorrect progress for the pull request. If you're unsure about any of these, don't hesitate to ask. We're here to help! --> - [ ] I submit my changes into the `develop` branch - [ ] I have added a description of my changes into the [CHANGELOG](https://github.com/opencv/cvat/blob/develop/CHANGELOG.md) file - [ ] I have updated the documentation accordingly - [ ] I have added tests to cover my changes - [ ] I have linked related issues (see [GitHub docs]( https://help.github.com/en/github/managing-your-work-on-github/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword)) - [ ] I have increased versions of npm packages if it is necessary ([cvat-canvas](https://github.com/opencv/cvat/tree/develop/cvat-canvas#versioning), [cvat-core](https://github.com/opencv/cvat/tree/develop/cvat-core#versioning), [cvat-data](https://github.com/opencv/cvat/tree/develop/cvat-data#versioning) and [cvat-ui](https://github.com/opencv/cvat/tree/develop/cvat-ui#versioning)) ### License - [ ] I submit _my code changes_ under the same [MIT License]( https://github.com/opencv/cvat/blob/develop/LICENSE) that covers the project. Feel free to contact the maintainers if that's a concern.
…ncv/cvat into mk/make_cs_manifest_optional
…ncv/cvat into mk/make_cs_manifest_optional
SpecLad
reviewed
May 16, 2023
SpecLad
reviewed
May 16, 2023
SpecLad
reviewed
May 16, 2023
Co-authored-by: Roman Donchenko <roman@cvat.ai>
SpecLad
reviewed
May 16, 2023
SpecLad
approved these changes
May 16, 2023
… into mk/make_cs_manifest_optional
Merged
nmanovic
added a commit
that referenced
this pull request
Jun 2, 2023
## \[2.4.5] - 2023-06-02 ### Added - Integrated support for sharepoint and cloud storage files, along with directories to be omitted during task creation (server) (<#6074>) - Enabled task creation with directories from cloud storage or sharepoint (<#6074>) - Enhanced task creation to support any data type supported by the server by default, from cloud storage without the necessity for the `use_cache` option (<#6074>) - Added capability for task creation with data from cloud storage without the `use_cache` option (<#6074>) ### Changed - User can now access resource links from any organization or sandbox, granted it's available to them (<#5892>) - Cloud storage manifest files have been made optional (<#6074>) - Updated Django to the 4.2.x version (<#6122>) - Renamed certain Nuclio functions to adhere to a common naming convention. For instance, `onnx-yolov7` -> `onnx-wongkinyiu-yolov7`, `ultralytics-yolov5` -> `pth-ultralytics-yolov5` (<#6140>) ### Deprecated - Deprecated the endpoint `/cloudstorages/{id}/content` (<#6074>) ### Fixed - Fixed the issue of skeletons dumping on created tasks/projects (<#6157>) - Resolved an issue related to saving annotations for skeleton tracks (<#6075>) --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: Boris Sekachev <boris.sekachev@yandex.ru> Co-authored-by: Roman Donchenko <roman@cvat.ai> Co-authored-by: Maria Khrustaleva <maya17grd@gmail.com> Co-authored-by: Boris Sekachev <sekachev.bs@gmail.com> Co-authored-by: Nikita Manovich <nikita@cvat.ai> Co-authored-by: Anastasia Yasakova <yasakova_anastasiya@mail.ru> Co-authored-by: Snyk bot <snyk-bot@snyk.io> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Kirill Sizov <kirill.sizov@cvat.ai> Co-authored-by: Paweł Kotiuk <kotiuk@zohomail.eu> Co-authored-by: SK <450723+senthilkumarkj@users.noreply.github.com> Co-authored-by: Kirill Lakhov <kirill.9992@gmail.com>
mikhail-treskin
pushed a commit
to retailnext/cvat
that referenced
this pull request
Jul 1, 2023
### Motivation and context Resolved cvat-ai#6037 Related cvat-ai#4400 Related cvat-ai#6028  Co-authored-by: Boris Sekachev <boris.sekachev@yandex.ru> Co-authored-by: Boris Sekachev <sekachev.bs@gmail.com> Co-authored-by: Roman Donchenko <roman@cvat.ai> Co-authored-by: Nikita Manovich <nikita@cvat.ai>
|
What's the downside of creating a task directly vs using a manifest? From the UI there's this message:
Can this be quantified for an S3 folder of say 30k images? |
|
Roughly saying each request to S3 costs money. In case of using manifest, when you browse the cloud storage from CVAT, this is for free, because it uses data from manifest. |
|
And browsing itself works faster |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Motivation and context
Resolved #6037
Related #4400
Related #6028
This PR contains several improvements in working with cloud storage/share data:
use_cachevideo) withoutuse_cachecloudstorages/{id}/contentendpointHow has this been tested?
Added REST API tests for cloud storage content and task creation actions.
Checklist
developbranch(cvat-canvas,
cvat-core,
cvat-data and
cvat-ui)
License
Feel free to contact the maintainers if that's a concern.