Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create manifest file directly in S3. #4400

Open
2 tasks done
ztyree42 opened this issue Mar 1, 2022 · 0 comments
Open
2 tasks done

Create manifest file directly in S3. #4400

ztyree42 opened this issue Mar 1, 2022 · 0 comments
Assignees
Labels
enhancement New feature or request

Comments

@ztyree42
Copy link

ztyree42 commented Mar 1, 2022

My actions before raising this issue

Expected Behaviour

create.py should accept an s3 path in lieu of local file path.
create.py functionality should be accessible from UI.

Current Behaviour

User must download dataset locally -> create manifest -> upload to s3.

Possible Solution

Steps to Reproduce (for bugs)

Context

My organization has many datasets in s3 already so downloading each dataset just to be able to create the manifest.json is out of the question. s3fs is not a solution for files and folders already existing in s3 (it works fine if you're starting with an empty bucket and like that workflow). Moreover the AWS Registry of Open Data hints at a workflow wherein datasets never have to be downloaded locally but this seems out of reach with the current CVAT manifest schema. Finally, for those users attempting to host CVAT on EC2 and expose it to end users through the browser adding a new dataset to s3 now requires SSH access to the EC2 undermining the ability to allow users to create new annotation tasks (or add new datasets) without also giving SSH access, which I find extreme.

Your Environment

  • Git hash commit (git log -1):
  • Docker version docker version (e.g. Docker 17.0.05):
  • Are you using Docker Swarm or Kubernetes?
  • Operating System and version (e.g. Linux, Windows, MacOS):
  • Code example or link to GitHub repo or gist to reproduce problem:
  • Other diagnostic information / logs:
    Logs from `cvat` container

Next steps

You may join our Gitter channel for community support.

@Marishka17 Marishka17 added the enhancement New feature or request label Mar 15, 2022
nmanovic added a commit that referenced this issue May 20, 2023
### Motivation and context
Resolved #6037
Related #4400
Related #6028


![image](https://user-images.githubusercontent.com/49038720/236890662-c44b578e-5808-4fde-a216-2dcab6e95ab0.png)

Co-authored-by: Boris Sekachev <boris.sekachev@yandex.ru>
Co-authored-by: Boris Sekachev <sekachev.bs@gmail.com>
Co-authored-by: Roman Donchenko <roman@cvat.ai>
Co-authored-by: Nikita Manovich <nikita@cvat.ai>
mikhail-treskin pushed a commit to retailnext/cvat that referenced this issue Jul 1, 2023
### Motivation and context
Resolved cvat-ai#6037
Related cvat-ai#4400
Related cvat-ai#6028


![image](https://user-images.githubusercontent.com/49038720/236890662-c44b578e-5808-4fde-a216-2dcab6e95ab0.png)

Co-authored-by: Boris Sekachev <boris.sekachev@yandex.ru>
Co-authored-by: Boris Sekachev <sekachev.bs@gmail.com>
Co-authored-by: Roman Donchenko <roman@cvat.ai>
Co-authored-by: Nikita Manovich <nikita@cvat.ai>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants