Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve SDK UX with task creation #5502

Merged
merged 18 commits into from
Jan 2, 2023
Merged

Improve SDK UX with task creation #5502

merged 18 commits into from
Jan 2, 2023

Conversation

zhiltsov-max
Copy link
Contributor

@zhiltsov-max zhiltsov-max commented Dec 21, 2022

Motivation and context

Extracted from #5083

  • Added a default arg for task data uploading - the data is now considered local by default
  • Added an option to wait for the data processing in task data uploading
  • Moved data splitting by requests for TUS closer to the point of use

How has this been tested?

Checklist

License

  • I submit my code changes under the same MIT License that covers the project.
    Feel free to contact the maintainers if that's a concern.

@nmanovic nmanovic requested review from SpecLad and removed request for nmanovic and azhavoro December 27, 2022 20:29
@nmanovic
Copy link
Contributor

@SpecLad , could you please review?

cvat-sdk/cvat_sdk/core/proxies/tasks.py Outdated Show resolved Hide resolved
cvat-sdk/cvat_sdk/core/proxies/tasks.py Outdated Show resolved Hide resolved
cvat-sdk/cvat_sdk/core/proxies/tasks.py Show resolved Hide resolved
cvat-sdk/cvat_sdk/core/uploading.py Outdated Show resolved Hide resolved
@nmanovic
Copy link
Contributor

@zhiltsov-max , could you please look at tests?

"root:root",
[3667](https://github.com/opencv/cvat/actions/runs/3810958003/jobs/6483468902#step:9:3668)
                "-v",
[3668](https://github.com/opencv/cvat/actions/runs/3810958003/jobs/6483468902#step:9:3669)
                f"{tmp_dir}:/local",
[3669](https://github.com/opencv/cvat/actions/runs/3810958003/jobs/6483468902#step:9:3670)
                "--entrypoint",
[3670](https://github.com/opencv/cvat/actions/runs/3810958003/jobs/6483468902#step:9:3671)
                "python3",
[3671](https://github.com/opencv/cvat/actions/runs/3810958003/jobs/6483468902#step:9:3672)
                "cvat/server:dev",
[3672](https://github.com/opencv/cvat/actions/runs/3810958003/jobs/6483468902#step:9:3673)
                "utils/dataset_manifest/create.py",
[3673](https://github.com/opencv/cvat/actions/runs/3810958003/jobs/6483468902#step:9:3674)
                "--output-dir",
[3674](https://github.com/opencv/cvat/actions/runs/3810958003/jobs/6483468902#step:9:3675)
                "/local",
[3675](https://github.com/opencv/cvat/actions/runs/3810958003/jobs/6483468902#step:9:3676)
                "/local",
[3676](https://github.com/opencv/cvat/actions/runs/3810958003/jobs/6483468902#step:9:3677)
            ]
[3677](https://github.com/opencv/cvat/actions/runs/3810958003/jobs/6483468902#step:9:3678)
>           subprocess.run(command, check=True)
[3678](https://github.com/opencv/cvat/actions/runs/3810958003/jobs/6483468902#step:9:3679)

[3679](https://github.com/opencv/cvat/actions/runs/3810958003/jobs/6483468902#step:9:3680)
tests/python/rest_api/test_tasks.py:754: 
[3680](https://github.com/opencv/cvat/actions/runs/3810958003/jobs/6483468902#step:9:3681)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
[3681](https://github.com/opencv/cvat/actions/runs/3810958003/jobs/6483468902#step:9:3682)
/opt/hostedtoolcache/Python/3.8.15/x64/lib/python3.8/subprocess.py:495: in run
[3682](https://github.com/opencv/cvat/actions/runs/3810958003/jobs/6483468902#step:9:3683)
    stdout, stderr = process.communicate(input, timeout=timeout)
[3683](https://github.com/opencv/cvat/actions/runs/3810958003/jobs/6483468902#step:9:3684)
/opt/hostedtoolcache/Python/3.8.15/x64/lib/python3.8/subprocess.py:1020: in communicate
[3684](https://github.com/opencv/cvat/actions/runs/3810958003/jobs/6483468902#step:9:3685)
    self.wait()
[3685](https://github.com/opencv/cvat/actions/runs/3810958003/jobs/6483468902#step:9:3686)
/opt/hostedtoolcache/Python/3.8.15/x64/lib/python3.8/subprocess.py:1083: in wait
[3686](https://github.com/opencv/cvat/actions/runs/3810958003/jobs/6483468902#step:9:3687)
    return self._wait(timeout=timeout)
[3687](https://github.com/opencv/cvat/actions/runs/3810958003/jobs/6483468902#step:9:3688)
/opt/hostedtoolcache/Python/3.8.15/x64/lib/python3.8/subprocess.py:1806: in _wait
[3688](https://github.com/opencv/cvat/actions/runs/3810958003/jobs/6483468902#step:9:3689)
    (pid, sts) = self._try_wait(0)
[3689](https://github.com/opencv/cvat/actions/runs/3810958003/jobs/6483468902#step:9:3690)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
[3690](https://github.com/opencv/cvat/actions/runs/3810958003/jobs/6483468902#step:9:3691)

[3691](https://github.com/opencv/cvat/actions/runs/3810958003/jobs/6483468902#step:9:3692)
self = <subprocess.Popen object at 0x7eff066ec7c0>, wait_flags = 0
[3692](https://github.com/opencv/cvat/actions/runs/3810958003/jobs/6483468902#step:9:3693)

[3693](https://github.com/opencv/cvat/actions/runs/3810958003/jobs/6483468902#step:9:3694)
    def _try_wait(self, wait_flags):
[3694](https://github.com/opencv/cvat/actions/runs/3810958003/jobs/6483468902#step:9:3695)
        """All callers to this function MUST hold self._waitpid_lock."""
[3695](https://github.com/opencv/cvat/actions/runs/3810958003/jobs/6483468902#step:9:3696)
        try:
[3696](https://github.com/opencv/cvat/actions/runs/3810958003/jobs/6483468902#step:9:3697)
>           (pid, sts) = os.waitpid(self.pid, wait_flags)
[3697](https://github.com/opencv/cvat/actions/runs/3810958003/jobs/6483468902#step:9:3698)
E           Failed: Timeout >15.0s

@nmanovic nmanovic merged commit c808471 into develop Jan 2, 2023
@nmanovic nmanovic deleted the zm/improve-sdk-ux branch January 2, 2023 17:16
mikhail-treskin pushed a commit to retailnext/cvat that referenced this pull request Jul 1, 2023
Extracted from cvat-ai#5083

- Added a default arg for task data uploading
- Added an option to wait for the data processing in task data uploading
- Moved data splitting by requests for TUS closer to the point of use
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants