Fix logging in the TransT serverless function #6290
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
SpecLad
force-pushed
the
transt-conda
branch
2 times, most recently
from
8cacc2d
to
66b1ea4
Compare
|
I've tested this patch on GPU, it works fine for me. |
The Docker image for this function includes Conda, and uses `conda run` to run the Nuclio processor. Unfortunately, `conda run` buffers the entire output of the child process until that process exits, and since the processor never exits, its logs are never printed (and slowly consume memory). Conda is actually completely useless here, so just get rid of it, which fixes the problem. In addition, apply a few other improvements: * Synchronize the PyTorch and TransT versions between the CPU and GPU variants. * Replace `opencv-python` with `opencv-python-headless`, which has fewer dependencies. * Use the `ADD` command instead of wget. Altogether, these improvements shave ~780 MB off the size of the CPU image (didn't check for the GPU one).
Codecov Report
@@ Coverage Diff @@
## develop #6290 +/- ##
===========================================
+ Coverage 73.14% 78.50% +5.36%
===========================================
Files 320 320
Lines 37249 37249
Branches 6748 6748
===========================================
+ Hits 27244 29243 +1999
+ Misses 9725 7725 -2000
- Partials 280 281 +1
|
sizov-kirill
approved these changes
Jun 14, 2023
Merged
bsekachev
added a commit
that referenced
this pull request
Jun 16, 2023
## \[2.4.7] - 2023-06-16 ### Added - \[API\] API Now supports the creation and removal of Ground Truth jobs. (<#6204>) - \[API\] We've introduced task quality estimation endpoints. (<#6204>) - \[CLI\] An option to select the organization. (<#6317>) ### Fixed - Issues with running serverless models for EXIF-rotated images. (<#6275>) - File uploading issues when using https configuration. (<#6308>) - Dataset export error with `outside` property of tracks. (<#5971>) - Broken logging in the TransT serverless function. (<#6290>) --------- Co-authored-by: Anastasia Yasakova <yasakova_anastasiya@mail.ru> Co-authored-by: Mariia Acoca <39969264+mdacoca@users.noreply.github.com> Co-authored-by: Maxim Zhiltsov <zhiltsov.max35@gmail.com> Co-authored-by: klakhov <kirill.9992@gmail.com> Co-authored-by: Roman Donchenko <roman@cvat.ai> Co-authored-by: Kirill Sizov <kirill.sizov@cvat.ai> Co-authored-by: Boris Sekachev <boris.sekachev@yandex.ru> Co-authored-by: Paweł Kotiuk <kotiuk@zohomail.eu> Co-authored-by: Nikita Manovich <nikita@cvat.ai>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Motivation and context
The Docker image for this function includes Conda, and uses
conda runto run the Nuclio processor. Unfortunately,conda runbuffers the entire output of the child process until that process exits, and since the processor never exits, its logs are never printed (and slowly consume memory).Conda is actually completely useless here, so just get rid of it, which fixes the problem.
In addition, apply a few other improvements:
Synchronize the PyTorch and TransT versions between the CPU and GPU variants.
Replace
opencv-pythonwithopencv-python-headless, which has fewer dependencies.Use the
ADDcommand instead of wget.Altogether, these improvements shave ~780 MB off the size of the CPU image (didn't check for the GPU one).
How has this been tested?
Checklist
developbranch[ ] I have updated the documentation accordingly[ ] I have added tests to cover my changes[ ] I have linked related issues (see GitHub docs)[ ] I have increased versions of npm packages if it is necessary(cvat-canvas,
cvat-core,
cvat-data and
cvat-ui)
License
Feel free to contact the maintainers if that's a concern.