-
Notifications
You must be signed in to change notification settings - Fork 445
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
OTX export for visual prompting task #2274
OTX export for visual prompting task #2274
Conversation
b6d22b6
to
4b22336
Compare
24b6b62
to
469279e
Compare
TODOs - check weights in eval - apply transforms
34b9817
to
6e803a5
Compare
src/otx/algorithms/visual_prompting/configs/base/configuration_enums.py
Outdated
Show resolved
Hide resolved
...thms/visual_prompting/adapters/pytorch_lightning/models/visual_prompters/segment_anything.py
Show resolved
Hide resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I left some comments. Frankly there are many codes that is not fully understood.
I think we should revisit this after FF. Before merging this PR I want to ensure there's no big accuracy drop before and after export. Do you have some data for this? I want to make sure of this since preprocessing and postprocessing of OV model is implemented in this PR, not from ModelAPI.
From this version, exported model showed slight mDice performance drop from 0.9503 to 0.9488. |
Summary
This PR includes:
otx export
otx eval
with OpenVINO IRI followed a flow of
OpenVINOSegmentationTask
forOpenVINOVisualPromptingTask
, so they look so similar.But, for visual prompting task, there are two models, image encoder and decoder.
I think this point is just difference between both tasks.
TODOs
How to test
Unit test (local)
$ tox -vv -e unittest-all-py310 -- tests/unit/algorithms/visual_prompting ====================== 165 passed, 110 warnings in 33.36s ======================
Integration test (local)
$ tox -vv -e tests-all-py310 -- tests/integration/cli/visual_prompting/test_visual_prompting.py =================== 8 passed, 1 warning in 122.65s (0:02:02) ===================
Checklist
License
Feel free to contact the maintainers if that's a concern.