[Onnx clip]Adding the clip_onnx to our avaible models for faster inference #245
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
pandu-k
requested changes
Dec 20, 2022
There was a problem hiding this comment.
The main piece of feedback is that onnx_clip_utils seems to be an outdated copy of clip_utils. Can you make ONNX_CLIP a subclass of CLIP? Having two copies of the same code isn't very DRY - unless you want to decouple the APIs for these two types of models.
We can already see issues with copying the code: clip_utils has already had an update with better error message, which hasn't propagated to your code: https://github.com/marqo-ai/marqo/blob/mainline/src/marqo/s2_inference/clip_utils.py#L62
wanliAlex
changed the title
|
Yes, the end to end indexing and searching are tested on a 50k image dataset. |
pandu-k
approved these changes
Dec 29, 2022
pandu-k
temporarily deployed
to
marqo-test-suite
GitHub Actions
Inactive
pandu-k
temporarily deployed
to
marqo-test-suite
GitHub Actions
Inactive
pandu-k
temporarily deployed
to
marqo-test-suite
GitHub Actions
Inactive
pandu-k
temporarily deployed
to
marqo-test-suite
GitHub Actions
Inactive
pandu-k
temporarily deployed
to
marqo-test-suite
GitHub Actions
Inactive
pandu-k
had a problem deploying
to
marqo-test-suite
GitHub Actions
Failure
pandu-k
had a problem deploying
to
marqo-test-suite
GitHub Actions
Error
pandu-k
had a problem deploying
to
marqo-test-suite
GitHub Actions
Failure
pandu-k
temporarily deployed
to
marqo-test-suite
GitHub Actions
Inactive
pandu-k
temporarily deployed
to
marqo-test-suite
GitHub Actions
Inactive
pandu-k
temporarily deployed
to
marqo-test-suite
GitHub Actions
Inactive
pandu-k
temporarily deployed
to
marqo-test-suite
GitHub Actions
Inactive
pandu-k
temporarily deployed
to
marqo-test-suite
GitHub Actions
Inactive
wanliAlex
added a commit
that referenced
this pull request
Dec 29, 2022
…rence (#245) * onnx32/openai/ViT-L/14 * onnx32/openai/ViT-L/14 * onnx32/openai/ViT-L/14 * onnx32/openai/ViT-L/14 * add a timer * add a timer * add a timer * add a timer * add a timer * add a timer * add a timer * add a timer * add a timer * add a timer * add a timer * add a timer * add a timer * add a timer * add a timer * cleaning * add test for onnx_clip * make sure onnx-16 model still use float32 for textual inference for best accuracy. * make sure onnx-16 model still use float32 for textual inference for best accuracy. * we merge the hf models. * update id to _device_id
pandu-k
pushed a commit
that referenced
this pull request
Jan 6, 2023
* add APIs in marqo * test new model cache key * cleaning * Add todo * add multi-gpu support * add multi-gpu support * space adding * adding cpu usage, RAM usage api * adding cpu usage, RAM usage api * adding cpu usage, RAM usage api * adding cpu usage, RAM usage api * revert back model cache key * revert back model cache key * add test_eject_model test * add test_eject_model test * add test_eject_model test * add test_eject_model test * add test_eject_model test * add test_eject_model test * add test_eject_model test * add test_eject_model test * add test_eject_model test * add test_eject_model test * add test_eject_model test * add test_eject_model test * add test_eject_model test * add test_eject_model test * add test_eject_model test * add unit test * add unit test * add unit test * add unit test * add unit test * add unit test * add unit test * test cuda only when cuda is available * format update * format update * format update * test_edge_case_cpu fix (remove cuda memory test) * add separators for readable information in model cache key * add separators for readable information in model cache key * add separators for readable information in model cache key * add separators for readable information in model cache key * add separators for readable information in model cache key * adding test * adding test * adding test * adding test * adding test * adding test * update id to _device_id * [Onnx clip]Adding the clip_onnx to our avaible models for faster inference (#245) * onnx32/openai/ViT-L/14 * onnx32/openai/ViT-L/14 * onnx32/openai/ViT-L/14 * onnx32/openai/ViT-L/14 * add a timer * add a timer * add a timer * add a timer * add a timer * add a timer * add a timer * add a timer * add a timer * add a timer * add a timer * add a timer * add a timer * add a timer * add a timer * cleaning * add test for onnx_clip * make sure onnx-16 model still use float32 for textual inference for best accuracy. * make sure onnx-16 model still use float32 for textual inference for best accuracy. * we merge the hf models. * update id to _device_id * mainline merge * mainline merge * mainline merge * reduce a model for testing stability * reduce a model for testing stability * update * update * update * add test for generic model * add test for generic model * add test for generic model * add test for generic model * add test for generic model * add test for generic model * add test for generic model * revision * revision
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What kind of change does this PR introduce? (Bug fix, feature, docs update, ...)
features
What is the current behavior? (You can also link to an open issue here)
no onnx model for clip
What is the new behavior (if this is a feature change)?
we have onnx model for clip model
Does this PR introduce a breaking change? (What changes might users need to make in their application due to this PR?)
no
Have unit tests been run against this PR? (Has there also been any additional testing?)
not yet
Related Python client changes (link commit/PR here)
no
Related documentation changes (link commit/PR here)
not yet
Other information:
the model passes the performance test on a 50k image dataset
the detailed performance can be checked at link
check this site for available models on huggingface
check this repo for onnx model generation
Please check if the PR fulfills these requirements