Added a Gradio UI for multi-modal inferencing using Llama 3.2 Vision/ #718
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…ty, image processing and text generation
init27
requested changes
Oct 8, 2024
recipes/quickstart/inference/local_inference/multi_modal_infer_Gradio_UI.py
Outdated
Show resolved
Hide resolved
…sssing hugigng-face token from the arguments
…ature and top_p sliders there.
|
@init27 I did the changes you asked, please check and let me know... I'll be happy to make it better |
Modified readme for new code for passing token via argument
Used small case "g" in gradio
|
@init27 added the changes you asked, please check... |
|
@init27 please let me know if anything required... |
|
@init27 can you check the PR, I renamed the filename too. |
init27
approved these changes
Oct 27, 2024
|
@himanshushukla12 sorry for missing this-looks great! Can you please make sure the CI/CD is green and merge? |
|
@init27 please check, everything is working fine. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What does this PR do?
This PR introduces multi-modal inference using the Gradio UI for Llama 3.2 vision models. The Gradio UI allows users to upload images and generate descriptive text based on a prompt, with adjustable parameters such as top-k, max-tokens, temperature and top-p for fine-tuning text generation. With chatbox like interface.
Additionally, this PR:
Integrates the transformers and accelerate libraries for efficient model loading and inference.
Implements memory management for releasing GPU resources after inference.
Adds support for Hugging Face tokens to authenticate and access Llama models.