-
Notifications
You must be signed in to change notification settings - Fork 9
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
web_demo doesn't work #3
Comments
Hi, for a smooth training and inference, we need to keep a proper version of the |
hi,我尝试在4090上运行web demo,出现cuda out of memory,如何采用多卡来进行web demo的部署? @qijimrc |
I tried to run the gradio demo as follows:
python web_demo.py --from_pretrained cogcom-base-17b --local_tokenizer lmsys/vicuna-7b-v1.5 --bf16
resulting in this error:
The text was updated successfully, but these errors were encountered: