New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Feature Request] baracuda able to run GPT4ALL4bit and AlpacaLora4bit #322

Open

elephantpanda opened this issue Mar 30, 2023 · 0 comments

elephantpanda commented Mar 30, 2023 •

edited

Loading

Will Baracuda be able to run this model? Or similar models:

https://huggingface.co/Sosaka/GPT4All-7B-4bit-ggml

or

https://github.com/johnsmith0031/alpaca_lora_4bit

This would be excellent if it could. This is the way AI is heading with these small optimised models.

I have been able to run 4bit Llama model on Quadro P5000 GPU (similar to GeForce 1080) using pytorch.

elephantpanda changed the title ~~[Feature Request]~~ [Feature Request] baracuda able to run GPT4ALL4bit and AlpacaLora4bit

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment