You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
It is exciting to see a inference framework with such outstanding performance. My question is whether it can run on mobile devices like Llama.cpp LLM and perform even better? As far as I know, currently only Llama.cpp and mlc-llm can perform inference on mobile devices, with the former only capable of simple operation and the latter requiring cumbersome preprocessing of the model. If BitNet can address these issues, it would be a significant advantage.
reacted with thumbs up emoji reacted with thumbs down emoji reacted with laugh emoji reacted with hooray emoji reacted with confused emoji reacted with heart emoji reacted with rocket emoji reacted with eyes emoji
-
It is exciting to see a inference framework with such outstanding performance. My question is whether it can run on mobile devices like
Llama.cpp
LLM and perform even better? As far as I know, currently onlyLlama.cpp
andmlc-llm
can perform inference on mobile devices, with the former only capable of simple operation and the latter requiring cumbersome preprocessing of the model. If BitNet can address these issues, it would be a significant advantage.Beta Was this translation helpful? Give feedback.
All reactions