lmstudio_en
LM Studio is a cross-platform (macOS, Windows, Linux) large model chat program capable of loading GGUF format models (from llama.cpp). Below, I'll briefly explain how to use it on macOS as an example. For other uses, please try it yourself and consult the official manual for more information.
Official download link: LM Studio
Create a models directory and organize the model files in a format similar to that used by Hugging Face's model repository. You can copy the models or set up symbolic links. For example, the hfl/llama-3-chinese-8b-instruct model could be organized in the following directory structure, with the GGUF model (in llama.cpp format) in the last layer:
- hfl
- llama-3-chinese-8b-instruct
- ggml-model-q8_0.gguf
-
After opening the program, click on the last folder icon TAB on the left side.
-
In the Local models folder section, click change and point it to the newly created models directory from Step 2.
-
Once set up, all recognized models will be listed (if none are recognized, try restarting the program).
-
Click on “choose default preset” and select the Llama 3 instruction template.
-
Click the third conversation button on the left to enter the chat program.
-
At the top, select a model to load; on the right sidebar, you can set system instructions, enable GPU, context window size, etc.; on the left sidebar, you can start a new conversation.
These steps should help you get started with using LM Studio to run large language models and engage in interactive chat sessions using your model of choice!