LLMFarm-MiniCPM

中文版

Based on LLMFarm, we run MiniCPM on ios devices. Note that the models run on ios are quantized to 4-bit and may lose some performance. The original models can be found here.

Deploy MiniCPM on IOS

The first method is to directly download our converted model，You can skip the subsequent conversion steps.

The second method is to download the original model from the huggingface and follow the steps below to convert and quantify it.

convert model

1. downloading model
2. git clone https://github.com/OpenBMB/llama.cpp.git
3. cd llama.cpp && make -j8
4. python3 convert.py ${hf_model_dir} --vocab-type hfft --outtype f32
5. ./quantize ${hf_model_dir}/ggml-model-f32.gguf ${output_dir}/minicpm-q4_1.gguf q4_1

compile

1. git clone https://github.com/OpenBMB/LLMFarm-MiniCPM.git
2. cd LLMFarm-MiniCPM && git submodule update --init --recursive
3. Open this project with Xcode
4. Setting Siging & Capabilities
5. Select a device My MaC or your iphone
6. run

chat

1. add chat
2. select a model
3. Set template: CPM
4. Start chat