Example.mov
SwiftyMLC is an example of how to integrate MLC into a Swift project. See the mlc Swift SDK documentation for more information.
- Clone the project
- Recursively initialize the mlc submodule
git submodule update --init --recursive
- Point to an iOS device (a limitation of MLC is that it can only be run on device)
- Run!
- Set scheme to
releasefor improved performance (at the downside of reduced debug ability). - The
Increased Memory Limitcapability has also been enabled.
Ensure you have the following dependencies installed:
- MLC LLM Python Package
- CMake >= 3.24,
- Git and Git-LFS,
- Rust and Cargo, which are required by Hugging Face’s tokenizer.
Add the new model to mlc-package-config.json taking a look at the existing models.
Notice that name, bytes and group aren't part of the original mlc-package-config.json spec.
These are properties used for the ModelsScreen
You can retrieve the bytes of a model using the Chrome HF Model Size extension.
Run the following in the project directory.
export MLC_LLM_SOURCE_DIR={path to this local repository}/mlc-llm
mlc_llm package
./model-details.sh
Model details maps name and bytes (custom properties) to mlc-app-config.json
If you ever change the file system structure (i.e. rename ABC to CBA), you may need to execute the above with
export MLC_DOWNLOAD_CACHE_POLICY=REDO
Also if you get an error you could try deleting the build folder.