Open
Description
Is your feature request related to a problem? Please describe.
There are currently far too many models which cannot be loaded and run by LocalAI and performance/compatibility optimizations available by adding forks.
Describe the solution you'd like
Review and potentially add runners for projects like:
- https://github.com/ikawrakow/ik_llama.cpp
- https://github.com/EricLBuehler/mistral.rs
- https://github.com/evilsocket/cake
- ...
to homologate execution models for system+model nuances under the cover of "front-end."