Skip to content

Expanded model back-ends #5447

Open
Open
@sempervictus

Description

@sempervictus

Is your feature request related to a problem? Please describe.
There are currently far too many models which cannot be loaded and run by LocalAI and performance/compatibility optimizations available by adding forks.

Describe the solution you'd like
Review and potentially add runners for projects like:

  1. https://github.com/ikawrakow/ik_llama.cpp
  2. https://github.com/EricLBuehler/mistral.rs
  3. https://github.com/evilsocket/cake
  4. ...
    to homologate execution models for system+model nuances under the cover of "front-end."

Metadata

Metadata

Assignees

No one assigned

    Labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions