epic: Implement new Model Folder and model.yaml

[dan-menlo]

Goal

• We should have a model folder that is able to handle different models
  • Built-in models (e.g. janhq/llama3:7b-tensorrt-llm)
  • Huggingface GGUF repos with multiple quants (e.g. bartowski/llama3-gguf)
  • Huggingface specific GGUF (may have multiple from same directory)
  • In future: Nvidia NGC or TensorRT Cloud
• Do we use sub-folders?
• How does model.yaml work?
• Model detection should not depend on model folder

Tasklist

• feat: cortex run with new model folder structure #1300
• feat: models delete with new model folder structure #1320
• feat: update data folder path #1121
• epic: SQLite database implementation #1241

Decisions

• Cortex.cpp: Model Folder #1113
• Cortex.cpp: model.yaml Format #1123
• Cortex.cpp: Built-in model library format #1178
• Legacy model folder structure: Discussion: Jan Data Structures jan#3541 (comment)

Bugs

• bug: cortex pull invalid_url creates a model folder #1270
• bug(mac): cortexcpp folder installs with wrong permissions #1274

Edge Cases

• What if we download multiple GGUFs from the same Huggingface repo?
  • Saved in the same repo folder, with multiple .gguf files (see feat: models delete with new model folder structure #1320 (comment))
• How does cortex model update <model> work?
  • feat: update data folder path #1121

[freelerobot]

Legacy model folder structure: janhq/jan#3541 (comment)