Skip to content

Conversation

mudler
Copy link
Owner

@mudler mudler commented Aug 22, 2025

Description

This PR adds mlx-vlm as backend.

It is blocked by: Blaizzy/mlx-vlm#470
Extracted from: #6049 , leaving here for reference when it's unblocked upstream.

Notes for Reviewers

Once the PR is merged, install the mlx-vlm-development backend, and create a model file like the following:

mudler@mudlers-Mac-mini LocalAI % cat models/mlx-gemma.yaml
name: mlx-gemma
backend: mlx-vlm
parameters:
  model: "mlx-community/gemma-3n-E2B-it-4bit"
template:
  use_tokenizer_template: true
known_usecases:
- chat

Signed commits

  • Yes, I signed my commits.

Copy link

netlify bot commented Aug 22, 2025

Deploy Preview for localai ready!

Name Link
🔨 Latest commit dd3cee3
🔍 Latest deploy log https://app.netlify.com/projects/localai/deploys/68aa2d0fffa1c00008e748b6
😎 Deploy Preview https://deploy-preview-6119--localai.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

@mudler mudler marked this pull request as draft August 22, 2025 06:44
@mudler mudler added the roadmap label Aug 22, 2025
@mudler mudler mentioned this pull request Aug 23, 2025
1 task
mudler added 2 commits August 23, 2025 22:38
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
@mudler mudler marked this pull request as ready for review August 23, 2025 20:47
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
@mudler mudler merged commit 6971f71 into master Aug 23, 2025
21 of 24 checks passed
@mudler mudler deleted the feat/mlx-vlm branch August 23, 2025 21:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant