Skip to content

examples : support MiniCPM-V-2 #13828

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 3 commits into
base: master
Choose a base branch
from

Conversation

guoQiNing
Copy link

Hi, this is an attempt to add support for MiniCPM-V-2.

This code is based on the llama.cpp repository at commit 7841fc72, and PR #6919 is a helpful reference.

I also notice the recent updates about libmtmd and mtmd-cli, and I'll continue adapting this code for mtmd compatibility soon, to help its merge into the latest llama.cpp.

Thank you for your consideration.

@github-actions github-actions bot added examples python python script changes labels May 27, 2025
@ngxson
Copy link
Collaborator

ngxson commented May 27, 2025

If you can provide me a concise list of differences between minicpm-v-2 and the original minicpm-v, I can help doing this faster

@ngxson
Copy link
Collaborator

ngxson commented May 27, 2025

But wait, are you sure you are adding support for a new model?

To remind, we already support minicpmv-2.5, 2.6 and minicpm-o, so maybe there is no more work needed

@guoQiNing
Copy link
Author

But wait, are you sure you are adding support for a new model?
To remind, we already support minicpmv-2.5, 2.6 and minicpm-o, so maybe there is no more work needed

Yes, I've noted the existing support for recent MiniCPM-V series models in llama.cpp.

However, MiniCPM-V 2.0 is a capable 2B parameter model, while the subsequent ones (minicpmv-2.5, 2.6 and minicpm-o) are all around 8B. And Its visual model architecture also has significant differences.

When using MiniCPM-V 2.0, I noticed that the related PR #6919 has not been merged and has bug in GPU environments. This PR is an attempt to provide fix and update.

I will try to provide a concise list of differences between MiniCPM-V 2.0 and the original MiniCPM-V soon.

Thank you so much for your reply!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
examples python python script changes
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants