Question about LLamaSharp's Dependency on llama.cpp Updates

[sangyuxiaowu]

Hello everyone,

I've noticed that the underlying llama.cpp dependency for the LLamaSharp project updates very frequently, especially when new large models are released. llama.cpp provides timely support in such instances. Similar projects like ollama also synchronize their updates to allow users to experience the new models.

As a C# wrapper for llama.cpp, does the version of llama.cpp that LLamaSharp depends on have a strong correlation with the LLamaSharp-related code? Is it possible to provide automated pre-releases based on the daily updated versions of llama.cpp, without affecting the current official release schedule?

This question arose because Google released the new Gemma 2 model last night, which requires the latest version of llama.cpp for support, but the LLamaSharp version is already three weeks old.

Thank you all!

[martindevans]

llama.cpp make no guarantees about c-api compatibility from version to version, so unfortunately there is some manual work required for every update. I aim to do this approximately monthly.

[sangyuxiaowu]

Thank you for your response and your hard work.

I attempted to replace the ggml.dll, llama.dll, and llama_shared.dll from the CUDA 11 package released by llama.cpp in my project. Although it works, I did encounter some strange issues. During inference, the GPU utilization is below 20%, while the CPU utilization exceeds 50%. Additionally, there are anomalies with the stop words <end_of_turn> in the inference results. , which seem to be due to emoji issues.