Precompiled wheels

[ParisNeo]

Hi,

Is there a way to have multiple precompiled wheels for this library?
I use it on multiple platforms and it fails to install so often because of build errors

Best regards

[gaby]

@ParisNeo I don't think there's a way, llama-cpp optimizes the code during build-time based on your CPU.

[abetlen]

@ParisNeo there are pre-built wheels that I originally set up for the textgeneration-webui guys to simplify their installation for windows users, those are in the releases. The issue is that as @gaby points out, without compiling on the host system llama.cpp will be slow because there's no way to map wheels to the optimizations used in llama.cpp.

One solution we're working on right now is docker images that build on startup (with optional caching), does that work for you?

[gjmulder]

Related to #243

[gaby]

@abetlen Is this done?

[ParisNeo]

Hi. I am sorry for being so late, this was lost ins my old issues. Hhhh
I really want to have a place where I can get latest stable versions of the libraries. I keep getting problems every time a new version is out. It is giving me a heart attack whenever I hear about a new version. Lol. Yesterday spent 6 hours trying to make it work and when I thought it did, I found out that it works on windows but not linux. x-)