Shared library with base name 'llama' not found, windows

[Mohamedballouch]

can someone advise me on this issue, windows
from .llama_cpp import * File "C:\Users\usr1\Anaconda3\envs\chatgpt1\lib\site-packages\llama_cpp\llama_cpp.py", line 46, in
_lib = _load_shared_library(_lib_base_name)
File "C:\Users\moham\Anaconda3\envs\chatgpt1\lib\site-packages\llama_cpp\llama_cpp.py", line 40, in _load_shared_library
raise FileNotFoundError(f"Shared library with base name '{lib_base_name}' not found")
FileNotFoundError: Shared library with base name 'llama' not found

[abetlen]

Hey @Mohamedballouch can you let me know the contents of the C:\Users\usr1\Anaconda3\envs\chatgpt1\lib\site-packages\llama_cpp folder, are there any .dll files located there?

[Mohamedballouch]

Hey man @abetlen , no dll in the folder

[abetlen]

So this seems to be an issue with anaconda, I'll have to investigate, if it's possible I'd recommend using a different package manager (pip or poetry should both work).

Alternatively you can build the .dll and copy it into the folder above

# build this repo
git clone https://github.com/ggerganov/llama.cpp
cd llama.cpp
mkdir build
cd build
cmake ..
cmake --build . --config Release -DBUILD_SHARED_LIBS

[Mohamedballouch]

Thanks man
i did the build of .dll but the last command didn't work with me

[viktor02]

So this seems to be an issue with anaconda, I'll have to investigate, if it's possible I'd recommend using a different package manager (pip or poetry should both work).

Alternatively you can build the .dll and copy it into the folder above

# build this repo
git clone https://github.com/ggerganov/llama.cpp
cd llama.cpp
mkdir build
cd build
cmake ..
cmake --build . --config Release -DBUILD_SHARED_LIBS

Same on pip installation

[viktor02]

I manage to build a llama.dll, but it broken

[lolxdmainkaisemaanlu]

Facing the same issue on windows, I tried on WSL and it moves past this error but the process gets killed because I don't have enough RAM for WSL + 13B llama... Would appreciate it if there was a way to get this to work on native windows... Thanks.

[abetlen]

Thanks man i did the build of .dll but the last command didn't work with me

Sorry, I think the -DBUILD_SHARED_LIBS is on the previous cmake command, try

# build this repo
git clone https://github.com/ggerganov/llama.cpp
cd llama.cpp
mkdir build
cd build
cmake ..  -DBUILD_SHARED_LIBS
cmake --build . --config Release

@viktor02 can you try that as well, that might be why the function isn't being linked in properly.

[lolxdmainkaisemaanlu]

Thanks man i did the build of .dll but the last command didn't work with me

Sorry, I think the -DBUILD_SHARED_LIBS is on the previous cmake command, try

# build this repo
git clone https://github.com/ggerganov/llama.cpp
cd llama.cpp
mkdir build
cd build
cmake ..  -DBUILD_SHARED_LIBS
cmake --build . --config Release

@viktor02 can you try that as well, that might be why the function isn't being linked in properly.

Your suggestion worked for me but instead of cmake .. -DBUILD_SHARED_LIBS the correct command was cmake .. -DBUILD_SHARED_LIBS=ON for people like me who are unfamiliar with cmake.

[viktor02]

Thanks man i did the build of .dll but the last command didn't work with me

Sorry, I think the -DBUILD_SHARED_LIBS is on the previous cmake command, try

# build this repo
git clone https://github.com/ggerganov/llama.cpp
cd llama.cpp
mkdir build
cd build
cmake ..  -DBUILD_SHARED_LIBS
cmake --build . --config Release

@viktor02 can you try that as well, that might be why the function isn't being linked in properly.

Thank you, it's working

[MillionthOdin16]

I just went through same process that people are having issues with while getting oobabooga/text-generation-webui setup and noticed llama_cpp is checking the package's folder for the llama.dll. It's happy once I manually place it there, but otherwise can't find it.

It looks like the pip install process generates the llama.dll, but places it within the bin folder at the root of the virtual environment, which isn't being checked when llama_cpp is run.

[abetlen]

Yeah if you check the github actions tab I'm currently struggling through this one too lol. Checking the cmake logs it looks like it gets built but not placed in the same place as other platforms?

But I'd assume this line should handle that, no?
https://github.com/abetlen/llama-cpp-python/blob/main/CMakeLists.txt#L9

[MillionthOdin16]

@abetlen
Figured it out. You are right except for windows being weird.

install(TARGETS llama
        LIBRARY DESTINATION llama_cpp
        RUNTIME DESTINATION llama_cpp
)

By adding the runtime destination to the cmake config it correctly places the dll on windows. On Windows, shared libraries are typically installed in the bin directory, whereas on Unix-like systems, they are installed in the lib directory. So that's why it ended up in bin.

[abetlen]

I nearly cried seeing this, thank you!

[MillionthOdin16]

Dude, for real. I can't imagine trying to troubleshoot a weird python/windows discrepancy from workflows 💀

Okay, so now it can find the lib, but I don't know the best way to streamline the build process. A lot of people are going to have this auto-configured environment, and it has lots of packages, so we might be able to follow one of those packages approaches for setting up builds. https://github.com/oobabooga/one-click-installers/blob/oobabooga-windows/install.bat

I'm not really familiar with getting build environments setup from python.

[Brandeyy]

I am getting a similar error. The .dll file is present in der correct directory though. Havent really found any info on why this could be happening.