.Net 8 is broken

[chuckbeasley]

Last night, I spent quite a few hours trying to get the demo projects working against .Net 8 and a NVidia GeForce RTX 4070 using the GPU NuGet package. The instructions do NOT work. I had to download and install CUDA 12.x and other items. I wrote up how to get it running here. I don't know what it'll take to get this fixed in the main branch, but it needs attention. The DLLs supplied don't work.

Chuck B.

[martindevans]

Was this using the main branch, or the nuget package with the CUDA backend package install?

At the moment installing the CUDA toolkit is expected, but you shouldn't need to install anything else (like cudart or cublas).

There was some discussion in #350 about removing the requirement to install the cuda toolkit. We decided in the end that we'd like to distribute nuget packages with the dependencies (see this comment), that way we can depend on those in the CUDA backend nuget packages. No one has done that work yet though, if you're interested that'd be a very welcome PR!

[chuckbeasley]

Initially, I used only VS 2022 and NuGet packages, LlamaSharp and LlamaSharp.Backend.Gpu. Since that failed, I started doing research. I found issues #371 and #350. Nothing I tried to resolve the issue worked until hours later I installed the dlls from the llama.cpp release links I referenced in my blog post.