Skip to content

Conversation

@CISC
Copy link
Collaborator

@CISC CISC commented Dec 7, 2025

Add CUDA 13.1 release.

Successful run: https://github.com/CISC/llama.cpp/actions/runs/19998436189/job/57349842014
Release: https://github.com/CISC/llama.cpp/releases/tag/b7313

Note

Interestingly the ggml-cuda.dll is a fraction of the size of 12.4 release, not entirely sure why, haven't tested it either.

@github-actions github-actions bot added the devops improvements to build systems and github actions label Dec 7, 2025
@am17an
Copy link
Collaborator

am17an commented Dec 7, 2025

I think it is because they drop support for older architectures

@CISC
Copy link
Collaborator Author

CISC commented Dec 7, 2025

Can anyone confirm the release binaries work?

@fuutott
Copy link

fuutott commented Dec 7, 2025

D:\llama>d:/llama/latest/llama-bench.exe   -m d:\models\lmstudio-community\gpt-oss-120b-GGUF\gpt-oss-120b-MXFP4-00001-of-00002.gguf   -p 512   -n 512   -b 1024   -ub 512   -ngl 99  -mmp 0   -fa 1   -o md   -r 3   -d 0
ggml_cuda_init: GGML_CUDA_FORCE_MMQ:    no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 CUDA devices:
  Device 0: NVIDIA RTX PRO 6000 Blackwell Workstation Edition, compute capability 12.0, VMM: yes
load_backend: loaded CUDA backend from d:\llama\latest\ggml-cuda.dll
load_backend: loaded RPC backend from d:\llama\latest\ggml-rpc.dll
load_backend: loaded CPU backend from d:\llama\latest\ggml-cpu-sapphirerapids.dll
| model                          |       size |     params | backend    | ngl | n_batch | fa | mmap |            test |                  t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | ------: | -: | ---: | --------------: | -------------------: |
| gpt-oss 120B MXFP4 MoE         |  59.02 GiB |   116.83 B | CUDA       |  99 |    1024 |  1 |    0 |           pp512 |     4718.55 ± 125.71 |
| gpt-oss 120B MXFP4 MoE         |  59.02 GiB |   116.83 B | CUDA       |  99 |    1024 |  1 |    0 |           tg512 |        237.14 ± 1.22 |

build: 80acf82d (7313)

@CISC CISC merged commit 0a540f9 into master Dec 7, 2025
17 checks passed
@CISC CISC deleted the cisc/ci-release-cuda-13 branch December 7, 2025 13:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

devops improvements to build systems and github actions

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants