Skip to content

Conversation

@SomeoneSerge
Copy link
Collaborator

@SomeoneSerge SomeoneSerge commented Jul 30, 2024

TODO:

  • Verify the closure sizes haven't increased

  • I have read the contributing guidelines

  • Self-reported review complexity:

    • Low
    • Medium
    • High

@github-actions github-actions bot added nix Issues specific to consuming flake.nix, or generally concerned with ❄ Nix-based llama.cpp deployment devops improvements to build systems and github actions labels Jul 30, 2024
Listing individual outputs no longer necessary to reduce
the runtime closure size after NixOS/nixpkgs#323056
@SomeoneSerge SomeoneSerge force-pushed the cuda-propagatedBuildInputs branch from d755dba to 5a618f0 Compare July 30, 2024 13:20
@SomeoneSerge SomeoneSerge marked this pull request as ready for review July 30, 2024 13:21
@SomeoneSerge SomeoneSerge requested a review from philiptaron July 30, 2024 13:21
Copy link
Collaborator

@philiptaron philiptaron left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here's how I tried to find out closure sizes.

$ nix derivation show github:someoneserge/llama.cpp/cuda-propagatedBuildInputs#cuda | jq -r keys[]
/nix/store/vxnjq87z41h3ckilplmgbr7waqf530dl-llama-cpp-cuda-0.0.0.drv
$ nix-build /nix/store/vxnjq87z41h3ckilplmgbr7waqf530dl-llama-cpp-cuda-0.0.0.drv
...
$ nix path-info --closure-size --human-readable /nix/store/vxnjq87z41h3ckilplmgbr7waqf530dl-llama-cpp-cuda-0.0.0.drv^*
/nix/store/mvw7b379xpx31c9hf60ypygv5jpmc9dd-llama-cpp-cuda-0.0.0	   1.0 GiB

So the size for the new version is ~1.0 GiB.

$ nix derivation show github:ggerganov/llama.cpp#cuda | jq -r keys[]
/nix/store/hh7x8cpmdjwj18bgh5qim5kvigpcvvdg-llama-cpp-cuda-0.0.0.drv
$ nix-build /nix/store/hh7x8cpmdjwj18bgh5qim5kvigpcvvdg-llama-cpp-cuda-0.0.0.drv
...
$ nix path-info --closure-size --human-readable /nix/store/hh7x8cpmdjwj18bgh5qim5kvigpcvvdg-llama-cpp-cuda-0.0.0.drv^*
/nix/store/6b1ag2jh7zl5dj0q4a4f6ks4zhx634ih-llama-cpp-cuda-0.0.0	   1.1 GiB

According to this method, the new version is slightly smaller than the old, and everything still builds.

Accepted. ✅

@philiptaron philiptaron merged commit 268c566 into ggml-org:master Jul 30, 2024
arthw pushed a commit to arthw/llama.cpp that referenced this pull request Aug 2, 2024
Listing individual outputs no longer necessary to reduce the runtime closure size after NixOS/nixpkgs#323056.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

devops improvements to build systems and github actions nix Issues specific to consuming flake.nix, or generally concerned with ❄ Nix-based llama.cpp deployment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants