Excessive compile times and CUDA binary size due to excessive use of constexpr #13087
CommanderLake
started this conversation in
Ideas
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
Excessive use of
constexpr
is multiplying the compiled code paths and producing a ggml-cuda.dll binary that can now exceed 400MB for little to no performance benefit at the cost of excessive compile times, download times, required storage space and memory just to load the dll!Reduce unnecessary usage of
constexpr
, or better yet add an option to reduceconstexpr
usage to speed up compile times and reduce the compiled binary size.Beta Was this translation helpful? Give feedback.
All reactions