Fix for issue #876

[burningion]

For the following issue:

#876 (comment)

GPU support won't build without the change from native to any. Confirmed working with a 4090.

[ggerganov]

Do we know that all does not reduce the performance compared to native ?

[byte-6174]

It most certainly would in some sense. For example when I compile with all the binary is 1.5MB vs. 812KB when I give the exact architecture I am compiling for.
The exact architecture is the best way to make sure you have the most optimized code for it, as it does not create the intermediate ptx code. With option all, nvcc would generate the ptx and postpone compilation at runtime. Nvidia notes this would lead to app startup delays. It could be addressed by caching, however.

[byte-6174]

for example I just ran w a binary the load time for a model (models/ggml-base.en.bin) was:

-gpu-architecture=all:
whisper_print_timings: load time = 4452.92 ms

vs
-gpu-architecture=sm_72:
whisper_print_timings: load time = 224.77 ms

[ggerganov]

So, is it worth switching to all?
Maybe have native by default and only use all upon setting certain build option

[byte-6174]

yes, cuda optimizations are highly specific to the hardware. all will allow compilation without errors, but with suboptimal binary.
whereas native would work sometimes based on arch. / cuda version, but when it works will produce the best binary 😄

In that regard, I like the above suggestion. However, I'm reading in other comments though for some arch/cuda combos neither works :( and one has to specify the exact arch like sm_72 etc.
So there is some finetuning of the makefile when it comes to cuda.

[FerLuisxd]

It doesn't work on 2080 and version 12.1
See #1082