make process hangs if LLAMA_CUBLAS=1, at the line that includes the file scripts/get-flags.mk for the second time

[yigber]

Prerequisites

Please answer the following questions for yourself before submitting an issue.

• I am running the latest code. Development is very rapid so there are no tagged versions as of now.
• I carefully followed the README.md.
• I searched using keywords relevant to my issue to make sure that I am creating a new issue that is not already open (or closed).
• I reviewed the Discussions, and have a new bug or useful enhancement to share.

Expected Behavior

I run make LLAMA_CUBLAS=1 and that process hangs. I used make --debug=f to figure out that make gets stuck at the line that includes get-flags.mk for the second time (it is already included a few lines before).

Environment and Context

+---------------------------------------------------------------------------------------+
| NVIDIA-SMI 535.129.03 Driver Version: 535.129.03 CUDA Version: 12.2 |
|-----------------------------------------+----------------------+----------------------+
| GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|=========================================+======================+======================|
| 0 NVIDIA A100-SXM4-80GB On | 00000000:00:05.0 Off | 0 |
| N/A 42C P0 86W / 400W | 591MiB / 81920MiB | 0% Default |
| | | Disabled |
+-----------------------------------------+----------------------+----------------------+

• Physical (or virtual) hardware you are using, e.g. for Linux:

$ lscpu
Architecture: x86_64
CPU op-mode(s): 32-bit, 64-bit
Address sizes: 46 bits physical, 48 bits virtual
Byte Order: Little Endian
CPU(s): 12
On-line CPU(s) list: 0-11
Vendor ID: GenuineIntel
Model name: Intel(R) Xeon(R) Gold 6342 CPU @ 2.80GHz
CPU family: 6
Model: 106
Thread(s) per core: 1
Core(s) per socket: 12
Socket(s): 1
Stepping: 6
BogoMIPS: 5600.21
Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush acpi mmx fxsr sse sse2 s
s ht syscall nx pdpe1gb rdtscp lm constant_tsc rep_good nopl cpuid tsc_known_freq pni pclmulqdq ssse3 fm
a cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor la
hf_lm abm 3dnowprefetch cpuid_fault invpcid_single ssbd ibrs ibpb stibp ibrs_enhanced fsgsbase tsc_adjus
t bmi1 avx2 smep bmi2 erms invpcid rtm rdseed adx smap clflushopt clwb sha_ni xsaveopt xsavec xgetbv1 xs
aves umip pku ospke gfni vaes vpclmulqdq rdpid md_clear flush_l1d arch_capabilities
Virtualization features:
Hypervisor vendor: Xen
Virtualization type: full

Caches (sum of all):
L1d: 576 KiB (12 instances)
L1i: 384 KiB (12 instances)
L2: 15 MiB (12 instances)
L3: 432 MiB (12 instances)
NUMA:
NUMA node(s): 1
NUMA node0 CPU(s): 0-11
Vulnerabilities:
Gather data sampling: Not affected
Itlb multihit: Not affected
L1tf: Not affected
Mds: Not affected
Meltdown: Not affected
Mmio stale data: Mitigation; Clear CPU buffers; SMT Host state unknown
Retbleed: Not affected
Spec rstack overflow: Not affected
Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Spectre v1: Mitigation; usercopy/swapgs barriers and __user pointer sanitization
Spectre v2: Mitigation; Enhanced IBRS, IBPB conditional, RSB filling, PBRSB-eIBRS SW sequence
Srbds: Not affected
Tsx async abort: Not affected

• Operating System, e.g. for Linux:

$ uname -a

Linux psbngks26deu 6.2.0-39-generic #40~22.04.1-Ubuntu SMP PREEMPT_DYNAMIC Thu Nov 16 10:53:04 UTC 2 x86_64 x86_64 x86_64 GNU/Linux

• SDK version, e.g. for Linux:
  Python 3.11.7
  GNU Make 4.3
  Built for x86_64-pc-linux-gnu
  g++ (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0

[dspasyuk]

Try building this:
use: nvcc -o cuda_check cuda_check.c -lcuda
https://www.dropbox.com/scl/fi/1t4qsrka6p0r73vzfd8vv/cuda_check.c?rlkey=wy3m3thyn5c0ydmzb3v97dt07&dl=0

[github-actions]

This issue is stale because it has been open for 30 days with no activity.

[github-actions]

This issue was closed because it has been inactive for 14 days since being marked as stale.