Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Unknown CUDA arch (compute) or GPU not supported error while installing on docker ubuntu with cuda 12.1 #1819

Open
AvisP opened this issue Jul 19, 2024 · 1 comment
Open

Unknown CUDA arch (compute) or GPU not supported error while installing on docker ubuntu with cuda 12.1 #1819

AvisP opened this issue Jul 19, 2024 · 1 comment

Comments

@AvisP
Copy link

AvisP commented Jul 19, 2024

I am trying to install nvida apex on a docker image nvidia/cuda:12.1.0-devel-ubuntu22.04 running on windows 11. wsl2 is installed and I am able to do nvidia-smi and nvcc --version in both docker and wsl. I have a 4080 GTX graphics card. While installing I am getting this error message about ValueError: Unknown CUDA arch (compute) or GPU not supported

Here is the full log if anyone can suggest some ideas

root@8fbff3ef7fc2:/Python-3.10.14/apex# TORCH_CUDA_ARCH_LIST="compute capability" pip install -v --disable-pip-version-check --no-cache-dir --no-build-isolation --config-settings "--build-option=--cpp_ext" --config-settings "--build-option=--cuda_ext" ./
Using pip 24.1.2 from /usr/local/lib/python3.10/site-packages/pip (python 3.10)
Processing /Python-3.10.14/apex
  Running command Preparing metadata (pyproject.toml)


  torch.__version__  = 2.3.1+cu121


  running dist_info
  creating /tmp/pip-modern-metadata-fd1ybqub/apex.egg-info
  writing /tmp/pip-modern-metadata-fd1ybqub/apex.egg-info/PKG-INFO
  writing dependency_links to /tmp/pip-modern-metadata-fd1ybqub/apex.egg-info/dependency_links.txt
  writing requirements to /tmp/pip-modern-metadata-fd1ybqub/apex.egg-info/requires.txt
  writing top-level names to /tmp/pip-modern-metadata-fd1ybqub/apex.egg-info/top_level.txt
  writing manifest file '/tmp/pip-modern-metadata-fd1ybqub/apex.egg-info/SOURCES.txt'
  reading manifest file '/tmp/pip-modern-metadata-fd1ybqub/apex.egg-info/SOURCES.txt'
  adding license file 'LICENSE'
  writing manifest file '/tmp/pip-modern-metadata-fd1ybqub/apex.egg-info/SOURCES.txt'
  creating '/tmp/pip-modern-metadata-fd1ybqub/apex-0.1.dist-info'
  Preparing metadata (pyproject.toml) ... done
Requirement already satisfied: packaging>20.6 in /usr/local/lib/python3.10/site-packages (from apex==0.1) (24.1)
Building wheels for collected packages: apex
  Running command Building wheel for apex (pyproject.toml)


  torch.__version__  = 2.3.1+cu121



  Compiling cuda extensions with
  nvcc: NVIDIA (R) Cuda compiler driver
  Copyright (c) 2005-2023 NVIDIA Corporation
  Built on Mon_Apr__3_17:16:06_PDT_2023
  Cuda compilation tools, release 12.1, V12.1.105
  Build cuda_12.1.r12.1/compiler.32688072_0
  from /usr/local/cuda/bin

  running bdist_wheel
  /usr/local/lib/python3.10/site-packages/torch/utils/cpp_extension.py:499: UserWarning: Attempted to use ninja as the BuildExtension backend but we could not find ninja.. Falling back to using the slow distutils backend.
    warnings.warn(msg.format('we could not find ninja.'))
  running build
  running build_py
  creating build
  creating build/lib.linux-x86_64-cpython-310
  creating build/lib.linux-x86_64-cpython-310/apex
  copying apex/__init__.py -> build/lib.linux-x86_64-cpython-310/apex
  copying apex/_autocast_utils.py -> build/lib.linux-x86_64-cpython-310/apex
  creating build/lib.linux-x86_64-cpython-310/apex/amp
  copying apex/amp/__init__.py -> build/lib.linux-x86_64-cpython-310/apex/amp
  copying apex/amp/rnn_compat.py -> build/lib.linux-x86_64-cpython-310/apex/amp
  copying apex/amp/compat.py -> build/lib.linux-x86_64-cpython-310/apex/amp
  copying apex/amp/frontend.py -> build/lib.linux-x86_64-cpython-310/apex/amp
  copying apex/amp/_initialize.py -> build/lib.linux-x86_64-cpython-310/apex/amp
  copying apex/amp/opt.py -> build/lib.linux-x86_64-cpython-310/apex/amp
  copying apex/amp/_amp_state.py -> build/lib.linux-x86_64-cpython-310/apex/amp
  copying apex/amp/handle.py -> build/lib.linux-x86_64-cpython-310/apex/amp
  copying apex/amp/wrap.py -> build/lib.linux-x86_64-cpython-310/apex/amp
  copying apex/amp/utils.py -> build/lib.linux-x86_64-cpython-310/apex/amp
  copying apex/amp/scaler.py -> build/lib.linux-x86_64-cpython-310/apex/amp
  copying apex/amp/__version__.py -> build/lib.linux-x86_64-cpython-310/apex/amp
  copying apex/amp/_process_optimizer.py -> build/lib.linux-x86_64-cpython-310/apex/amp
  copying apex/amp/amp.py -> build/lib.linux-x86_64-cpython-310/apex/amp
  creating build/lib.linux-x86_64-cpython-310/apex/optimizers
  copying apex/optimizers/__init__.py -> build/lib.linux-x86_64-cpython-310/apex/optimizers
  copying apex/optimizers/fused_adam.py -> build/lib.linux-x86_64-cpython-310/apex/optimizers
  copying apex/optimizers/fused_sgd.py -> build/lib.linux-x86_64-cpython-310/apex/optimizers
  copying apex/optimizers/fused_lamb.py -> build/lib.linux-x86_64-cpython-310/apex/optimizers
  copying apex/optimizers/fused_novograd.py -> build/lib.linux-x86_64-cpython-310/apex/optimizers
  copying apex/optimizers/fused_adagrad.py -> build/lib.linux-x86_64-cpython-310/apex/optimizers
  copying apex/optimizers/fused_mixed_precision_lamb.py -> build/lib.linux-x86_64-cpython-310/apex/optimizers
  creating build/lib.linux-x86_64-cpython-310/apex/normalization
  copying apex/normalization/__init__.py -> build/lib.linux-x86_64-cpython-310/apex/normalization
  copying apex/normalization/fused_layer_norm.py -> build/lib.linux-x86_64-cpython-310/apex/normalization
  creating build/lib.linux-x86_64-cpython-310/apex/fused_dense
  copying apex/fused_dense/__init__.py -> build/lib.linux-x86_64-cpython-310/apex/fused_dense
  copying apex/fused_dense/fused_dense.py -> build/lib.linux-x86_64-cpython-310/apex/fused_dense
  creating build/lib.linux-x86_64-cpython-310/apex/multi_tensor_apply
  copying apex/multi_tensor_apply/__init__.py -> build/lib.linux-x86_64-cpython-310/apex/multi_tensor_apply
  copying apex/multi_tensor_apply/multi_tensor_apply.py -> build/lib.linux-x86_64-cpython-310/apex/multi_tensor_apply
  creating build/lib.linux-x86_64-cpython-310/apex/RNN
  copying apex/RNN/__init__.py -> build/lib.linux-x86_64-cpython-310/apex/RNN
  copying apex/RNN/models.py -> build/lib.linux-x86_64-cpython-310/apex/RNN
  copying apex/RNN/cells.py -> build/lib.linux-x86_64-cpython-310/apex/RNN
  copying apex/RNN/RNNBackend.py -> build/lib.linux-x86_64-cpython-310/apex/RNN
  creating build/lib.linux-x86_64-cpython-310/apex/transformer
  copying apex/transformer/__init__.py -> build/lib.linux-x86_64-cpython-310/apex/transformer
  copying apex/transformer/log_util.py -> build/lib.linux-x86_64-cpython-310/apex/transformer
  copying apex/transformer/microbatches.py -> build/lib.linux-x86_64-cpython-310/apex/transformer
  copying apex/transformer/parallel_state.py -> build/lib.linux-x86_64-cpython-310/apex/transformer
  copying apex/transformer/enums.py -> build/lib.linux-x86_64-cpython-310/apex/transformer
  copying apex/transformer/utils.py -> build/lib.linux-x86_64-cpython-310/apex/transformer
  copying apex/transformer/_ucc_util.py -> build/lib.linux-x86_64-cpython-310/apex/transformer
  creating build/lib.linux-x86_64-cpython-310/apex/mlp
  copying apex/mlp/__init__.py -> build/lib.linux-x86_64-cpython-310/apex/mlp
  copying apex/mlp/mlp.py -> build/lib.linux-x86_64-cpython-310/apex/mlp
  creating build/lib.linux-x86_64-cpython-310/apex/parallel
  copying apex/parallel/optimized_sync_batchnorm_kernel.py -> build/lib.linux-x86_64-cpython-310/apex/parallel
  copying apex/parallel/__init__.py -> build/lib.linux-x86_64-cpython-310/apex/parallel
  copying apex/parallel/distributed.py -> build/lib.linux-x86_64-cpython-310/apex/parallel
  copying apex/parallel/multiproc.py -> build/lib.linux-x86_64-cpython-310/apex/parallel
  copying apex/parallel/LARC.py -> build/lib.linux-x86_64-cpython-310/apex/parallel
  copying apex/parallel/sync_batchnorm_kernel.py -> build/lib.linux-x86_64-cpython-310/apex/parallel
  copying apex/parallel/sync_batchnorm.py -> build/lib.linux-x86_64-cpython-310/apex/parallel
  copying apex/parallel/optimized_sync_batchnorm.py -> build/lib.linux-x86_64-cpython-310/apex/parallel
  creating build/lib.linux-x86_64-cpython-310/apex/contrib
  copying apex/contrib/__init__.py -> build/lib.linux-x86_64-cpython-310/apex/contrib
  creating build/lib.linux-x86_64-cpython-310/apex/fp16_utils
  copying apex/fp16_utils/__init__.py -> build/lib.linux-x86_64-cpython-310/apex/fp16_utils
  copying apex/fp16_utils/fp16_optimizer.py -> build/lib.linux-x86_64-cpython-310/apex/fp16_utils
  copying apex/fp16_utils/fp16util.py -> build/lib.linux-x86_64-cpython-310/apex/fp16_utils
  copying apex/fp16_utils/loss_scaler.py -> build/lib.linux-x86_64-cpython-310/apex/fp16_utils
  creating build/lib.linux-x86_64-cpython-310/apex/amp/lists
  copying apex/amp/lists/tensor_overrides.py -> build/lib.linux-x86_64-cpython-310/apex/amp/lists
  copying apex/amp/lists/__init__.py -> build/lib.linux-x86_64-cpython-310/apex/amp/lists
  copying apex/amp/lists/functional_overrides.py -> build/lib.linux-x86_64-cpython-310/apex/amp/lists
  copying apex/amp/lists/torch_overrides.py -> build/lib.linux-x86_64-cpython-310/apex/amp/lists
  creating build/lib.linux-x86_64-cpython-310/apex/transformer/pipeline_parallel
  copying apex/transformer/pipeline_parallel/_timers.py -> build/lib.linux-x86_64-cpython-310/apex/transformer/pipeline_parallel
  copying apex/transformer/pipeline_parallel/__init__.py -> build/lib.linux-x86_64-cpython-310/apex/transformer/pipeline_parallel
  copying apex/transformer/pipeline_parallel/utils.py -> build/lib.linux-x86_64-cpython-310/apex/transformer/pipeline_parallel
  copying apex/transformer/pipeline_parallel/p2p_communication.py -> build/lib.linux-x86_64-cpython-310/apex/transformer/pipeline_parallel
  creating build/lib.linux-x86_64-cpython-310/apex/transformer/amp
  copying apex/transformer/amp/__init__.py -> build/lib.linux-x86_64-cpython-310/apex/transformer/amp
  copying apex/transformer/amp/grad_scaler.py -> build/lib.linux-x86_64-cpython-310/apex/transformer/amp
  creating build/lib.linux-x86_64-cpython-310/apex/transformer/_data
  copying apex/transformer/_data/__init__.py -> build/lib.linux-x86_64-cpython-310/apex/transformer/_data
  copying apex/transformer/_data/_batchsampler.py -> build/lib.linux-x86_64-cpython-310/apex/transformer/_data
  creating build/lib.linux-x86_64-cpython-310/apex/transformer/functional
  copying apex/transformer/functional/__init__.py -> build/lib.linux-x86_64-cpython-310/apex/transformer/functional
  copying apex/transformer/functional/fused_rope.py -> build/lib.linux-x86_64-cpython-310/apex/transformer/functional
  copying apex/transformer/functional/fused_softmax.py -> build/lib.linux-x86_64-cpython-310/apex/transformer/functional
  creating build/lib.linux-x86_64-cpython-310/apex/transformer/tensor_parallel
  copying apex/transformer/tensor_parallel/__init__.py -> build/lib.linux-x86_64-cpython-310/apex/transformer/tensor_parallel
  copying apex/transformer/tensor_parallel/mappings.py -> build/lib.linux-x86_64-cpython-310/apex/transformer/tensor_parallel
  copying apex/transformer/tensor_parallel/layers.py -> build/lib.linux-x86_64-cpython-310/apex/transformer/tensor_parallel
  copying apex/transformer/tensor_parallel/random.py -> build/lib.linux-x86_64-cpython-310/apex/transformer/tensor_parallel
  copying apex/transformer/tensor_parallel/cross_entropy.py -> build/lib.linux-x86_64-cpython-310/apex/transformer/tensor_parallel
  copying apex/transformer/tensor_parallel/data.py -> build/lib.linux-x86_64-cpython-310/apex/transformer/tensor_parallel  copying apex/transformer/tensor_parallel/memory.py -> build/lib.linux-x86_64-cpython-310/apex/transformer/tensor_parallel
  copying apex/transformer/tensor_parallel/utils.py -> build/lib.linux-x86_64-cpython-310/apex/transformer/tensor_parallel
  creating build/lib.linux-x86_64-cpython-310/apex/transformer/layers
  copying apex/transformer/layers/__init__.py -> build/lib.linux-x86_64-cpython-310/apex/transformer/layers
  copying apex/transformer/layers/layer_norm.py -> build/lib.linux-x86_64-cpython-310/apex/transformer/layers
  creating build/lib.linux-x86_64-cpython-310/apex/transformer/testing
  copying apex/transformer/testing/__init__.py -> build/lib.linux-x86_64-cpython-310/apex/transformer/testing
  copying apex/transformer/testing/distributed_test_base.py -> build/lib.linux-x86_64-cpython-310/apex/transformer/testing
  copying apex/transformer/testing/commons.py -> build/lib.linux-x86_64-cpython-310/apex/transformer/testing
  copying apex/transformer/testing/global_vars.py -> build/lib.linux-x86_64-cpython-310/apex/transformer/testing
  copying apex/transformer/testing/standalone_gpt.py -> build/lib.linux-x86_64-cpython-310/apex/transformer/testing
  copying apex/transformer/testing/standalone_transformer_lm.py -> build/lib.linux-x86_64-cpython-310/apex/transformer/testing
  copying apex/transformer/testing/standalone_bert.py -> build/lib.linux-x86_64-cpython-310/apex/transformer/testing
  copying apex/transformer/testing/arguments.py -> build/lib.linux-x86_64-cpython-310/apex/transformer/testing
  creating build/lib.linux-x86_64-cpython-310/apex/transformer/pipeline_parallel/schedules
  copying apex/transformer/pipeline_parallel/schedules/__init__.py -> build/lib.linux-x86_64-cpython-310/apex/transformer/pipeline_parallel/schedules
  copying apex/transformer/pipeline_parallel/schedules/fwd_bwd_pipelining_without_interleaving.py -> build/lib.linux-x86_64-cpython-310/apex/transformer/pipeline_parallel/schedules
  copying apex/transformer/pipeline_parallel/schedules/fwd_bwd_pipelining_with_interleaving.py -> build/lib.linux-x86_64-cpython-310/apex/transformer/pipeline_parallel/schedules
  copying apex/transformer/pipeline_parallel/schedules/common.py -> build/lib.linux-x86_64-cpython-310/apex/transformer/pipeline_parallel/schedules
  copying apex/transformer/pipeline_parallel/schedules/fwd_bwd_no_pipelining.py -> build/lib.linux-x86_64-cpython-310/apex/transformer/pipeline_parallel/schedules
  creating build/lib.linux-x86_64-cpython-310/apex/contrib/xentropy
  copying apex/contrib/xentropy/__init__.py -> build/lib.linux-x86_64-cpython-310/apex/contrib/xentropy
  copying apex/contrib/xentropy/softmax_xentropy.py -> build/lib.linux-x86_64-cpython-310/apex/contrib/xentropy
  creating build/lib.linux-x86_64-cpython-310/apex/contrib/conv_bias_relu
  copying apex/contrib/conv_bias_relu/__init__.py -> build/lib.linux-x86_64-cpython-310/apex/contrib/conv_bias_relu
  copying apex/contrib/conv_bias_relu/conv_bias_relu.py -> build/lib.linux-x86_64-cpython-310/apex/contrib/conv_bias_relu  creating build/lib.linux-x86_64-cpython-310/apex/contrib/group_norm
  copying apex/contrib/group_norm/__init__.py -> build/lib.linux-x86_64-cpython-310/apex/contrib/group_norm
  copying apex/contrib/group_norm/group_norm.py -> build/lib.linux-x86_64-cpython-310/apex/contrib/group_norm
  creating build/lib.linux-x86_64-cpython-310/apex/contrib/openfold_triton
  copying apex/contrib/openfold_triton/__init__.py -> build/lib.linux-x86_64-cpython-310/apex/contrib/openfold_triton
  copying apex/contrib/openfold_triton/_layer_norm_config_hopper.py -> build/lib.linux-x86_64-cpython-310/apex/contrib/openfold_triton
  copying apex/contrib/openfold_triton/_layer_norm_config_ampere.py -> build/lib.linux-x86_64-cpython-310/apex/contrib/openfold_triton
  copying apex/contrib/openfold_triton/layer_norm.py -> build/lib.linux-x86_64-cpython-310/apex/contrib/openfold_triton
  copying apex/contrib/openfold_triton/_layer_norm_backward_kernels.py -> build/lib.linux-x86_64-cpython-310/apex/contrib/openfold_triton
  copying apex/contrib/openfold_triton/fused_adam_swa.py -> build/lib.linux-x86_64-cpython-310/apex/contrib/openfold_triton
  copying apex/contrib/openfold_triton/_layer_norm_forward_kernels.py -> build/lib.linux-x86_64-cpython-310/apex/contrib/openfold_triton
  copying apex/contrib/openfold_triton/mha.py -> build/lib.linux-x86_64-cpython-310/apex/contrib/openfold_triton
  copying apex/contrib/openfold_triton/_mha_kernel.py -> build/lib.linux-x86_64-cpython-310/apex/contrib/openfold_triton
  creating build/lib.linux-x86_64-cpython-310/apex/contrib/optimizers
  copying apex/contrib/optimizers/__init__.py -> build/lib.linux-x86_64-cpython-310/apex/contrib/optimizers
  copying apex/contrib/optimizers/fused_adam.py -> build/lib.linux-x86_64-cpython-310/apex/contrib/optimizers
  copying apex/contrib/optimizers/fp16_optimizer.py -> build/lib.linux-x86_64-cpython-310/apex/contrib/optimizers
  copying apex/contrib/optimizers/fused_sgd.py -> build/lib.linux-x86_64-cpython-310/apex/contrib/optimizers
  copying apex/contrib/optimizers/fused_lamb.py -> build/lib.linux-x86_64-cpython-310/apex/contrib/optimizers
  copying apex/contrib/optimizers/distributed_fused_adam.py -> build/lib.linux-x86_64-cpython-310/apex/contrib/optimizers  copying apex/contrib/optimizers/distributed_fused_lamb.py -> build/lib.linux-x86_64-cpython-310/apex/contrib/optimizers  creating build/lib.linux-x86_64-cpython-310/apex/contrib/cudnn_gbn
  copying apex/contrib/cudnn_gbn/__init__.py -> build/lib.linux-x86_64-cpython-310/apex/contrib/cudnn_gbn
  copying apex/contrib/cudnn_gbn/batch_norm.py -> build/lib.linux-x86_64-cpython-310/apex/contrib/cudnn_gbn
  creating build/lib.linux-x86_64-cpython-310/apex/contrib/clip_grad
  copying apex/contrib/clip_grad/__init__.py -> build/lib.linux-x86_64-cpython-310/apex/contrib/clip_grad
  copying apex/contrib/clip_grad/clip_grad.py -> build/lib.linux-x86_64-cpython-310/apex/contrib/clip_grad
  creating build/lib.linux-x86_64-cpython-310/apex/contrib/sparsity
  copying apex/contrib/sparsity/__init__.py -> build/lib.linux-x86_64-cpython-310/apex/contrib/sparsity
  copying apex/contrib/sparsity/sparse_masklib.py -> build/lib.linux-x86_64-cpython-310/apex/contrib/sparsity
  copying apex/contrib/sparsity/asp.py -> build/lib.linux-x86_64-cpython-310/apex/contrib/sparsity
  copying apex/contrib/sparsity/permutation_lib.py -> build/lib.linux-x86_64-cpython-310/apex/contrib/sparsity
  creating build/lib.linux-x86_64-cpython-310/apex/contrib/index_mul_2d
  copying apex/contrib/index_mul_2d/__init__.py -> build/lib.linux-x86_64-cpython-310/apex/contrib/index_mul_2d
  copying apex/contrib/index_mul_2d/index_mul_2d.py -> build/lib.linux-x86_64-cpython-310/apex/contrib/index_mul_2d
  creating build/lib.linux-x86_64-cpython-310/apex/contrib/groupbn
  copying apex/contrib/groupbn/__init__.py -> build/lib.linux-x86_64-cpython-310/apex/contrib/groupbn
  copying apex/contrib/groupbn/batch_norm.py -> build/lib.linux-x86_64-cpython-310/apex/contrib/groupbn
  creating build/lib.linux-x86_64-cpython-310/apex/contrib/fmha
  copying apex/contrib/fmha/__init__.py -> build/lib.linux-x86_64-cpython-310/apex/contrib/fmha
  copying apex/contrib/fmha/fmha.py -> build/lib.linux-x86_64-cpython-310/apex/contrib/fmha
  creating build/lib.linux-x86_64-cpython-310/apex/contrib/nccl_allocator
  copying apex/contrib/nccl_allocator/__init__.py -> build/lib.linux-x86_64-cpython-310/apex/contrib/nccl_allocator
  copying apex/contrib/nccl_allocator/nccl_allocator.py -> build/lib.linux-x86_64-cpython-310/apex/contrib/nccl_allocator  creating build/lib.linux-x86_64-cpython-310/apex/contrib/layer_norm
  copying apex/contrib/layer_norm/__init__.py -> build/lib.linux-x86_64-cpython-310/apex/contrib/layer_norm
  copying apex/contrib/layer_norm/layer_norm.py -> build/lib.linux-x86_64-cpython-310/apex/contrib/layer_norm
  creating build/lib.linux-x86_64-cpython-310/apex/contrib/transducer
  copying apex/contrib/transducer/__init__.py -> build/lib.linux-x86_64-cpython-310/apex/contrib/transducer
  copying apex/contrib/transducer/_transducer_ref.py -> build/lib.linux-x86_64-cpython-310/apex/contrib/transducer
  copying apex/contrib/transducer/transducer.py -> build/lib.linux-x86_64-cpython-310/apex/contrib/transducer
  creating build/lib.linux-x86_64-cpython-310/apex/contrib/multihead_attn
  copying apex/contrib/multihead_attn/__init__.py -> build/lib.linux-x86_64-cpython-310/apex/contrib/multihead_attn
  copying apex/contrib/multihead_attn/mask_softmax_dropout_func.py -> build/lib.linux-x86_64-cpython-310/apex/contrib/multihead_attn
  copying apex/contrib/multihead_attn/fast_self_multihead_attn_norm_add_func.py -> build/lib.linux-x86_64-cpython-310/apex/contrib/multihead_attn
  copying apex/contrib/multihead_attn/fast_self_multihead_attn_func.py -> build/lib.linux-x86_64-cpython-310/apex/contrib/multihead_attn
  copying apex/contrib/multihead_attn/self_multihead_attn_func.py -> build/lib.linux-x86_64-cpython-310/apex/contrib/multihead_attn
  copying apex/contrib/multihead_attn/encdec_multihead_attn.py -> build/lib.linux-x86_64-cpython-310/apex/contrib/multihead_attn
  copying apex/contrib/multihead_attn/encdec_multihead_attn_func.py -> build/lib.linux-x86_64-cpython-310/apex/contrib/multihead_attn
  copying apex/contrib/multihead_attn/self_multihead_attn.py -> build/lib.linux-x86_64-cpython-310/apex/contrib/multihead_attn
  copying apex/contrib/multihead_attn/fast_encdec_multihead_attn_func.py -> build/lib.linux-x86_64-cpython-310/apex/contrib/multihead_attn
  copying apex/contrib/multihead_attn/fast_encdec_multihead_attn_norm_add_func.py -> build/lib.linux-x86_64-cpython-310/apex/contrib/multihead_attn
  creating build/lib.linux-x86_64-cpython-310/apex/contrib/bottleneck
  copying apex/contrib/bottleneck/__init__.py -> build/lib.linux-x86_64-cpython-310/apex/contrib/bottleneck
  copying apex/contrib/bottleneck/halo_exchangers.py -> build/lib.linux-x86_64-cpython-310/apex/contrib/bottleneck
  copying apex/contrib/bottleneck/test.py -> build/lib.linux-x86_64-cpython-310/apex/contrib/bottleneck
  copying apex/contrib/bottleneck/bottleneck.py -> build/lib.linux-x86_64-cpython-310/apex/contrib/bottleneck
  creating build/lib.linux-x86_64-cpython-310/apex/contrib/gpu_direct_storage
  copying apex/contrib/gpu_direct_storage/__init__.py -> build/lib.linux-x86_64-cpython-310/apex/contrib/gpu_direct_storage
  creating build/lib.linux-x86_64-cpython-310/apex/contrib/peer_memory
  copying apex/contrib/peer_memory/__init__.py -> build/lib.linux-x86_64-cpython-310/apex/contrib/peer_memory
  copying apex/contrib/peer_memory/peer_memory.py -> build/lib.linux-x86_64-cpython-310/apex/contrib/peer_memory
  copying apex/contrib/peer_memory/peer_halo_exchanger_1d.py -> build/lib.linux-x86_64-cpython-310/apex/contrib/peer_memory
  creating build/lib.linux-x86_64-cpython-310/apex/contrib/focal_loss
  copying apex/contrib/focal_loss/__init__.py -> build/lib.linux-x86_64-cpython-310/apex/contrib/focal_loss
  copying apex/contrib/focal_loss/focal_loss.py -> build/lib.linux-x86_64-cpython-310/apex/contrib/focal_loss
  creating build/lib.linux-x86_64-cpython-310/apex/contrib/test
  copying apex/contrib/test/__init__.py -> build/lib.linux-x86_64-cpython-310/apex/contrib/test
  creating build/lib.linux-x86_64-cpython-310/apex/contrib/sparsity/permutation_search_kernels
  copying apex/contrib/sparsity/permutation_search_kernels/exhaustive_search.py -> build/lib.linux-x86_64-cpython-310/apex/contrib/sparsity/permutation_search_kernels
  copying apex/contrib/sparsity/permutation_search_kernels/__init__.py -> build/lib.linux-x86_64-cpython-310/apex/contrib/sparsity/permutation_search_kernels
  copying apex/contrib/sparsity/permutation_search_kernels/call_permutation_search_kernels.py -> build/lib.linux-x86_64-cpython-310/apex/contrib/sparsity/permutation_search_kernels
  copying apex/contrib/sparsity/permutation_search_kernels/channel_swap.py -> build/lib.linux-x86_64-cpython-310/apex/contrib/sparsity/permutation_search_kernels
  copying apex/contrib/sparsity/permutation_search_kernels/permutation_utilities.py -> build/lib.linux-x86_64-cpython-310/apex/contrib/sparsity/permutation_search_kernels
  creating build/lib.linux-x86_64-cpython-310/apex/contrib/test/xentropy
  copying apex/contrib/test/xentropy/__init__.py -> build/lib.linux-x86_64-cpython-310/apex/contrib/test/xentropy
  copying apex/contrib/test/xentropy/test_label_smoothing.py -> build/lib.linux-x86_64-cpython-310/apex/contrib/test/xentropy
  creating build/lib.linux-x86_64-cpython-310/apex/contrib/test/conv_bias_relu
  copying apex/contrib/test/conv_bias_relu/__init__.py -> build/lib.linux-x86_64-cpython-310/apex/contrib/test/conv_bias_relu
  copying apex/contrib/test/conv_bias_relu/test_conv_bias_relu.py -> build/lib.linux-x86_64-cpython-310/apex/contrib/test/conv_bias_relu
  creating build/lib.linux-x86_64-cpython-310/apex/contrib/test/group_norm
  copying apex/contrib/test/group_norm/__init__.py -> build/lib.linux-x86_64-cpython-310/apex/contrib/test/group_norm
  copying apex/contrib/test/group_norm/test_group_norm.py -> build/lib.linux-x86_64-cpython-310/apex/contrib/test/group_norm
  creating build/lib.linux-x86_64-cpython-310/apex/contrib/test/optimizers
  copying apex/contrib/test/optimizers/__init__.py -> build/lib.linux-x86_64-cpython-310/apex/contrib/test/optimizers
  copying apex/contrib/test/optimizers/test_dist_adam.py -> build/lib.linux-x86_64-cpython-310/apex/contrib/test/optimizers
  copying apex/contrib/test/optimizers/test_distributed_fused_lamb.py -> build/lib.linux-x86_64-cpython-310/apex/contrib/test/optimizers
  creating build/lib.linux-x86_64-cpython-310/apex/contrib/test/cudnn_gbn
  copying apex/contrib/test/cudnn_gbn/__init__.py -> build/lib.linux-x86_64-cpython-310/apex/contrib/test/cudnn_gbn
  copying apex/contrib/test/cudnn_gbn/test_cudnn_gbn_with_two_gpus.py -> build/lib.linux-x86_64-cpython-310/apex/contrib/test/cudnn_gbn
  creating build/lib.linux-x86_64-cpython-310/apex/contrib/test/clip_grad
  copying apex/contrib/test/clip_grad/__init__.py -> build/lib.linux-x86_64-cpython-310/apex/contrib/test/clip_grad
  copying apex/contrib/test/clip_grad/test_clip_grad.py -> build/lib.linux-x86_64-cpython-310/apex/contrib/test/clip_grad  creating build/lib.linux-x86_64-cpython-310/apex/contrib/test/index_mul_2d
  copying apex/contrib/test/index_mul_2d/__init__.py -> build/lib.linux-x86_64-cpython-310/apex/contrib/test/index_mul_2d  copying apex/contrib/test/index_mul_2d/test_index_mul_2d.py -> build/lib.linux-x86_64-cpython-310/apex/contrib/test/index_mul_2d
  creating build/lib.linux-x86_64-cpython-310/apex/contrib/test/fmha
  copying apex/contrib/test/fmha/__init__.py -> build/lib.linux-x86_64-cpython-310/apex/contrib/test/fmha
  copying apex/contrib/test/fmha/test_fmha.py -> build/lib.linux-x86_64-cpython-310/apex/contrib/test/fmha
  creating build/lib.linux-x86_64-cpython-310/apex/contrib/test/layer_norm
  copying apex/contrib/test/layer_norm/__init__.py -> build/lib.linux-x86_64-cpython-310/apex/contrib/test/layer_norm
  copying apex/contrib/test/layer_norm/test_fast_layer_norm.py -> build/lib.linux-x86_64-cpython-310/apex/contrib/test/layer_norm
  creating build/lib.linux-x86_64-cpython-310/apex/contrib/test/transducer
  copying apex/contrib/test/transducer/__init__.py -> build/lib.linux-x86_64-cpython-310/apex/contrib/test/transducer
  copying apex/contrib/test/transducer/test_transducer_loss.py -> build/lib.linux-x86_64-cpython-310/apex/contrib/test/transducer
  copying apex/contrib/test/transducer/test_transducer_joint.py -> build/lib.linux-x86_64-cpython-310/apex/contrib/test/transducer
  creating build/lib.linux-x86_64-cpython-310/apex/contrib/test/multihead_attn
  copying apex/contrib/test/multihead_attn/__init__.py -> build/lib.linux-x86_64-cpython-310/apex/contrib/test/multihead_attn
  copying apex/contrib/test/multihead_attn/test_encdec_multihead_attn_norm_add.py -> build/lib.linux-x86_64-cpython-310/apex/contrib/test/multihead_attn
  copying apex/contrib/test/multihead_attn/test_self_multihead_attn_norm_add.py -> build/lib.linux-x86_64-cpython-310/apex/contrib/test/multihead_attn
  copying apex/contrib/test/multihead_attn/test_fast_self_multihead_attn_bias.py -> build/lib.linux-x86_64-cpython-310/apex/contrib/test/multihead_attn
  copying apex/contrib/test/multihead_attn/test_self_multihead_attn.py -> build/lib.linux-x86_64-cpython-310/apex/contrib/test/multihead_attn
  copying apex/contrib/test/multihead_attn/test_mha_fused_softmax.py -> build/lib.linux-x86_64-cpython-310/apex/contrib/test/multihead_attn
  copying apex/contrib/test/multihead_attn/test_encdec_multihead_attn.py -> build/lib.linux-x86_64-cpython-310/apex/contrib/test/multihead_attn
  creating build/lib.linux-x86_64-cpython-310/apex/contrib/test/bottleneck
  copying apex/contrib/test/bottleneck/__init__.py -> build/lib.linux-x86_64-cpython-310/apex/contrib/test/bottleneck
  copying apex/contrib/test/bottleneck/test_bottleneck_module.py -> build/lib.linux-x86_64-cpython-310/apex/contrib/test/bottleneck
  creating build/lib.linux-x86_64-cpython-310/apex/contrib/test/peer_memory
  copying apex/contrib/test/peer_memory/__init__.py -> build/lib.linux-x86_64-cpython-310/apex/contrib/test/peer_memory
  copying apex/contrib/test/peer_memory/test_peer_halo_exchange_module.py -> build/lib.linux-x86_64-cpython-310/apex/contrib/test/peer_memory
  creating build/lib.linux-x86_64-cpython-310/apex/contrib/test/focal_loss
  copying apex/contrib/test/focal_loss/__init__.py -> build/lib.linux-x86_64-cpython-310/apex/contrib/test/focal_loss
  copying apex/contrib/test/focal_loss/test_focal_loss.py -> build/lib.linux-x86_64-cpython-310/apex/contrib/test/focal_loss
  running build_ext
  /usr/local/lib/python3.10/site-packages/torch/utils/cpp_extension.py:428: UserWarning: There are no g++ version bounds defined for CUDA version 12.1
    warnings.warn(f'There are no {compiler_name} version bounds defined for CUDA version {cuda_str_version}')
  building 'apex_C' extension
  creating build/temp.linux-x86_64-cpython-310
  creating build/temp.linux-x86_64-cpython-310/csrc
  gcc -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -I/usr/local/lib/python3.10/site-packages/torch/include -I/usr/local/lib/python3.10/site-packages/torch/include/torch/csrc/api/include -I/usr/local/lib/python3.10/site-packages/torch/include/TH -I/usr/local/lib/python3.10/site-packages/torch/include/THC -I/usr/local/include/python3.10 -c csrc/flatten_unflatten.cpp -o build/temp.linux-x86_64-cpython-310/csrc/flatten_unflatten.o -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE=\"_gcc\" -DPYBIND11_STDLIB=\"_libstdcpp\" -DPYBIND11_BUILD_ABI=\"_cxxabi1011\" -DTORCH_EXTENSION_NAME=apex_C -D_GLIBCXX_USE_CXX11_ABI=0 -std=c++17
  g++ -shared build/temp.linux-x86_64-cpython-310/csrc/flatten_unflatten.o -L/usr/local/lib/python3.10/site-packages/torch/lib -lc10 -ltorch -ltorch_cpu -ltorch_python -o build/lib.linux-x86_64-cpython-310/apex_C.cpython-310-x86_64-linux-gnu.so
  building 'amp_C' extension
  gcc -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -I/usr/local/lib/python3.10/site-packages/torch/include -I/usr/local/lib/python3.10/site-packages/torch/include/torch/csrc/api/include -I/usr/local/lib/python3.10/site-packages/torch/include/TH -I/usr/local/lib/python3.10/site-packages/torch/include/THC -I/usr/local/cuda/include -I/usr/local/include/python3.10 -c csrc/amp_C_frontend.cpp -o build/temp.linux-x86_64-cpython-310/csrc/amp_C_frontend.o -O3 -DVERSION_GE_1_1 -DVERSION_GE_1_3 -DVERSION_GE_1_5 -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE=\"_gcc\" -DPYBIND11_STDLIB=\"_libstdcpp\" -DPYBIND11_BUILD_ABI=\"_cxxabi1011\" -DTORCH_EXTENSION_NAME=amp_C -D_GLIBCXX_USE_CXX11_ABI=0 -std=c++17
  Traceback (most recent call last):
    File "/usr/local/lib/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
      main()
    File "/usr/local/lib/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
      json_out['return_val'] = hook(**hook_input['kwargs'])
    File "/usr/local/lib/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 251, in build_wheel
      return _build_backend().build_wheel(wheel_directory, config_settings,
    File "/usr/local/lib/python3.10/site-packages/setuptools/build_meta.py", line 412, in build_wheel
      return self._build_with_temp_dir(['bdist_wheel'], '.whl',
    File "/usr/local/lib/python3.10/site-packages/setuptools/build_meta.py", line 397, in _build_with_temp_dir
      self.run_setup()
    File "/usr/local/lib/python3.10/site-packages/setuptools/build_meta.py", line 335, in run_setup
      exec(code, locals())
    File "<string>", line 863, in <module>
    File "/usr/local/lib/python3.10/site-packages/setuptools/__init__.py", line 87, in setup
      return distutils.core.setup(**attrs)
    File "/usr/local/lib/python3.10/site-packages/setuptools/_distutils/core.py", line 185, in setup
      return run_commands(dist)
    File "/usr/local/lib/python3.10/site-packages/setuptools/_distutils/core.py", line 201, in run_commands
      dist.run_commands()
    File "/usr/local/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 968, in run_commands
      self.run_command(cmd)
    File "/usr/local/lib/python3.10/site-packages/setuptools/dist.py", line 1217, in run_command
      super().run_command(command)
    File "/usr/local/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 987, in run_command
      cmd_obj.run()
    File "/usr/local/lib/python3.10/site-packages/wheel/bdist_wheel.py", line 368, in run
      self.run_command("build")
    File "/usr/local/lib/python3.10/site-packages/setuptools/_distutils/cmd.py", line 319, in run_command
      self.distribution.run_command(command)
    File "/usr/local/lib/python3.10/site-packages/setuptools/dist.py", line 1217, in run_command
      super().run_command(command)
    File "/usr/local/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 987, in run_command
      cmd_obj.run()
    File "/usr/local/lib/python3.10/site-packages/setuptools/_distutils/command/build.py", line 132, in run
      self.run_command(cmd_name)
    File "/usr/local/lib/python3.10/site-packages/setuptools/_distutils/cmd.py", line 319, in run_command
      self.distribution.run_command(command)
    File "/usr/local/lib/python3.10/site-packages/setuptools/dist.py", line 1217, in run_command
root@8fbff3ef7fc2:/Python-3.10.14/apex#
root@8fbff3ef7fc2:/Python-3.10.14/apex# -packages/setuptools/_distutils/dist.py", line 987, in run_command
root@8fbff3ef7fc2:/Python-3.10.14/apex#
root@8fbff3ef7fc2:/Python-3.10.14/apex# -packages/setuptools/command/build_ext.py", line 84, in run
root@8fbff3ef7fc2:/Python-3.10.14/apex#
root@8fbff3ef7fc2:/Python-3.10.14/apex# -packages/setuptools/_distutils/command/build_ext.py", line 346, in run
root@8fbff3ef7fc2:/Python-3.10.14/apex#
root@8fbff3ef7fc2:/Python-3.10.14/apex# -packages/torch/utils/cpp_extension.py", line 870, in build_extensions
root@8fbff3ef7fc2:/Python-3.10.14/apex#
root@8fbff3ef7fc2:/Python-3.10.14/apex# -packages/setuptools/_distutils/command/build_ext.py", line 466, in build_extensions
root@8fbff3ef7fc2:/Python-3.10.14/apex#
root@8fbff3ef7fc2:/Python-3.10.14/apex# -packages/setuptools/_distutils/command/build_ext.py", line 492, in _build_extensions_serial
root@8fbff3ef7fc2:/Python-3.10.14/apex#
root@8fbff3ef7fc2:/Python-3.10.14/apex# -packages/setuptools/command/build_ext.py", line 246, in build_extension
root@8fbff3ef7fc2:/Python-3.10.14/apex# xt)
root@8fbff3ef7fc2:/Python-3.10.14/apex# -packages/setuptools/_distutils/command/build_ext.py", line 547, in build_extension
root@8fbff3ef7fc2:/Python-3.10.14/apex#
root@8fbff3ef7fc2:/Python-3.10.14/apex# -packages/setuptools/_distutils/ccompiler.py", line 599, in compile
root@8fbff3ef7fc2:/Python-3.10.14/apex# gs, extra_postargs, pp_opts)
root@8fbff3ef7fc2:/Python-3.10.14/apex# -packages/torch/utils/cpp_extension.py", line 606, in unix_wrap_single_compile
root@8fbff3ef7fc2:/Python-3.10.14/apex#
root@8fbff3ef7fc2:/Python-3.10.14/apex# -packages/torch/utils/cpp_extension.py", line 573, in unix_cuda_flags
root@8fbff3ef7fc2:/Python-3.10.14/apex# gs))
root@8fbff3ef7fc2:/Python-3.10.14/apex# -packages/torch/utils/cpp_extension.py", line 2000, in _get_cuda_arch_flags
root@8fbff3ef7fc2:/Python-3.10.14/apex# ch ({arch}) or GPU not supported")
root@8fbff3ef7fc2:/Python-3.10.14/apex# ) or GPU not supported
root@8fbff3ef7fc2:/Python-3.10.14/apex#
root@8fbff3ef7fc2:/Python-3.10.14/apex#
root@8fbff3ef7fc2:/Python-3.10.14/apex# oml) did not run successfully.
root@8fbff3ef7fc2:/Python-3.10.14/apex#
root@8fbff3ef7fc2:/Python-3.10.14/apex#
root@8fbff3ef7fc2:/Python-3.10.14/apex#
root@8fbff3ef7fc2:/Python-3.10.14/apex# process, and is likely not a problem with pip.
root@8fbff3ef7fc2:/Python-3.10.14/apex# 0 /usr/local/lib/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py build_wheel /tmp/tmp0id63e2c
root@8fbff3ef7fc2:/Python-3.10.14/apex#
root@8fbff3ef7fc2:/Python-3.10.14/apex# l) ... error
root@8fbff3ef7fc2:/Python-3.10.14/apex#
root@8fbff3ef7fc2:/Python-3.10.14/apex#
root@8fbff3ef7fc2:/Python-3.10.14/apex# e wheels for some pyproject.toml based projects (apex)
root@8fbff3ef7fc2:/Python-3.10.14/apex#
root@8fbff3ef7fc2:/Python-3.10.14/apex#
root@8fbff3ef7fc2:/Python-3.10.14/apex#
root@8fbff3ef7fc2:/Python-3.10.14/apex#
root@8fbff3ef7fc2:/Python-3.10.14/apex#
root@8fbff3ef7fc2:/Python-3.10.14/apex#
root@8fbff3ef7fc2:/Python-3.10.14/apex#
root@8fbff3ef7fc2:/Python-3.10.14/apex#
root@8fbff3ef7fc2:/Python-3.10.14/apex#
root@8fbff3ef7fc2:/Python-3.10.14/apex#
root@8fbff3ef7fc2:/Python-3.10.14/apex#
root@8fbff3ef7fc2:/Python-3.10.14/apex#
root@8fbff3ef7fc2:/Python-3.10.14/apex#
root@8fbff3ef7fc2:/Python-3.10.14/apex#
root@8fbff3ef7fc2:/Python-3.10.14/apex#
root@8fbff3ef7fc2:/Python-3.10.14/apex#
root@8fbff3ef7fc2:/Python-3.10.14/apex#
root@8fbff3ef7fc2:/Python-3.10.14/apex#
root@8fbff3ef7fc2:/Python-3.10.14/apex#
root@8fbff3ef7fc2:/Python-3.10.14/apex#
root@8fbff3ef7fc2:/Python-3.10.14/apex#
root@8fbff3ef7fc2:/Python-3.10.14/apex# TORCH_CUDA_ARCH_LIST="compute capability" pip install -v --disable-pip-version-check --no-cache-dir --no-build-isolation --config-settings "--build-option=--cpp_ext" --config-settings "--build-option=--cuda_ext" ./
Using pip 24.1.2 from /usr/local/lib/python3.10/site-packages/pip (python 3.10)
Processing /Python-3.10.14/apex
  Running command Preparing metadata (pyproject.toml)


  torch.__version__  = 2.3.1+cu121


  running dist_info
  creating /tmp/pip-modern-metadata-ppgfu8vt/apex.egg-info
  writing /tmp/pip-modern-metadata-ppgfu8vt/apex.egg-info/PKG-INFO
  writing dependency_links to /tmp/pip-modern-metadata-ppgfu8vt/apex.egg-info/dependency_links.txt
  writing requirements to /tmp/pip-modern-metadata-ppgfu8vt/apex.egg-info/requires.txt
  writing top-level names to /tmp/pip-modern-metadata-ppgfu8vt/apex.egg-info/top_level.txt
  writing manifest file '/tmp/pip-modern-metadata-ppgfu8vt/apex.egg-info/SOURCES.txt'
  reading manifest file '/tmp/pip-modern-metadata-ppgfu8vt/apex.egg-info/SOURCES.txt'
  adding license file 'LICENSE'
  writing manifest file '/tmp/pip-modern-metadata-ppgfu8vt/apex.egg-info/SOURCES.txt'
  creating '/tmp/pip-modern-metadata-ppgfu8vt/apex-0.1.dist-info'
  Preparing metadata (pyproject.toml) ... done
Requirement already satisfied: packaging>20.6 in /usr/local/lib/python3.10/site-packages (from apex==0.1) (24.1)
Building wheels for collected packages: apex
  Running command Building wheel for apex (pyproject.toml)


  torch.__version__  = 2.3.1+cu121



  Compiling cuda extensions with
  nvcc: NVIDIA (R) Cuda compiler driver
  Copyright (c) 2005-2023 NVIDIA Corporation
  Built on Mon_Apr__3_17:16:06_PDT_2023
  Cuda compilation tools, release 12.1, V12.1.105
  Build cuda_12.1.r12.1/compiler.32688072_0
  from /usr/local/cuda/bin

  running bdist_wheel
  /usr/local/lib/python3.10/site-packages/torch/utils/cpp_extension.py:499: UserWarning: Attempted to use ninja as the BuildExtension backend but we could not find ninja.. Falling back to using the slow distutils backend.
    warnings.warn(msg.format('we could not find ninja.'))
  running build
  running build_py
  running build_ext
  /usr/local/lib/python3.10/site-packages/torch/utils/cpp_extension.py:428: UserWarning: There are no g++ version bounds defined for CUDA version 12.1
    warnings.warn(f'There are no {compiler_name} version bounds defined for CUDA version {cuda_str_version}')
  building 'amp_C' extension
  gcc -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -I/usr/local/lib/python3.10/site-packages/torch/include -I/usr/local/lib/python3.10/site-packages/torch/include/torch/csrc/api/include -I/usr/local/lib/python3.10/site-packages/torch/include/TH -I/usr/local/lib/python3.10/site-packages/torch/include/THC -I/usr/local/cuda/include -I/usr/local/include/python3.10 -c csrc/amp_C_frontend.cpp -o build/temp.linux-x86_64-cpython-310/csrc/amp_C_frontend.o -O3 -DVERSION_GE_1_1 -DVERSION_GE_1_3 -DVERSION_GE_1_5 -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE=\"_gcc\" -DPYBIND11_STDLIB=\"_libstdcpp\" -DPYBIND11_BUILD_ABI=\"_cxxabi1011\" -DTORCH_EXTENSION_NAME=amp_C -D_GLIBCXX_USE_CXX11_ABI=0 -std=c++17
  Traceback (most recent call last):
    File "/usr/local/lib/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
      main()
    File "/usr/local/lib/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
      json_out['return_val'] = hook(**hook_input['kwargs'])
    File "/usr/local/lib/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 251, in build_wheel
      return _build_backend().build_wheel(wheel_directory, config_settings,
    File "/usr/local/lib/python3.10/site-packages/setuptools/build_meta.py", line 412, in build_wheel
      return self._build_with_temp_dir(['bdist_wheel'], '.whl',
    File "/usr/local/lib/python3.10/site-packages/setuptools/build_meta.py", line 397, in _build_with_temp_dir
      self.run_setup()
    File "/usr/local/lib/python3.10/site-packages/setuptools/build_meta.py", line 335, in run_setup
      exec(code, locals())
    File "<string>", line 863, in <module>
    File "/usr/local/lib/python3.10/site-packages/setuptools/__init__.py", line 87, in setup
      return distutils.core.setup(**attrs)
    File "/usr/local/lib/python3.10/site-packages/setuptools/_distutils/core.py", line 185, in setup
      return run_commands(dist)
    File "/usr/local/lib/python3.10/site-packages/setuptools/_distutils/core.py", line 201, in run_commands
      dist.run_commands()
    File "/usr/local/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 968, in run_commands
      self.run_command(cmd)
    File "/usr/local/lib/python3.10/site-packages/setuptools/dist.py", line 1217, in run_command
      super().run_command(command)
    File "/usr/local/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 987, in run_command
      cmd_obj.run()
    File "/usr/local/lib/python3.10/site-packages/wheel/bdist_wheel.py", line 368, in run
      self.run_command("build")
    File "/usr/local/lib/python3.10/site-packages/setuptools/_distutils/cmd.py", line 319, in run_command
      self.distribution.run_command(command)
    File "/usr/local/lib/python3.10/site-packages/setuptools/dist.py", line 1217, in run_command
      super().run_command(command)
    File "/usr/local/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 987, in run_command
      cmd_obj.run()
    File "/usr/local/lib/python3.10/site-packages/setuptools/_distutils/command/build.py", line 132, in run
      self.run_command(cmd_name)
    File "/usr/local/lib/python3.10/site-packages/setuptools/_distutils/cmd.py", line 319, in run_command
      self.distribution.run_command(command)
    File "/usr/local/lib/python3.10/site-packages/setuptools/dist.py", line 1217, in run_command
      super().run_command(command)
    File "/usr/local/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 987, in run_command
      cmd_obj.run()
    File "/usr/local/lib/python3.10/site-packages/setuptools/command/build_ext.py", line 84, in run
      _build_ext.run(self)
    File "/usr/local/lib/python3.10/site-packages/setuptools/_distutils/command/build_ext.py", line 346, in run
      self.build_extensions()
    File "/usr/local/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 870, in build_extensions
      build_ext.build_extensions(self)
    File "/usr/local/lib/python3.10/site-packages/setuptools/_distutils/command/build_ext.py", line 466, in build_extensions
      self._build_extensions_serial()
    File "/usr/local/lib/python3.10/site-packages/setuptools/_distutils/command/build_ext.py", line 492, in _build_extensions_serial
      self.build_extension(ext)
    File "/usr/local/lib/python3.10/site-packages/setuptools/command/build_ext.py", line 246, in build_extension
      _build_ext.build_extension(self, ext)
    File "/usr/local/lib/python3.10/site-packages/setuptools/_distutils/command/build_ext.py", line 547, in build_extension
      objects = self.compiler.compile(
    File "/usr/local/lib/python3.10/site-packages/setuptools/_distutils/ccompiler.py", line 599, in compile
      self._compile(obj, src, ext, cc_args, extra_postargs, pp_opts)
    File "/usr/local/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 606, in unix_wrap_single_compile
      cflags = unix_cuda_flags(cflags)
    File "/usr/local/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 573, in unix_cuda_flags
      cflags + _get_cuda_arch_flags(cflags))
    File "/usr/local/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 2000, in _get_cuda_arch_flags
      raise ValueError(f"Unknown CUDA arch ({arch}) or GPU not supported")
  ValueError: Unknown CUDA arch (compute) or GPU not supported
  error: subprocess-exited-with-error

  × Building wheel for apex (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> See above for output.

  note: This error originates from a subprocess, and is likely not a problem with pip.
  full command: /usr/local/bin/python3.10 /usr/local/lib/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py build_wheel /tmp/tmpbssz_jla
  cwd: /Python-3.10.14/apex
  Building wheel for apex (pyproject.toml) ... error
  ERROR: Failed building wheel for apex
Failed to build apex
ERROR: ERROR: Failed to build installable wheels for some pyproject.toml based projects (apex)

nvida-smi output

Fri Jul 19 17:29:55 2024
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 555.58.02              Driver Version: 556.12         CUDA Version: 12.5     |
|-----------------------------------------+------------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
|                                         |                        |               MIG M. |
|=========================================+========================+======================|
|   0  NVIDIA GeForce RTX 4080 ...    On  |   00000000:01:00.0 Off |                  N/A |
| N/A   47C    P8              2W /  155W |       0MiB /  12282MiB |      0%      Default |
|                                         |                        |                  N/A |
+-----------------------------------------+------------------------+----------------------+

+-----------------------------------------------------------------------------------------+
| Processes:                                                                              |
|  GPU   GI   CI        PID   Type   Process name                              GPU Memory |
|        ID   ID                                                               Usage      |
|=========================================================================================|
|  No running processes found                                                             |
+-----------------------------------------------------------------------------------------+
@mik300
Copy link

mik300 commented Nov 7, 2024

I got the same error, any fix?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@AvisP @mik300