Rebuild for CUDA 12 w/arch + Windows support #56

regro-cf-autotick-bot · 2023-11-17T04:49:35Z

This PR has been triggered in an effort to update cuda120.

Notes and instructions for merging this PR:

Please merge the PR only after the tests have passed.
Feel free to push to the bot's branch to update this PR if needed.

Please note that if you close this PR we presume that the feedstock has been rebuilt, so if you are going to perform the rebuild yourself don't close this PR until the your rebuild has been merged.

Here are some more details about this specific migrator:

The transition to CUDA 12 SDK includes new packages for all CUDA libraries and build tools. Notably, the cudatoolkit package no longer exists, and packages should depend directly on the specific CUDA libraries (libcublas, libcusolver, etc) as needed. For an in-depth overview of the changes and to report problems see this issue. Please feel free to raise any issues encountered there. Thank you! 🙏

If this PR was opened in error or needs to be updated please add the bot-rerun label to this PR. The bot will close this PR and schedule another one. If you do not have permissions to add this label, you can use the phrase @conda-forge-admin, please rerun bot in a PR comment to have the conda-forge-admin add it for you.

_{This PR was created by the regro-cf-autotick-bot. The regro-cf-autotick-bot is a service to automatically track the dependency graph, migrate packages, and propose package version updates for conda-forge. Feel free to drop us a line if there are any issues! This PR was generated by https://github.com/regro/cf-scripts/actions/runs/6899793194, please use this URL for debugging.}

The transition to CUDA 12 SDK includes new packages for all CUDA libraries and build tools. Notably, the cudatoolkit package no longer exists, and packages should depend directly on the specific CUDA libraries (libcublas, libcusolver, etc) as needed. For an in-depth overview of the changes and to report problems [see this issue]( conda-forge/conda-forge.github.io#1963 ). Please feel free to raise any issues encountered there. Thank you! 🙏

…nda-forge-pinning 2023.11.16.09.46.51

conda-forge-webservices · 2023-11-17T04:49:41Z

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipe) and found it was in an excellent condition.

traversaro · 2023-11-17T07:58:57Z

Some aarch64/ppc64le cuda builds are failing with:

  #$
  "/home/conda/feedstock_root/build_artifacts/librealsense_1700196953429/_build_env/bin"/aarch64-conda-linux-gnu-c++
  -D__CUDA_ARCH_LIST__=520 -E -x c++ -D__CUDACC__ -D__NVCC__
  -I"/home/conda/feedstock_root/build_artifacts/librealsense_1700196953429/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehol/targets/sbsa-linux/include"
  "-I/home/conda/feedstock_root/build_artifacts/librealsense_1700196953429/_build_env/bin/../targets/sbsa-linux/include"
  -D__CUDACC_VER_MAJOR__=12 -D__CUDACC_VER_MINOR__=0
  -D__CUDACC_VER_BUILD__=76 -D__CUDA_API_VER_MAJOR__=12
  -D__CUDA_API_VER_MINOR__=0 -D__NVCC_DIAG_PRAGMA_SUPPORT__=1 -include
  "cuda_runtime.h" "CMakeCUDACompilerId.cu" -o
  "tmp/CMakeCUDACompilerId.cpp4.ii"

  #$ cudafe++ --c++17 --gnu_version=120300 --display_error_number
  --orig_src_file_name "CMakeCUDACompilerId.cu" --orig_src_path_name
  "/home/conda/feedstock_root/build_artifacts/librealsense_1700196953429/work/build/CMakeFiles/3.27.8/CompilerIdCUDA/CMakeCUDACompilerId.cu"
  --allow_managed --unsigned_chars --unsigned_wchar_t --m64 --parse_templates
  --gen_c_file_name "tmp/CMakeCUDACompilerId.cudafe1.cpp" --stub_file_name
  "CMakeCUDACompilerId.cudafe1.stub.c" --gen_module_id_file
  --module_id_file_name "tmp/CMakeCUDACompilerId.module_id"
  "tmp/CMakeCUDACompilerId.cpp4.ii"

  nvcc error : 'cudafe++' died due to signal 11 (Invalid memory reference)

  nvcc error : 'cudafe++' core dumped

  # --error 0x8b --

Probably it is just an out of memory issue?

traversaro · 2023-11-17T08:01:16Z

@conda-forge-admin re-render

…nda-forge-pinning 2023.11.16.09.46.51

traversaro · 2023-11-17T10:35:28Z

Some aarch64/ppc64le cuda builds are failing with:

  #$
  "/home/conda/feedstock_root/build_artifacts/librealsense_1700196953429/_build_env/bin"/aarch64-conda-linux-gnu-c++
  -D__CUDA_ARCH_LIST__=520 -E -x c++ -D__CUDACC__ -D__NVCC__
  -I"/home/conda/feedstock_root/build_artifacts/librealsense_1700196953429/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehol/targets/sbsa-linux/include"
  "-I/home/conda/feedstock_root/build_artifacts/librealsense_1700196953429/_build_env/bin/../targets/sbsa-linux/include"
  -D__CUDACC_VER_MAJOR__=12 -D__CUDACC_VER_MINOR__=0
  -D__CUDACC_VER_BUILD__=76 -D__CUDA_API_VER_MAJOR__=12
  -D__CUDA_API_VER_MINOR__=0 -D__NVCC_DIAG_PRAGMA_SUPPORT__=1 -include
  "cuda_runtime.h" "CMakeCUDACompilerId.cu" -o
  "tmp/CMakeCUDACompilerId.cpp4.ii"

  #$ cudafe++ --c++17 --gnu_version=120300 --display_error_number
  --orig_src_file_name "CMakeCUDACompilerId.cu" --orig_src_path_name
  "/home/conda/feedstock_root/build_artifacts/librealsense_1700196953429/work/build/CMakeFiles/3.27.8/CompilerIdCUDA/CMakeCUDACompilerId.cu"
  --allow_managed --unsigned_chars --unsigned_wchar_t --m64 --parse_templates
  --gen_c_file_name "tmp/CMakeCUDACompilerId.cudafe1.cpp" --stub_file_name
  "CMakeCUDACompilerId.cudafe1.stub.c" --gen_module_id_file
  --module_id_file_name "tmp/CMakeCUDACompilerId.module_id"
  "tmp/CMakeCUDACompilerId.cpp4.ii"

  nvcc error : 'cudafe++' died due to signal 11 (Invalid memory reference)

  nvcc error : 'cudafe++' core dumped

  # --error 0x8b --

Probably it is just an out of memory issue?

The file that is failing is a simple file used by CMake to identify the CUDA compiler use, so I think it is not an out of memory issue.

jakirkham · 2023-12-02T04:43:36Z

@conda-forge-admin , please re-render

…nda-forge-pinning 2023.12.01.08.31.55

jakirkham · 2023-12-02T04:49:52Z

@conda-forge-admin , please re-render

…nda-forge-pinning 2023.12.01.08.31.55

jakirkham · 2023-12-02T04:54:14Z

@conda-forge-admin , please re-render

…nda-forge-pinning 2023.12.01.08.31.55

jakirkham · 2023-12-02T05:24:43Z

Cleaned up disk space on Azure and switched to cross-compilation, which appears to have addressed the memory issue

jakirkham · 2023-12-02T05:28:02Z

Seeing these errors from CMake on CI

CMake Error at /home/conda/feedstock_root/build_artifacts/librealsense_1701494354003/_build_env/share/cmake-3.27/Modules/FindPackageHandleStandardArgs.cmake:230 (message):
  Could NOT find CUDA (missing: CUDA_TOOLKIT_ROOT_DIR CUDA_CUDART_LIBRARY)
  (found version "12.0")

Maybe this logic needs to be reworked a bit

librealsense-feedstock/recipe/build.sh

Lines 26 to 29 in 45ef8a2

    
           elif [[ ! -z "${cuda_compiler_version+x}" && "${cuda_compiler_version}" != "None" ]] 
        
           then 
        
               echo "==> cuda_compiler_version=${cuda_compiler_version}, use CMake's CUDA support" 
        
               CMAKE_ARGS="${CMAKE_ARGS} -DBUILD_WITH_CUDA=ON -DCMAKE_CUDA_ARCHITECTURES=all"

traversaro · 2023-12-02T06:43:39Z

Thanks @jakirkham ! Yes now that only 11.8 + 12 are supported we can simplify that code (I can work on this in a few days), thanks for chiming in.

jakirkham · 2024-01-04T01:09:52Z

@conda-forge-admin , please re-render

jakirkham · 2024-01-04T01:30:42Z

@conda-forge-admin , please re-render

traversaro · 2024-03-07T08:55:09Z

@conda-forge-admin , please re-render

…nda-forge-pinning 2024.03.07.00.29.48

jakirkham · 2024-03-07T09:29:04Z

Would add cuda-version to requirements/host like so

requirements:
  ...
  host:
    ...
    - cuda-version {{ cuda_compiler_version }}  # [(cuda_compiler_version or "None") != "None"]

This will ensure that requirements/build and requirements/host use the same CUDA version for compilers, libraries, etc.

traversaro · 2024-03-07T13:44:04Z

@conda-forge-admin , please re-render

…nda-forge-pinning 2024.03.07.13.21.27

traversaro · 2024-03-07T14:49:34Z

Travis build of ppc64 are super-slow. I think it make sense to disable CUDA for ppc64, and switch back ppc64 to use cross-compilation (where CUDA was failing).

jakirkham

Thanks Silvio! 🙏

Added a few suggestions to see if we can get cross-compilation to work

recipe/build.sh

conda-forge.yml

github-actions · 2024-03-07T21:40:36Z

Hi! This is the friendly conda-forge automerge bot!

I considered the following status checks when analyzing this PR:

linter: passed
travis: failed
azure: passed

Thus the PR was not passing and not merged.

Co-authored-by: jakirkham <jakirkham@gmail.com>

traversaro · 2024-03-07T21:42:59Z

@conda-forge-admin , please re-render

github-actions · 2024-03-07T21:43:07Z

Hi! This is the friendly conda-forge automerge bot!

Commits were made to this PR after the automerge label was added. For security reasons, I have disabled automerge by removing the automerge label. Please add the automerge label again (or ask a maintainer to do so) if you'd like to enable automerge again!

traversaro · 2024-03-07T21:43:22Z

Thanks Silvio! 🙏

Added a few suggestions to see if we can get cross-compilation to work

Thanks! I just applied them, let's see if they work.

…nda-forge-pinning 2024.03.07.20.43.08

jakirkham · 2024-03-07T23:13:31Z

recipe/meta.yaml

@@ -44,6 +44,7 @@ requirements:
    - python
    - libudev  # [linux and cdt_name!='cos6']
    - libusb
+    - cuda-version {{ cuda_compiler_version }}  # [(cuda_compiler_version or "None") != "None"]


It looks like CMake can't find cudart. So let's add it as a dependency and see if that helps

We should double check whether it is linking to cudart or not. If not, we may want to add this to ignore_run_exports_from above

Suggested change

- cuda-version {{ cuda_compiler_version }} # [(cuda_compiler_version or "None") != "None"]

- cuda-version {{ cuda_compiler_version }} # [(cuda_compiler_version or "None") != "None"]

- cuda-cudart-dev # [(cuda_compiler_version or "None") != "None"]

Ah, I missed this comment and added it like this dc6ed15 (I added it only for cuda==12 builds).

traversaro · 2024-03-07T23:28:20Z

@conda-forge-admin , please re-render

github-actions · 2024-03-07T23:33:14Z

Hi! This is the friendly automated conda-forge-webservice.

I tried to rerender for you, but it looks like there was nothing to do.

This message was generated by GitHub actions workflow run https://github.com/conda-forge/librealsense-feedstock/actions/runs/8196232282.

jakirkham · 2024-03-08T00:44:44Z

Ok think I understand what is happening

librealsense still uses some deprecated CMake features that need to be changed. Raised upstream issue ( IntelRealSense/librealsense#12736 ), which includes more info about the changes needed

Think we can disable CUDA 12 on linux_aarch64 and linux_ppc64le until this is fixed upstream. We can also raise an issue on the feedstock to track these upstream changes that we are waiting on

traversaro · 2024-03-08T00:46:56Z

Think we can disable CUDA 12 on linux_aarch64 and linux_ppc64le until this is fixed upstream. We can also raise an issue on the feedstock to track these upstream changes that we are waiting on

Can't we use the non-crosscompile build, at least aarch64 ? I do not care a lot for ppc64le, but we use a lot this library on Jetson boards, and having CUDA 12 builds but not for aarch64 may be confusing for users from my organization.

jakirkham · 2024-03-08T00:51:41Z

Yeah wasn't sure what level of effort we were going for. So just proposed the simplest approach

Certainly we could do native builds if that is tenable for you

Another option might be to patch the CMake files to enable cross-compilation here. @Tobias-Fischer did this for dlib. So he may be able to advise

traversaro · 2024-03-08T19:05:26Z

@conda-forge-admin , please re-render

…nda-forge-pinning 2024.03.08.18.51.49

regro-cf-autotick-bot added 2 commits November 17, 2023 04:47

MNT: Re-rendered with conda-build 3.27.0, conda-smithy 3.29.0, and co…

0d9cc3b

…nda-forge-pinning 2023.11.16.09.46.51

regro-cf-autotick-bot requested a review from traversaro as a code owner November 17, 2023 04:49

Update conda-forge.yml

9af31eb

MNT: Re-rendered with conda-build 3.27.0, conda-smithy 3.29.0, and co…

74fa883

…nda-forge-pinning 2023.11.16.09.46.51

Sort keys in conda-forge.yml

cefea51

conda-forge-webservices[bot] and others added 2 commits December 2, 2023 04:46

MNT: Re-rendered with conda-build 3.27.0, conda-smithy 3.30.1, and co…

49fedd2

…nda-forge-pinning 2023.12.01.08.31.55

Free space on Azure

4f3d7ac

conda-forge-webservices[bot] and others added 2 commits December 2, 2023 04:53

MNT: Re-rendered with conda-build 3.27.0, conda-smithy 3.30.1, and co…

e8f4918

…nda-forge-pinning 2023.12.01.08.31.55

Switch arch builds to cross-compilation

6fb1171

MNT: Re-rendered with conda-build 3.27.0, conda-smithy 3.30.1, and co…

731cf8d

…nda-forge-pinning 2023.12.01.08.31.55

Update build.sh

a51651d

traversaro closed this Mar 7, 2024

traversaro reopened this Mar 7, 2024

MNT: Re-rendered with conda-build 24.1.2, conda-smithy 3.31.1, and co…

0574bd7

…nda-forge-pinning 2024.03.07.00.29.48

Update meta.yaml

3f890ce

Update conda-forge.yml

d4c3cfc

MNT: Re-rendered with conda-build 24.1.2, conda-smithy 3.31.1, and co…

4ea2937

…nda-forge-pinning 2024.03.07.13.21.27

traversaro approved these changes Mar 7, 2024

View reviewed changes

traversaro added the automerge Merge the PR when CI passes label Mar 7, 2024

jakirkham reviewed Mar 7, 2024

View reviewed changes

recipe/build.sh Outdated Show resolved Hide resolved

conda-forge.yml Outdated Show resolved Hide resolved

Switch to cross-compilation

a468fd8

Co-authored-by: jakirkham <jakirkham@gmail.com>

github-actions bot removed the automerge Merge the PR when CI passes label Mar 7, 2024

MNT: Re-rendered with conda-build 24.1.2, conda-smithy 3.31.1, and co…

b5a0183

…nda-forge-pinning 2024.03.07.20.43.08

jakirkham reviewed Mar 7, 2024

View reviewed changes

Update meta.yaml

dc6ed15

traversaro added 2 commits March 8, 2024 20:01

Skip on ppc64le with CUDA 12

cb40b85

Update conda-forge.yml

42c14c3

traversaro mentioned this pull request Mar 8, 2024

Support ppc64le + CUDA 12.0 #58

Closed

MNT: Re-rendered with conda-build 24.1.2, conda-smithy 3.31.1, and co…

01d870e

…nda-forge-pinning 2024.03.08.18.51.49

traversaro added the automerge Merge the PR when CI passes label Mar 8, 2024

traversaro merged commit e32d8bf into conda-forge:main Mar 8, 2024
47 of 48 checks passed

regro-cf-autotick-bot deleted the rebuild-cuda120-0-3_h7be56b branch March 8, 2024 23:25

traversaro mentioned this pull request Dec 12, 2024

Cleanup CUDA support #66

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Rebuild for CUDA 12 w/arch + Windows support #56

Rebuild for CUDA 12 w/arch + Windows support #56

regro-cf-autotick-bot commented Nov 17, 2023

conda-forge-webservices bot commented Nov 17, 2023

traversaro commented Nov 17, 2023

traversaro commented Nov 17, 2023

traversaro commented Nov 17, 2023

jakirkham commented Dec 2, 2023

jakirkham commented Dec 2, 2023

jakirkham commented Dec 2, 2023

jakirkham commented Dec 2, 2023

jakirkham commented Dec 2, 2023 •

edited

Loading

traversaro commented Dec 2, 2023

jakirkham commented Jan 4, 2024

jakirkham commented Jan 4, 2024

traversaro commented Mar 7, 2024

jakirkham commented Mar 7, 2024

traversaro commented Mar 7, 2024

traversaro commented Mar 7, 2024

jakirkham left a comment

github-actions bot commented Mar 7, 2024

traversaro commented Mar 7, 2024

github-actions bot commented Mar 7, 2024

traversaro commented Mar 7, 2024

jakirkham Mar 7, 2024

traversaro Mar 7, 2024

traversaro commented Mar 7, 2024

github-actions bot commented Mar 7, 2024

jakirkham commented Mar 8, 2024

traversaro commented Mar 8, 2024

jakirkham commented Mar 8, 2024

traversaro commented Mar 8, 2024

	- cuda-version {{ cuda_compiler_version }} # [(cuda_compiler_version or "None") != "None"]
	- cuda-version {{ cuda_compiler_version }} # [(cuda_compiler_version or "None") != "None"]
	- cuda-cudart-dev # [(cuda_compiler_version or "None") != "None"]

Rebuild for CUDA 12 w/arch + Windows support #56

Rebuild for CUDA 12 w/arch + Windows support #56

Conversation

regro-cf-autotick-bot commented Nov 17, 2023

conda-forge-webservices bot commented Nov 17, 2023

traversaro commented Nov 17, 2023

traversaro commented Nov 17, 2023

traversaro commented Nov 17, 2023

jakirkham commented Dec 2, 2023

jakirkham commented Dec 2, 2023

jakirkham commented Dec 2, 2023

jakirkham commented Dec 2, 2023

jakirkham commented Dec 2, 2023 • edited Loading

traversaro commented Dec 2, 2023

jakirkham commented Jan 4, 2024

jakirkham commented Jan 4, 2024

traversaro commented Mar 7, 2024

jakirkham commented Mar 7, 2024

traversaro commented Mar 7, 2024

traversaro commented Mar 7, 2024

jakirkham left a comment

Choose a reason for hiding this comment

github-actions bot commented Mar 7, 2024

traversaro commented Mar 7, 2024

github-actions bot commented Mar 7, 2024

traversaro commented Mar 7, 2024

jakirkham Mar 7, 2024

Choose a reason for hiding this comment

traversaro Mar 7, 2024

Choose a reason for hiding this comment

traversaro commented Mar 7, 2024

github-actions bot commented Mar 7, 2024

jakirkham commented Mar 8, 2024

traversaro commented Mar 8, 2024

jakirkham commented Mar 8, 2024

traversaro commented Mar 8, 2024

jakirkham commented Dec 2, 2023 •

edited

Loading