Pkg 3792: initial feedstock, v12.3 #1

danpetry · 2024-02-05T23:00:10Z

Destination channel: defaults

Links

PKG-3792

Explanation of changes:

Third package of CUDA 12.3 build, following conda-forge's pattern which was developed in conjunction with Nvidia. I don't see a need to stage these then release them all at once. They won't cause a problem released one at a time.
Please check the main branch too, this has just been forked
Uploads to private anaconda.org channel because we don't have the license agreement for many of the packages in this buildout, including upstreams. (this particular package we have a license for)
Also removes things that aren't applicable in defaults
- binary patching to define $ORIGIN as runpath, as it's not needed because the patched binaries don't link against any conda-provided .sos (although I'm not sure why this was done in conda-forge either, awaiting info)
- adjust required glibc on aarch64, it's different between us and conda-forge

We don't have arm-variant as a package in defaults. It's left in run_constrained to support compatibility when mixed with conda-forge packages.

anaconda-pkg-build · 2024-02-05T23:01:07Z

Linter check found the following problems:

ERROR conda.cli.main_run:execute(124): `conda run conda-lint /tmp/abs_877lnzy2z6/clone` failed. (See above for error) The following problems have been found:

===== WARNINGS =====

clone/recipe/meta.yaml:336: license_file_overspecified: Using license_file and license_url is overspecified.

===== ERRORS =====

clone/recipe/meta.yaml:24: missing_hash: The recipe is missing a sha256 checksum for a source file
clone/recipe/meta.yaml:220: missing_tests: output "cuda-cudart-static": No tests were found.
clone/recipe/meta.yaml:24: invalid_url: https://developer.download.nvidia.com/compute/cuda/redist/cuda_cudart/linux/cuda_cudart-linux-12.3.101-archive.tar.xz : Not reachable: 404
clone/recipe/meta.yaml:138: missing_tests: output "cuda-cudart-dev": No tests were found.
clone/recipe/meta.yaml:336: missing_license_family: The recipe is missing the about/license_family key.
clone/recipe/meta.yaml:336: missing_dev_url: The recipe is missing a dev_url
===== Final Report: =====
6 Errors and 1 Warning were found

anaconda-pkg-build · 2024-02-06T21:38:09Z

Linter check found the following problems:

ERROR conda.cli.main_run:execute(124): `conda run conda-lint /tmp/abs_6ezi4pxqi2/clone` failed. (See above for error) The following problems have been found:

===== WARNINGS =====

clone/recipe/meta.yaml:336: license_file_overspecified: Using license_file and license_url is overspecified.

===== ERRORS =====

clone/recipe/meta.yaml:336: missing_license_family: The recipe is missing the about/license_family key.
clone/recipe/meta.yaml:220: missing_tests: output "cuda-cudart-static": No tests were found.
clone/recipe/meta.yaml:138: missing_tests: output "cuda-cudart-dev": No tests were found.
clone/recipe/meta.yaml:336: missing_dev_url: The recipe is missing a dev_url
===== Final Report: =====
4 Errors and 1 Warning were found

…ired rpath functionality

anaconda-pkg-build · 2024-02-07T02:16:31Z

Linter check found the following problems:

ERROR conda.cli.main_run:execute(124): `conda run conda-lint /tmp/abs_b3m9falo3d/clone` failed. (See above for error) The following problems have been found:

===== WARNINGS =====

clone/recipe/meta.yaml:336: license_file_overspecified: Using license_file and license_url is overspecified.

===== ERRORS =====

clone/recipe/meta.yaml:336: missing_license_family: The recipe is missing the about/license_family key.
clone/recipe/meta.yaml:220: missing_tests: output "cuda-cudart-static": No tests were found.
clone/recipe/meta.yaml:138: missing_tests: output "cuda-cudart-dev": No tests were found.
clone/recipe/meta.yaml:336: missing_dev_url: The recipe is missing a dev_url
===== Final Report: =====
4 Errors and 1 Warning were found

…with desired rpath functionality" This reverts commit d447b25.

This reverts commit d537729.

anaconda-pkg-build · 2024-02-07T02:56:44Z

Linter check found the following problems:

ERROR conda.cli.main_run:execute(124): `conda run conda-lint /tmp/abs_4bk6mnet7b/clone` failed. (See above for error) The following problems have been found:

===== WARNINGS =====

clone/recipe/meta.yaml:336: license_file_overspecified: Using license_file and license_url is overspecified.

===== ERRORS =====

clone/recipe/meta.yaml:220: missing_tests: output "cuda-cudart-static": No tests were found.
clone/recipe/meta.yaml:138: missing_tests: output "cuda-cudart-dev": No tests were found.
clone/recipe/meta.yaml:336: missing_dev_url: The recipe is missing a dev_url
clone/recipe/meta.yaml:336: missing_license_family: The recipe is missing the about/license_family key.
===== Final Report: =====
4 Errors and 1 Warning were found

anaconda-pkg-build · 2024-02-07T17:59:26Z

Linter check found the following problems:

ERROR conda.cli.main_run:execute(124): `conda run conda-lint /tmp/abs_c7lix3469g/clone` failed. (See above for error) The following problems have been found:

===== WARNINGS =====

clone/recipe/meta.yaml:336: license_file_overspecified: Using license_file and license_url is overspecified.

===== ERRORS =====

clone/recipe/meta.yaml:336: missing_license_family: The recipe is missing the about/license_family key.
clone/recipe/meta.yaml:220: missing_tests: output "cuda-cudart-static": No tests were found.
clone/recipe/meta.yaml:336: missing_dev_url: The recipe is missing a dev_url
clone/recipe/meta.yaml:138: missing_tests: output "cuda-cudart-dev": No tests were found.
===== Final Report: =====
4 Errors and 1 Warning were found

…ious commit)

anaconda-pkg-build · 2024-02-07T18:11:34Z

Linter check found the following problems:

ERROR conda.cli.main_run:execute(124): `conda run conda-lint /tmp/abs_36i7vje2b1/clone` failed. (See above for error) The following problems have been found:

===== WARNINGS =====

clone/recipe/meta.yaml:336: license_file_overspecified: Using license_file and license_url is overspecified.

===== ERRORS =====

clone/recipe/meta.yaml:336: missing_license_family: The recipe is missing the about/license_family key.
clone/recipe/meta.yaml:138: missing_tests: output "cuda-cudart-dev": No tests were found.
clone/recipe/meta.yaml:336: missing_dev_url: The recipe is missing a dev_url
clone/recipe/meta.yaml:220: missing_tests: output "cuda-cudart-static": No tests were found.
===== Final Report: =====
4 Errors and 1 Warning were found

…conda-forge's

anaconda-pkg-build · 2024-02-07T18:55:19Z

Linter check found the following problems:

ERROR conda.cli.main_run:execute(124): `conda run conda-lint /tmp/abs_f54ap9he2m/clone` failed. (See above for error) The following problems have been found:

===== WARNINGS =====

clone/recipe/meta.yaml:336: license_file_overspecified: Using license_file and license_url is overspecified.

===== ERRORS =====

clone/recipe/meta.yaml:220: missing_tests: output "cuda-cudart-static": No tests were found.
clone/recipe/meta.yaml:138: missing_tests: output "cuda-cudart-dev": No tests were found.
clone/recipe/meta.yaml:336: missing_dev_url: The recipe is missing a dev_url
clone/recipe/meta.yaml:336: missing_license_family: The recipe is missing the about/license_family key.
===== Final Report: =====
4 Errors and 1 Warning were found

JeanChristopheMorinPerso · 2024-02-07T19:23:22Z

recipe/build.sh

+                # the runtime dependencies are packaged in their ecosystem.
+                # For defaults, they're in the default system lib location.
+                # if [[ $j =~ \.so\. ]]; then
+                #     patchelf --set-rpath '$ORIGIN' ${PREFIX}/${targetsDir}/$j


I think we should avoid diverging from conda-forge too much. Having $ORIGIN in RUNPATH doesn't hurt in this case. NVidia might actually have a reason to prefer RUNPATH over RPATH here (because conda-build will set RPATH to $ORIGIN/. automatically).

So if anything, we should wait to get an answer on conda-forge#21 before proceeding with this change.

I'll wait for the answer from conda-forge

From conda-forge#21 (and conda-forge/cuda-feedstock#10), we need to keep this. We could create an issue on the cuda feedstock to suggest using --force-rpath which would silence conda-build.

yeah. This would also change precedence of $ORIGIN related to LD_LIBRARY_PATH so not sure whether that makes sense (it might do).

this is done

JeanChristopheMorinPerso · 2024-02-07T19:26:33Z

recipe/conda_build_config.yaml

@@ -1,2 +1,5 @@
 arm_variant_type: # [aarch64]
  - sbsa          # [aarch64]
+conda_glibc_ver:
+  - 2.17          # [not aarch64]


I feel like this is the kind of thing we should probably define in aggregate?

it's defined in package cbcs for everything that uses it (various compiler related packages). But yeah I agree. Would prefer this not to be blocked on that though, and move them all out at some later opportunity. Concretely, I'd generate a jira ticket for it, and if needed an epic for cbc reviewing. Would you be ok with that?

You know what, I'm even wondering we should shouldn't just do >=2.17 on both x86_64 and aarch64. If something is compiled with glibc 2.17, it will work on anything greater than that version. Like I don't know why they hard pin glibc there.

Do you know why they specify compilers and sysroot in this feedstock anyway? It's just binary repackaging.

I had this exact question with AnacondaRecipes/file-feedstock#1 yesterday and the only thing I can think of is that it's to make the DSO check happy.

I asked that here

I haven't had a reply and this is in line with other feedstocks (example). Would you be ok with keeping it in? Seems less risky if we don't know the reason. Would like to get the PR moving along. Don't think it's critical to unpin.

anaconda-pkg-build · 2024-02-07T23:44:14Z

Linter check found the following problems:

ERROR conda.cli.main_run:execute(124): `conda run conda-lint /tmp/abs_2208zcuso_/clone` failed. (See above for error) The following problems have been found:

===== WARNINGS =====

clone/recipe/meta.yaml:337: license_file_overspecified: Using license_file and license_url is overspecified.

===== ERRORS =====

clone/recipe/meta.yaml:337: missing_license_family: The recipe is missing the about/license_family key.
clone/recipe/meta.yaml:337: missing_dev_url: The recipe is missing a dev_url
clone/recipe/meta.yaml:139: missing_tests: output "cuda-cudart-dev": No tests were found.
clone/recipe/meta.yaml:221: missing_tests: output "cuda-cudart-static": No tests were found.
===== Final Report: =====
4 Errors and 1 Warning were found

This reverts commit be3fd48.

…see previous commit)" This reverts commit 8620c3e.

…tem." This reverts commit 4b349f0.

anaconda-pkg-build · 2024-02-13T19:09:05Z

Linter check found the following problems:

ERROR conda.cli.main_run:execute(124): `conda run conda-lint /tmp/abs_5dtu9q68v1/clone` failed. (See above for error) The following problems have been found:

===== WARNINGS =====

clone/recipe/meta.yaml:336: license_file_overspecified: Using license_file and license_url is overspecified.

===== ERRORS =====

clone/recipe/meta.yaml:220: missing_tests: output "cuda-cudart-static": No tests were found.
clone/recipe/meta.yaml:336: missing_license_family: The recipe is missing the about/license_family key.
clone/recipe/meta.yaml:336: missing_dev_url: The recipe is missing a dev_url
clone/recipe/meta.yaml:138: missing_tests: output "cuda-cudart-dev": No tests were found.
===== Final Report: =====
4 Errors and 1 Warning were found

anaconda-pkg-build · 2024-02-13T19:19:06Z

Linter check found the following problems:

ERROR conda.cli.main_run:execute(124): `conda run conda-lint /tmp/abs_208sijomxy/clone` failed. (See above for error) The following problems have been found:

===== WARNINGS =====

clone/recipe/meta.yaml:336: license_file_overspecified: Using license_file and license_url is overspecified.

===== ERRORS =====

clone/recipe/meta.yaml:138: missing_tests: output "cuda-cudart-dev": No tests were found.
clone/recipe/meta.yaml:336: missing_dev_url: The recipe is missing a dev_url
clone/recipe/meta.yaml:220: missing_tests: output "cuda-cudart-static": No tests were found.
clone/recipe/meta.yaml:336: missing_license_family: The recipe is missing the about/license_family key.
===== Final Report: =====
4 Errors and 1 Warning were found

cbouss

LGTM, the rpath story is interesting

skupr-anaconda · 2024-02-19T10:47:34Z

Looks like the doc_url is https://nvidia.github.io/cuda-python/module/cudart.html and dev_url is https://github.com/NVIDIA/cuda-python

JeanChristopheMorinPerso · 2024-02-19T15:00:56Z

@skupr-anaconda no, this feedstock is not python related. This feedstock is about cudart from the cudatoolkit.

anaconda-pkg-build · 2024-02-19T20:22:15Z

Linter check found the following problems:

ERROR conda.cli.main_run:execute(124): `conda run conda-lint /tmp/abs_6afpp0omz8/clone` failed. (See above for error) The following problems have been found:

===== WARNINGS =====

clone/recipe/meta.yaml:336: license_file_overspecified: Using license_file and license_url is overspecified.

===== ERRORS =====

clone/recipe/meta.yaml:336: missing_dev_url: The recipe is missing a dev_url
clone/recipe/meta.yaml:138: missing_tests: output "cuda-cudart-dev": No tests were found.
clone/recipe/meta.yaml:220: missing_tests: output "cuda-cudart-static": No tests were found.
clone/recipe/meta.yaml:336: missing_license_family: The recipe is missing the about/license_family key.
===== Final Report: =====
4 Errors and 1 Warning were found

anaconda-pkg-build · 2024-02-19T21:22:17Z

Linter check found the following problems:

ERROR conda.cli.main_run:execute(124): `conda run conda-lint /tmp/abs_15jxi0_64s/clone` failed. (See above for error) The following problems have been found:

===== WARNINGS =====

clone/recipe/meta.yaml:336: license_file_overspecified: Using license_file and license_url is overspecified.

===== ERRORS =====

clone/recipe/meta.yaml:138: missing_tests: output "cuda-cudart-dev": No tests were found.
clone/recipe/meta.yaml:336: missing_dev_url: The recipe is missing a dev_url
clone/recipe/meta.yaml:220: missing_tests: output "cuda-cudart-static": No tests were found.
clone/recipe/meta.yaml:336: missing_license_family: The recipe is missing the about/license_family key.
===== Final Report: =====
4 Errors and 1 Warning were found

danpetry added 2 commits February 5, 2024 16:46

Add abs.yaml which uploads the packages to a private channel

61e2bc2

Remove conda-forge build requirement on specific arm architecture

56fce9f

We don't have arm-variant as a package in defaults. It's left in run_constrained to support compatibility when mixed with conda-forge packages.

danpetry added 2 commits February 5, 2024 17:03

Skip s390x, CUDA doesn't support it

25daf28

set -ex in build.sh for debugging

d537729

Remove runpath; conda-build gives an error and it would mess with des…

d447b25

…ired rpath functionality

danpetry added 3 commits February 6, 2024 20:53

Revert "Remove runpath; conda-build gives an error and it would mess …

b95ff39

…with desired rpath functionality" This reverts commit d447b25.

Revert "set -ex in build.sh for debugging"

4d9a1fb

This reverts commit d537729.

Remove runpath entries; they aren't needed in defaults ecosystem.

4b349f0

danpetry closed this Feb 7, 2024

danpetry reopened this Feb 7, 2024

Remove rpath tests, because we're not patching the binaries (see prev…

8620c3e

…ious commit)

Adjust supported glibc version on linux-aarch64; it's different from …

b6386d8

…conda-forge's

JeanChristopheMorinPerso reviewed Feb 7, 2024

View reviewed changes

Build unbuilt packages for win

be3fd48

danpetry changed the title ~~Pkg 3792: initial feedstock, v12.3~~ Pkg 3792: initial feedstock, v12.3 skip CI Feb 9, 2024

danpetry added 4 commits February 9, 2024 17:33

Revert "Build unbuilt packages for win"

532e4d9

This reverts commit be3fd48.

Revert "Remove rpath tests, because we're not patching the binaries (…

14de473

…see previous commit)" This reverts commit 8620c3e.

Revert "Remove runpath entries; they aren't needed in defaults ecosys…

5907f27

…tem." This reverts commit 4b349f0.

Don't do an overlinking check to avoid the error check for as rpath

f2a9914

danpetry changed the title ~~Pkg 3792: initial feedstock, v12.3 skip CI~~ Pkg 3792: initial feedstock, v12.3 Feb 13, 2024

Don't upload to staging channels

8fea804

need to get the deps from the channel though

863565a

SC426 approved these changes Feb 13, 2024

View reviewed changes

cbouss approved these changes Feb 13, 2024

View reviewed changes

JeanChristopheMorinPerso approved these changes Feb 14, 2024

View reviewed changes

danpetry changed the title ~~Pkg 3792: initial feedstock, v12.3~~ Pkg 3792: initial feedstock, v12.3 skip CI Feb 19, 2024

Update abs.yaml for merge commit

b655e02

danpetry changed the title ~~Pkg 3792: initial feedstock, v12.3 skip CI~~ Pkg 3792: initial feedstock, v12.3 Feb 19, 2024

danpetry merged commit 09d15ed into main Feb 19, 2024
1 check passed

danpetry deleted the PKG-3792-update-for-defaults branch February 19, 2024 20:21

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Pkg 3792: initial feedstock, v12.3 #1

Pkg 3792: initial feedstock, v12.3 #1

danpetry commented Feb 5, 2024 •

edited

Loading

anaconda-pkg-build commented Feb 5, 2024

anaconda-pkg-build commented Feb 6, 2024

anaconda-pkg-build commented Feb 7, 2024

anaconda-pkg-build commented Feb 7, 2024

anaconda-pkg-build commented Feb 7, 2024

anaconda-pkg-build commented Feb 7, 2024

anaconda-pkg-build commented Feb 7, 2024

JeanChristopheMorinPerso Feb 7, 2024

danpetry Feb 7, 2024

JeanChristopheMorinPerso Feb 8, 2024

danpetry Feb 9, 2024

danpetry Feb 13, 2024

JeanChristopheMorinPerso Feb 7, 2024

danpetry Feb 7, 2024 •

edited

Loading

JeanChristopheMorinPerso Feb 8, 2024

danpetry Feb 9, 2024

JeanChristopheMorinPerso Feb 9, 2024

danpetry Feb 10, 2024

danpetry Feb 10, 2024

danpetry Feb 13, 2024

anaconda-pkg-build commented Feb 7, 2024

anaconda-pkg-build commented Feb 13, 2024

anaconda-pkg-build commented Feb 13, 2024

cbouss left a comment

skupr-anaconda commented Feb 19, 2024

JeanChristopheMorinPerso commented Feb 19, 2024

anaconda-pkg-build commented Feb 19, 2024

anaconda-pkg-build commented Feb 19, 2024

Pkg 3792: initial feedstock, v12.3 #1

Pkg 3792: initial feedstock, v12.3 #1

Conversation

danpetry commented Feb 5, 2024 • edited Loading

Links

Explanation of changes:

anaconda-pkg-build commented Feb 5, 2024

anaconda-pkg-build commented Feb 6, 2024

anaconda-pkg-build commented Feb 7, 2024

anaconda-pkg-build commented Feb 7, 2024

anaconda-pkg-build commented Feb 7, 2024

anaconda-pkg-build commented Feb 7, 2024

anaconda-pkg-build commented Feb 7, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

danpetry Feb 7, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

anaconda-pkg-build commented Feb 7, 2024

anaconda-pkg-build commented Feb 13, 2024

anaconda-pkg-build commented Feb 13, 2024

cbouss left a comment

Choose a reason for hiding this comment

skupr-anaconda commented Feb 19, 2024

JeanChristopheMorinPerso commented Feb 19, 2024

anaconda-pkg-build commented Feb 19, 2024

anaconda-pkg-build commented Feb 19, 2024

danpetry commented Feb 5, 2024 •

edited

Loading

danpetry Feb 7, 2024 •

edited

Loading