Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add new conda environments #120

Open
wants to merge 2 commits into
base: branch-0.35
Choose a base branch
from

Conversation

pentschev
Copy link
Member

UCXX is not necessarily CUDA dependent. This change adds two new environments: cpp.yaml for C++-only builds and all.yaml for C++ and Python builds.

@pentschev pentschev requested a review from a team as a code owner November 7, 2023 15:22
@pentschev pentschev force-pushed the additional-conda-environments branch from e454494 to af5b587 Compare November 7, 2023 15:25
$ mamba env create -n ucxx -f conda/environments/ucxx-cuda118_arch-x86_64.yml
$ conda activate ucxx
$ mamba env create -n ucxx -f conda/environments/all_cuda-118_arch-x86_64.yaml
$ conda activate all_cuda-118_arch-x86_64
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should match the name from -n ucxx above.

Suggested change
$ conda activate all_cuda-118_arch-x86_64
$ conda activate ucxx

@@ -0,0 +1,34 @@
# This file is generated by `rapids-dependency-file-generator`.
Copy link
Contributor

@bdice bdice Nov 7, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All environment files need to be generated by dependencies.yaml.

The failure in pre-commit is because rapids-dependency-file-generator looks for files that it generated and removes them before re-generating the new files. You have the header here saying it's a generated file, but there's no declaration of this environment file in dependencies.yaml.

This is how you tell it to generate a given environment file:

ucxx/dependencies.yaml

Lines 3 to 17 in 4464ca9

all:
output: conda
matrix:
cuda: ["11.8", "12.0"]
arch: [x86_64]
includes:
- build_cpp
- build_python
- checks
- cudatoolkit
- dev
- py_version
- run_python
- test_cpp
- test_python

For example, you might want to add a section like this for the C++ dependency list.

files:
  cpp:     # This is used as part of the environment's filename
    output: conda
    matrix:
      arch: [x86_64]
    includes:
      - build_cpp
      - checks
      - dev
      - test_cpp

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks Bradley, this is helpful. I'd still like to trim that more though, for example the above will result in:

# This file is generated by `rapids-dependency-file-generator`.
# To make changes, edit ../../dependencies.yaml and run `rapids-dependency-file-generator`.
channels:
- rapidsai
- rapidsai-nightly
- dask/label/dev
- conda-forge
- nvidia
dependencies:
- autoconf
- automake
- c-compiler
- cmake>=3.26.4
- cxx-compiler
- dask-cuda==23.12.*
- dask-cudf==23.12.*
- fmt>=9.1.0,<10
- gmock>=1.13.0
- gtest>=1.13.0
- librmm==23.12.*
- libtool
- ninja
- pip
- pkg-config
- pre-commit
- spdlog>=1.11.0,<1.12
name: cpp_arch-x86_64

That include a bunch of things we don't need: rapidsai/rapidsai-nightly/dask/label/dev/nvidia channels and dask-cuda/dask-cudf/librmm/pip/pkg-config. Is there a way to minimize that?

Additionally, this is supposed to be valid for aarch64 too, can we make it architecture agnostic?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's currently no way to drop extra channels from the generated files. You can file an issue to rapids-dependency-file-generator if you have a proposal for how the dependencies.yaml schema should accommodate that idea, but I'm not sure if it's worth changing. https://github.com/rapidsai/dependency-file-generator

If you depend only on architecture-agnostic packages, you can just drop the arch from the files: spec.

files:
  cpp:     # This is used as part of the environment's filename
    output: conda
    includes:
      - build_cpp
      - checks
      - dev
      - test_cpp

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants