Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add conda-forge integration #465

Merged
merged 8 commits into from
Jan 17, 2024
Merged

add conda-forge integration #465

merged 8 commits into from
Jan 17, 2024

Conversation

0xbe7a
Copy link
Contributor

@0xbe7a 0xbe7a commented Jan 6, 2024

Until anaconda/anaconda-client#696 is merged, anaconda-client cannot be used to upload packages with conda-forge-ci-setup. This PR adds a conda-forge upload as a special case of the anaconda upload, which makes a copy request after the upload.

conda-forge-ci-setup also uses conda-build under the hood, so this allows testing of rattler-build for conda-forge packages without being blocked by conda-build. In the long run, we probably want to integrate rattler-build properly into conda-forge-ci-setup / fix existing problems of conda-build / anaconda-client when uploading packages built by rattler-build

@0xbe7a 0xbe7a marked this pull request as ready for review January 6, 2024 17:23
@0xbe7a 0xbe7a marked this pull request as draft January 6, 2024 17:35
@0xbe7a
Copy link
Contributor Author

0xbe7a commented Jan 6, 2024

Currently the channel / label is hardcoded (see https://github.com/conda-forge/polarify-feedstock/blob/rattler-build/.scripts/build_steps.sh#L83). Do you think we could we could read this information from the package itself / include it in the package @wolfv ? I'm not quite sure if the "label / channel" concept is anaconda.org or a general conda concept and if it should leak into the package format.

The other two ways I see to give the information to the upload command is to run yq during the build and parse the conda_build_config itself or to do that at render time with conda_smithy and templatize it into the command as an argument

@wolfv
Copy link
Member

wolfv commented Jan 16, 2024

Isn't the channel / label stuff something that conda-smithy would handle? It's got to be somewhere passed in to the recipe.

Instead of using yg you can also just use the YAML parser of rattler-build. The .ci_config/...yaml files are already evaluated so that they contain "pure YAML" and should be easy to parse (as long as you select the proper one).

At the end of the day, a label is just a channel. Personally not a huge fan of labels, so if we could only talk about channels I would probably prefer it but since this code is anaconda specific anyways, i don't mind.

@0xbe7a 0xbe7a marked this pull request as ready for review January 16, 2024 21:15
@0xbe7a
Copy link
Contributor Author

0xbe7a commented Jan 16, 2024

I took a look at how conda-forge-ci-setup does it, and they parse the variant file and use the channel_targets field to determine the upload channel. Since the variant file is contained in the info/recipe folder of the package, we can determine the target channel directly from the package

@wolfv
Copy link
Member

wolfv commented Jan 16, 2024

great, will merge!

Just a side note: I think at some point we should refactor all the upload methods to return proper errors because some people might want to use the methods in a library (or from Python, when we have the bindings). But for now, miette does a nice job :)

@wolfv wolfv merged commit 85d9547 into prefix-dev:main Jan 17, 2024
11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants