Support installation via pixi (conda-forge)

[humphd]

Problem to solve

I'm working on a project that uses pixi as its package manager and script runner. Pixi is written in Rust, and allows for installation and management of multiple languages, a bit like npm. I'd like to be able to install hurl via pixi:

pixi add hurl
  × could not determine any available versions for hurl on linux-64. Either the package
  │ could not be found or version constraints on other dependencies result in a conflict.
  ╰─▶ Cannot solve the request because of: No candidates where found for hurl.

Proposal

In order to install hurl via pixi, I think hurl has to get added to conda-forge, though I've never done this.

[jcamiel]

Hi @humphd

I haven't heard of pixi nor conda, but it seems interesting! A conda "feedstock" looks a lot like a Homebrew "formula", we just need to read the doc to see how to write one for Hurl.

Thanks for the issue!

[humphd]

This would open up hurl usage to the wider Python, AI/ML, data science worlds. Thanks for considering it!

[jcamiel]

No sure if I've understood all the Conda subtilities (like conda, miniconca, Ananconda etc... 😅) but I've made a first PR for Hurl to conda-forge here => conda-forge/staged-recipes#24265
Let's hope everything will be good!

[jcamiel]

As I feared, Windows build is failing (I'm not surprised), other builds are OK. It's going to take a bit more time...

[humphd]

Amazing, thank you! I'll watch that PR. Appreciate all your efforts to make hurl accessible, as always.

[humphd]

@jcamiel I fear I've caused you a lot of churn trying to get the Windows builds to work on Conda. My apologies. If it's too much hassle, maybe it's not worth it.

[jcamiel]

😅@humphd Don't worry, I've seen other tools on conda-forge like ripgrep so it should be worthwhile. I've made (very little) progress so maybe we're near the end! I'll try a few days, check with @lepapareil if he seems something obvious (he's done all our Windows build before). For the conda "recipe", Linux and macOS has been done in a matter of hours, but Windows is just another league...

[humphd]

It's great that you're willing to fight through the pain, since (like the npm module and docker image), this opens the door for a lot more hurl users that wouldn't normally have it. I think it will be a great addition to the other installation methods.

[jcamiel]

First pass of checks OK for Windows, Linux and Mac 🥳 To be continued!

[jcamiel]

Hi @humphd there has been some progress! The "recipe" has been published on conda forge => https://anaconda.org/conda-forge/hurl
There are still some things to be resolved, I can't see *-arm64 variant (for linux and macos), I only see *-64 build. I'm going to ask how to do this to conda forge peoples, but we're close to completion!

[humphd]

Fantastic! Thanks for all the work thus far, and for following up on the -arm64 variant, which I know I'll need right away.

[jcamiel]

From the conda support, the relevant pointers for arm64 build are here:

• https://conda-forge.org/blog/posts/2020-10-29-macos-arm64/
• https://conda-forge.org/docs/maintainer/knowledge_base.html#cross-compilation
• https://conda-forge.org/docs/maintainer/knowledge_base.html#using-arch-rebuild-txt

For what I've understood I can make a PR to conda-forge-pinning-feedstock to migrate the Hurl recipe to arm64 => conda-forge/conda-forge-pinning-feedstock#5182

Edit: PR has been created on the recipe, just need to figure out how to make the build green => conda-forge/hurl-feedstock#1 😅

[jcamiel]

Hi @humphd I think this is almost completed!

On my M1:

$ conda install -c conda-forge hurl
$ hurl --version
hurl 4.1.0 (aarch64-apple-darwin20.0.0) libcurl/8.4.0 (SecureTransport) OpenSSL/3.2.0 zlib/1.2.13 zstd/1.5.5 libssh2/1.11.0 nghttp2/1.52.0
Features (libcurl):  alt-svc AsynchDNS HSTS HTTP2 IPv6 Largefile libz NTLM SPNEGO SSL TLS-SRP UnixSockets zstd
Features (built-in): brotli

One nice thing is that Hurl "by conda" uses conda provided libcurl which is rather up-to-date (Hurl "by brew" uses the system libcurl).

I'm going to try to add linux-aarch64. Could you try to check if you can install Hurl with pixi?

[humphd]

I'd say this is working well:

$ pixi init
✔ Initialized project in /private/tmp/hurl/.
/tmp/hurl

$ pixi add hurl
✔ Added hurl 4.1.0.*

$ cat pixi.toml
[project]
name = "hurl"
version = "0.1.0"
description = "Add a short description here"
authors = ["David Humphrey <david.andrew.humphrey@gmail.com>"]
channels = ["conda-forge"]
platforms = ["osx-arm64"]

[tasks]

[dependencies]
hurl = "4.1.0.*"

$ .pixi/env/bin/hurl --version
hurl 4.1.0 (aarch64-apple-darwin20.0.0) libcurl/8.4.0 (SecureTransport) OpenSSL/3.2.0 zlib/1.2.13 zstd/1.5.5 libssh2/1.11.0 nghttp2/1.58.0
Features (libcurl):  alt-svc AsynchDNS HSTS HTTP2 IPv6 Largefile libz NTLM SPNEGO SSL TLS-SRP UnixSockets zstd
Features (built-in): brotli

On the project I'm working on, we do use linux-aarch64 for Docker on arm:

platforms = ["linux-64", "linux-aarch64", "osx-64", "osx-arm64"]

When I switch to that, it obviously fails:

$ pixi install
  × Cannot solve the request because of: No candidates where found for hurl
  │ 4.1.0.*.
  │

Really appreciate your hard work on this.

[jcamiel]

Linux-aarch64 builds should be available just now @humphd can you check that you see it now ?

[humphd]

Works!

$ pixi install
✔ Project in /private/tmp/hurl is ready to use!
/tmp/hurl                                                                    7s
$ cat pixi.toml
[project]
name = "hurl"
version = "0.1.0"
description = "Add a short description here"
authors = ["David Humphrey <david.andrew.humphrey@gmail.com>"]
channels = ["conda-forge"]
platforms = ["linux-64", "linux-aarch64", "osx-64", "osx-arm64"]

[tasks]

[dependencies]
hurl = "4.1.0.*"

[humphd]

OK, I've installed this into our project using pixi and am using it in CI. Amazing work!

[jcamiel]

Fixed by #2197.