More documentation for hosted platform integration

[bollwyvl]

Problem description

Looking past "works on my machine," there are a host (ha!) of other places open source developers (at least the ones I hang out with) often look for their free compute and hosting, and answering each of these would help projects make the call to adopt pixi.

Perhaps these could be broken out into a section, similar to IDE integration.

github actions

Great! I see (and have pleasantly used) setup-pixi for GitHub Actions. I don't know what magic sauces is in there, but shucks it's fast with a warm cache. Being able to add a little more log output about what it's doing would be nice for debugging things, but otherwise, extremely pleased.

gitlab

Is also a thing. While I presently don't make use of their cloud offering, I do interact with some hosted instances. They offer n approaches (templates, components, containers, etc.) to reusable CI. Not sure if this is worth the effort, but would probably be appreciated by some.

readthedocs

My next personal stop on the journey will be ReadTheDocs, of which I see no particular reference, or convenient place to put in docs/.

While the curl|bash approach is... fine, it may make sense to rest atop their existing python: mambaforge baseline.

There's probably a way to get the build customization to set up the right pixi env(s), such that an existing mkdocs or sphinx build works as expected with all the magic RTD sauce (elasticsearch, version pickers, etc), but initially something as small as this could be enough to get people going:

version: 2

build:
  os: ubuntu-22.04
  tools:
    python: mambaforge-latest
  commands:
    - mamba install pixi==0.21.1
    - pixi run --environment=rtd rtd

where [feature.rtd.tasks-rtd].cmd could be:

cmd = "sphinx-build -W --color -b html docs $READTHEDOCS_OUTPUT/html"

binder

I see Binder is mentioned in the JupyterLab section of IDE Integration... given the current resource (read: free compute) exhaustion, this is maybe not a great advertising mechanism.

At any rate, eventually I'd like to be able to efficiently use pixi in MyBinder, but... see above. However, getting support for pixi.toml/pixi.lock into repo2docker is likely something that would benefit organizations that self-host, but would take some coordination.

From a repo2docker convention perspective...

I could imagine a text-based convention such as:

• give up if a file called pixi.lock does not exist
• give up if neither:
  • pixi.toml exists and contains tasks.binder and environment.binder
  • pyproject.toml exists and contains the string tool.pixi and tasks.binder and environment.binder
• COPY the pixi.toml (or pyproject.toml) and pixi.lock into the container
• somehow determine and install the minimum pixi version required (see old pixi cannot parse parse project manifests from new version #1346)
  • got this shiny $schema attribute which can contain the version... and give you free DX...
• somehow ensure pixi is installed
  • again, probably bootstrap off the known micromamba on the container to install it instead of curl|bash
• RUN pixi install --environment=binder
• copy the rest of the repo in
• RUN pixi run --environment=binder binder

container

This is kind of a superset of the binder case: getting to a really tight production build of a pixi-built application is pretty clutch for folk that deploy that way.

Some ideas...

• document going through the pixi run constructor route (a la Add pixi list --explict #1216), and deploying that directly with a multi-FROM description
• describe how to use an at-rest .pixi/envs/{env name} as the seed for an container
• some other thing I can't imagine
  • can't be too magic (e.g. FROM pixi) because you can't make everyone happy all the time

Thanks again for pixi!

[wolfv]

Regarding the point on containers - we do have pixi-docker: https://github.com/prefix-dev/pixi-docker (and a few more ideas that we want to work on).

Regarding documenting all this - yes! That would be great.

[bollwyvl]

pixi-docker

Cool, definitely belongs in docs as a starting point... for a dev machine.

But in a production image, with full auditing, one wouldn't want even the 45mb of pixi+deps, much less whatever your FROM happened to be when you published it. Not to be grumpy (clearly I am, just a bit), the musl crew will fight you for every kb, and auditors (or their duly-appointed robot overlords) will stonewall everything that can't be described in an SBOM (e.g. by way of #1216 then jake).

As to the rest: of the above, I can probably make moves on the RTD stuff, but would probably want some guidance on what the maintainers/community see that looking like in the docs tree to be useful.

[tdejager]

Hey @bollwyvl, thanks for the issue! What do you mean with the read-the-docs section as opposed to the documentation we have now?

[bollwyvl]

with the read-the-docs section as

I'm talking about the use case of:

As an open source project that publishes documentation at my-project.readthedocs.io, I want to use pixi to build my documentation.

The upstream documentation describes how to use an environment.yml to create env, at a specific $READTHEDOCS_VIRTUALENV, against which it calls python -m sphinx (or whatever). It's unclear how one would use pixi to get a reproducible documentation build environment, especially if any native apt packages are required, as that is not compatible with the "you-break-it-you-buy-it" commands approach.

[pavelzw]

I also stumbled upon the readthedocs issue when trying to convert to pixi for some project that uses readthedocs.
This needs to be implemented upstream, though

[pavelzw]

Re GitHub actions

Being able to add a little more log output about what it's doing would be nice for debugging things

I specifically implemented it in a way that the logs are very slim and easy to read in setup-pixi. You can enable debug logging to see what's going on under the hood and get more logs from setup-pixi.

[bollwyvl]

#1408 starts the RTD part and groups "not on my computer" stuff under Advanced // Third-Party Integrations

[pavelzw]

Re docker: I'm writing a blog post explaining how to use pixi projects dockerized since it fits a bit better there than in the original documentation. Should be ready next Tuesday or Wednesday

[pavelzw]

For readthedocs: i would guess it's easiest to just use their official ubuntu image and add pixi via asdf as in their uv example.

Depends on asdf-vm/asdf-plugins#1013

Then have something like

version: 2
build:
  os: ubuntu-22.04
  commands:
    - asdf plugin add pixi
    - asdf install pixi latest
    - asdf global pixi latest
    - pixi run build-docs
    - mkdir -p $READTHEDOCS_OUTPUT/html/
    - cp -r docs/_build/html/** $READTHEDOCS_OUTPUT/html/

[saulshanabrook]

Note that I opened readthedocs/readthedocs.org#11548 to see if pixi support could be added to RTD itself.