Start publishing nightly Python packages

[ScottTodd]

Overview

We have a substantial number of workflows downstream that install iree-turbine from the git source like so:

# Install latest iree-tubrine.
pip install --no-compile -f https://iree.dev/pip-release-links.html --src deps \
    -e "git+https://github.com/iree-org/iree-turbine.git#egg=iree-turbine"

As part of rolling out releases across all IREE ecosystem projects ("upstream": iree-base-compiler, iree-base-runtime, iree-turbine, etc. "downstream": sharktank, shortfin, shark-ai, etc.), we should publish nightly packages for iree-turbine and get them listed in a common pip index page: iree-org/iree#19193.

Once packages are published, we can teach the release index generator in iree-org/iree to scrape them as part of iree-org/iree#19193, then we'll be able to write

# Install iree-turbine nightly (which may pin versions of iree-base-compiler and iree-base-runtime)
pip install -f https://iree.dev/pip-release-links.html --pre iree-turbine

# ... or explicitly install all nightly packages
pip install -f https://iree.dev/pip-release-links.html --pre \
    iree-base-compiler iree-base-runtime iree-turbine

Implementation details

We can fork code from

• https://github.com/nod-ai/shark-ai/blob/main/sharktank/build_tools/build_linux_package.sh (python -m pip wheel and some boilerplate)
• https://github.com/nod-ai/shark-ai/blob/main/.github/workflows/build_packages.yml (compute version --> call build script --> upload)

Those were rewritten from similar files in https://github.com/iree-org/iree and inspired by files in https://github.com/llvm/torch-mlir and https://github.com/openxla/stablehlo. The shark-ai files are the most recently authored that follow the same version scheme as IREE.

The package publication in those pushes to a dev-wheels release like https://github.com/nod-ai/shark-ai/releases/tag/dev-wheels or https://github.com/openxla/stablehlo/releases/tag/dev-wheels, which have pip-compatible index pages like -f https://github.com/nod-ai/shark-ai/releases/expanded_assets/dev-wheels. More complete docs: https://github.com/nod-ai/shark-ai/blob/main/docs/nightly_releases.md

[ScottTodd]

Nightly wheels are now published to https://github.com/iree-org/iree-turbine/releases/tag/dev-wheels