Update build infrastructure to hatch

[philippjfr]

Attempt at #4428

This PR removes pyctdev, pyct, doit, tox and replaces it all with hatch. The new system has the following commands:

Commands

• hatch build -t wheel: Builds wheels
• hatch build -t sdist: Builds sdist
• hatch run test.py3.10:run: Runs tests on python 3.10
• hatch run test-ui.py3.10:run: Runs UI tests on python 3.10
• hatch run docs:build
• hatch run docs:serve
• hatch run jupyterlite:build
• hatch run jupyterlite:serve
• hatch -e dev.py310 shell: Set up Python 3.10 conda environment and launch into shell
• hatch -e pip.py310 shell: Set up Python 3.10 pip environment and launch into shell

Test Matrix

                            Standalone
┏━━━━━━━━━━━━━┳━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Name        ┃ Type    ┃ Features ┃ Dependencies ┃ Scripts       ┃
┡━━━━━━━━━━━━━╇━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ default     │ virtual │          │              │               │
├─────────────┼─────────┼──────────┼──────────────┼───────────────┤
│ docs        │ conda   │ doc      │              │ build         │
│             │         │          │              │ build-pyodide │
│             │         │          │              │ generate-rst  │
│             │         │          │              │ refmanual     │
│             │         │          │              │ serve         │
├─────────────┼─────────┼──────────┼──────────────┼───────────────┤
│ jupyterlite │ virtual │          │ jupyter      │ build         │
│             │         │          │ jupyterlite  │               │
└─────────────┴─────────┴──────────┴──────────────┴───────────────┘
                                       Matrices
┏━━━━━━━━━┳━━━━━━━━━┳━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━┓
┃ Name    ┃ Type    ┃ Envs           ┃ Features    ┃ Dependencies    ┃ Scripts        ┃
┡━━━━━━━━━╇━━━━━━━━━╇━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━┩
│ test    │ conda   │ test.py3.8     │ examples    │ python-graphviz │ run            │
│         │         │ test.py3.9     │ recommended │ vtk==9.0.1      │ run-coverage   │
│         │         │ test.py3.10    │ tests       │ xgboost         │ run-examples   │
│         │         │ test.py3.11    │             │                 │                │
│         │         │ test.py3.12    │             │                 │                │
├─────────┼─────────┼────────────────┼─────────────┼─────────────────┼────────────────┤
│ test-ui │ conda   │ test-ui.py3.8  │ recommended │ python-graphviz │ run            │
│         │         │ test-ui.py3.9  │ tests       │ vtk==9.0.1      │ run-coverage   │
│         │         │ test-ui.py3.10 │ ui          │ xgboost         │ run-examples   │
│         │         │ test-ui.py3.11 │             │                 │ start-jupyter  │
│         │         │ test-ui.py3.12 │             │                 │                │
├─────────┼─────────┼────────────────┼─────────────┼─────────────────┼────────────────┤
│ pip     │ virtual │ pip.py3.8      │ all         │                 │ bundle         │
│         │         │ pip.py3.9      │             │                 │ example-test   │
│         │         │ pip.py3.10     │             │                 │ lint           │
│         │         │ pip.py3.11     │             │                 │ pyodide-wheels │
│         │         │ pip.py3.12     │             │                 │ ui-test        │
│         │         │                │             │                 │ unit-test      │
├─────────┼─────────┼────────────────┼─────────────┼─────────────────┼────────────────┤
│ dev     │ conda   │ dev.py3.8      │ all         │                 │ bundle         │
│         │         │ dev.py3.9      │             │                 │ example-test   │
│         │         │ dev.py3.10     │             │                 │ lint           │
│         │         │ dev.py3.11     │             │                 │ pyodide-wheels │
│         │         │ dev.py3.12     │             │                 │ ui-test        │
│         │         │                │             │                 │ unit-test      │
└─────────┴─────────┴────────────────┴─────────────┴─────────────────┴────────────────┘

ToDo

• Fix conda recipe
• Write updated developer instructions

[philippjfr]

pre-commit.ci autofix

[cdeil]

On MacOS I get this error with this branch ( e8a17e0 ).

$ conda activate base
$ pip install hatch hatch-conda
$ hatch -e dev.py3.10 shell
pre-install [1] | pip install toml PyYAML
Collecting toml
  Using cached toml-0.10.2-py2.py3-none-any.whl (16 kB)
Collecting PyYAML
  Using cached PyYAML-6.0-cp310-cp310-macosx_10_9_x86_64.whl (197 kB)
Installing collected packages: toml, PyYAML
Successfully installed PyYAML-6.0 toml-0.10.2

pre-install [2] | python scripts/generate_conda_yaml.py -c conda-forge pyviz/label/dev -o conda-extras doc examples recommended tests ui --pip-only pip-extras
pre-install [3] | conda env update --file environment.yaml

ResolvePackageNotFound: 
  - python-python-graphviz


ERROR conda.cli.main_run:execute(47): `conda run conda env update --file environment.yaml` failed. (See above for error)
Collecting package metadata (repodata.json): ...working... done
Solving environment: ...working... failed

Failed with exit code: 1

[MarcSkovMadsen]

One easy way to contribute at the moment is the docs.

I cant get sphinx running because it runs for +30 mins and sphinx-autobuild creates ever growing recursive directories. But you can make smaller improvements directly to the text, run pre-commit and push.

Another easy way to contribute is help the community on Discourse. Most effort id directed to the release and not support at the moment. 👍

[maximlt]

TODO list:

• Integrate the suggestions from Simon and others
• Add stuff to build a conda package for the pyviz channel
• The build-pyodide command doesn't work on my laptop, the globs like examples/gallery/*.ipynb aren't expanded
• What's the bootstrapping story? I.e. how and where do people install hatch and hatch-conda? For example, my setup on Macos with Python 3.11 installed with homebrew and hatch with pipx led to errors. I had to create a virtual env and install it in there instead.
• Re-add panel examples
• Clean up pyctdev stuff (e.g. dodo.py)
• Re-enable environment caching
• On Windows conda install ... commands don't seem to work at all
• Add workflow that exerts the pip env
• Fix hanging tests on Linux
• Compare the built wheels and conda packages with those built previously with pyctdev
• Attempt to run docs build on Windows/Macos
• Make sure coverage is run correctly
• Enable libmamba solver
• ...