Move PyChaste Bindings into Chaste Trunk

[kwabenantim]

Summary
It might be a good idea to consider moving the Python bindings into the main Chaste trunk as the bindings are tightly coupled to upstream Chaste source.

Advantages:

• Easier to keep bindings in synch with develop branch and release versions.
• Bindings tests can be added to the CI on PRs to check if bindings break when other parts of Chaste are updated.
• Simpler to maintain a single documentation website.

Disadvantages:

• All developers will need to learn some python and pybind11 as well to test changes to bindings?

[kwabenantim]

We have concluded that this is a good idea.

To Do

• Release PyChaste 2024.1 #62
• Fix Automatic Wrapper Generator Bugs #40
• Test Chaste/PyChaste Diff #48
• Add PyChaste Support for VTK 8 #23
• Add PyChaste Support for VTK 9 #24
• Syntactic Sugar for Templated Classes cppwg#13
• Add PyChaste Test Suite to Chaste CI #49
• Move PyChaste website #34
• Add Bindings Tutorials #47

[kwabenantim]

Discussion @ Sheffield Hackathon

Coverage Testing

Add tests to check if there are new classes in cell_based without wrappers

Test against comprehensive list of supported use cases (100% coverage shouldn't be required initially for wrappers).

Configuring

Pybind11
Do not add Pybind11 code into trunk

Preferences in order:

1. Use scikit-build (link) to get pybind11
2. Use cmake fetchcontent
3. Do a pip install into the chaste_codegen Python env (rename env?)

Pull latest pybind11 stable version to keep up to date with latest Python and C++ version support.

cppwg
Do a pip install into the chaste_codegen Python env

Conda packaging

Add Chaste developers to anaconda pychaste channel

Use GitHub actions for building conda packages

Build for Linux/MacOS amd64 and arm64 (Linux arm64 on self-hosted)

Docker

Merge into Chaste docker

Developer Documentation

• Generating wrappers automatically
• Explanation of contents of configuration file
• Code inside wrappers
• Additional code e.g. for custom PyChaste visualisation
• Fixing common problems

[kwabenantim]

Directory structure:

Chaste/pychaste
│
├── dynamic
│   ├── config.yaml
│   ├── templates
│   └── wrappers
│       └── lib
│           ├── *.cppwg.cpp
│           └── *.cppwg.hpp
├── src
│   ├── cpp
│   │   ├── cell_based
│   │   └── visualization
│   └── py
│       ├── chaste
│       │   ├── cell_based
│       │   ├── core
│       │   ├── external
│       │   ├── mesh
│       │   ├── ode
│       │   ├── pde
│       │   └── visualization
│       ├── docs
│       │   ├── api
│       │   └── tutorial
│       ├── LICENSE
│       ├── MANIFEST.in
│       ├── pyproject.toml
│       ├── setup.cfg
│       └── setup.py
└── test
    ├── cell_based
    ├── core
    ├── data
    ├── mesh
    ├── ode
    ├── tutorial
    └── visualization

[kwabenantim]

Notes from Nottingham Hackathon

• Add xvfb to recommended packages in chaste-dependencies
• More informative error message about files and methods that needs changing
• Error message should say how to fix the problem
• Put PyChaste dev documentation on developer wiki
• Update PyChaste web pages (https://github.com/Chaste/Chaste.github.io/pull/101)

• Create a pychaste-dependencies ubuntu package in addition to chaste-dependencies

[bdevans]

Rather than listing python3-notebook as a dependency (which I believe is deprecated), shouldn't we use jupyter lab? It might in general be better to install python packages with pip (in a virtual environment) rather than through (slowly updated) ubuntu packages.

[kwabenantim]

Thanks @bdevans. I'll update the package list.

[bdevans]

Hi @kwabenantim, I made some progress on merging this in but I think it is blocked until the revised chaste-dependencies package (which fixes the libexpat1 bug) supports arm64. The error message is as follows:

The following packages have unmet dependencies:
 chaste-dependencies : Depends: libvtk9-dev (= 9.3.0+dfsg1-1build1) but it is not going to be installed
E: Unable to correct problems, you have held broken packages.

Is that something you can look at or is it a job for @fcooper8472?

See also:

1. Merge PyChaste installation steps into Chaste docker images chaste-docker#21
2. [WIP] Merge PyChaste chaste-docker#22

[kwabenantim]

Hi @bdevans. I think @fcooper8472 has been working on this. Could it be fixed temporarily by something similar to the changes in Chaste#321?

[bdevans]

Thanks @kwabenantim, I had no luck with that so I'll wait to see what @fcooper8472 says.