Add documentation

[paddyroddy]

A really lazy way: paddyroddy/python-template#9 (+ paddyroddy/python-template#10 + paddyroddy/python-template#11) as I'm an idiot) which allows you to just use github-pages deploying from a gh-pages branch with these settings

A better example that I just did looks like: https://astro-informatics.github.io/sleplet/

[matt-graham]

I've tried out a couple of similar workflows to yours @paddyroddy for building and deploying docs for two different projects based on the template

https://github.com/UCL/dxh/blob/main/.github/workflows/docs.yml
https://github.com/UCL/neso-calibration/blob/main/.github/workflows/docs.yml

but both using Sphinx here rather than pdoc. Maybe some variant of this with a choice in template setup of doc system to use could work. Using tox to build docs worked quite well in making it easy to build both locally and in GitHub workflow.

[paddyroddy]

I don't mind what we do, but I think it would be good to make it really straightforward for users who are likely to not go down the docs route

[samcunliffe]

Decision

We will move to mkdocks. This should go green and sphinx should go amber the justification needs rewording ad moving into the table.

• Consider deleting the sphinx extensions part (or move it).

[dstansby]

FWIW I'm 👎 on making sphinx amber - in my opinion it's more complicated than mkdocs, but there are advantages such as:

• More widely used by scientific Python projects
• A wider range of plugins to work with documenting Python libraries (e.g. sphinx-gallery, sphinx-automodapi)

So I think worth keeping both mkdocs and sphinx as green, but explaining briefly the different use cases for them?

[paddyroddy]

MkDocs also has a wide range of plugins: https://github.com/mkdocs/catalog

[samcunliffe]

🟢 At least one person in ARC uses this. We actively recommend using it above other tools. It is the single recommended tool for a given purpose.
🟠 We don’t discourage using this, but it may duplicate functionality of a green tool.

I don't think we should regularly "double-green" tools that 100% overlap functionality. Especially not when we're going to include one in the template.

So I would demote one of the two to amber (we don't discourage it!) and have the simplest(?) most beginnerfriendly-est(?) be the recommended one which is also in the template.

From memory, the arguments in favour of mkdocs were:

• simpler,
• markdown out of the box.

And in favour of sphinx:

• widely used.

Specifically against sphinx:

• the need to use plugins to get many features that we want by default,
• rst is dead, long live markdown.

---

@dstansby

So I think worth keeping both mkdocs and sphinx as green, but explaining briefly the different use cases for them?

I now can't find it, but did we have an amber tool elsewhere with a disclaimer like: if you want to do X you will find this tool better.

[matt-graham]

Specifically against sphinx:

the need to use plugins to get many features that we want by default,
rst is dead, long live markdown.

I think the first (need to use plugins / extensions) equally applies to Mkdocs as for example there is no built-in support for generating API documentation which I would say is something we would pretty much always want by default (and while autodoc is an extension in Sphinx it is maintained by the core Sphinx team).

With regards to the latter - there is decent Markdown support in Sphinx, see for example this guide and autodoc2 extension which provides support in docstrings. While I personally prefer writing documentation in Markdown because of familiarity, I think there are good arguments against having it as the default given the lack of standardization of Markdown, greater complexity in parsing it (and so more edge cases where you get unexpected formatting / differences between implementations) and the requirement to have specific syntax extensions to support the variety of directives needed when writing documentation (rather than general purpose text formatting provided by most Markdown flavours) as provided by MyST Markdown (https://mystmd.org/).

I think also the 'Mkdocs is simpler' is partly subjective - I've set up a lot of projects with Sphinx docs and after getting over the initial learning curve it was pretty easy to work with for basic use cases of generating API documentation plus a few top-level documentation pages. I think it gets more complex when you start to use lots of plug-ins, but that's opt-in.

Overall I don't have a strong preference between Sphinx and Mkdocs, I think both are great tools and would be happy with recommending either.

[samcunliffe]

At least one person in ARC uses this

Any ARC projects using MkDocs? I can think of 3 using Sphinx.

[paddyroddy]

after getting over the initial learning curve

This is enough of a reason against, seeing as we are aiming at beginners...

[paddyroddy]

Any ARC projects using MkDocs? I can think of 3 using Sphinx.

• The Heracles Euclid project (repo private)
• SRR XNAT
• ...

I'm sure @p-j-smith is doing some too? There are probably others I'm forgetting.

I think what annoys most about sphinx is the amount of boilerplate the comes with it, e.g. Makefile, etc.

[matt-graham]

For the purposes of illustrating that setting up Sphinx doesn't require much boilerplate (no Makefiles present 😛) and would be simple for a user to set up (no additional steps compared to current use of cookiecutter) I've opened a PR at #318 which shows how we could add autogenerated docs using Sphinx to template. Still happy for Mkdocs to be chosen as the recommendation and added to template, but I wanted to show that Sphinx set up isn't all that painful

[samcunliffe]

I think I rather softly vote for MkDocs → 🟢 and Sphinx → 🟠 .

Having used the latter quite a lot, I wouldn't say it's particularly "I don't care about configuration, please just set it up for me" -friendly. Which, we should remember, is our audience.

We can write some caveats into our recommended tooling website.

That said, I'd also be fine with Matt's PR #318.

[dstansby]

I don't think we should regularly "double-green" tools that 100% overlap functionality. Especially not when we're going to include one in the template.

👍 , my bad for not reading our own definitions of red/amber/green

the need to use plugins to get many features that we want by default,

What features are you thinking of that ship with mkdocs by default, but need a plugin in Sphinx?

seeing as we are aiming at beginners...

While beginner-freindliness is important I think is not the only consideration. For example I think using tools that are common in other open source Python projects (to avoid having to relearn when hopping across projects), and using tools that are commonly used across ARC are also important considerations.

I think Matt's PR shows that sphinx doesn't have to have a lot of boilerplate? So I'd medium-hard vote for MkDocs → 🟠 and Sphinx → 🟢.

[paddyroddy]

What features are you thinking of that ship with mkdocs by default, but need a plugin in Sphinx?

• For one that you need a plugin to use markdown I believe?
• Also navigation menu in MkDocs is just done with the following (not saying this syntax is nice, just showing it works out of the box)
  

To make a reasonable looking website for SRR I only installed mkdocs, mkdocs-include-markdown-plugin (to include the project README.md), and the theme mkdocs-material. It really is that simple.

[paddyroddy]

For the purposes of illustrating that setting up Sphinx doesn't require much boilerplate

It has:

• configuration from a python file rather than a YAML (only benefit I can see if having access to Python for updating some bits)
• an empty _static folder to be committed
• using RST (which many users won't know, i.e. me)
• a command so complex you're having to abstract it behind tox, rather than mkdocs serve

Other than your familiarity with it, and the fact that you both don't think any open source projects use MkDocs. I don't know how you can underestimate the complexity and confusion it will give new users - which, like it or not, will be many of our user base. I appreciate you might be long-term sphinx uses, but I've only used it in the last year or two, and my eyes were opened upon discovering MkDocs. I'm not saying it's perfect, but I see little reason for me to use sphinx other than in a pre-existing project.

[paddyroddy]

FWIW I'm heavily in favour of MkDocs → 🟢 and Sphinx → 🟠 based on how we define the traffic system and comments above. However, I do think it should be documented that many projects do and will continue to use sphinx.

[paddyroddy]

I do also fear that #318 being worked one will mean it gets merged. For a fair comparison, an mkdocs PR could also be made.

[dstansby]

Can you link to a project that uses mkdocs so we can make a comparison with how complicated the setup/config is?

the fact that you both don't think any open source projects use MkDocs

Please can we keep discussion civil! In my experience most scientific Python projects do use sphinx - I'm happy to be be shown otherwise though.

I do agree that mkdocs has some nice advantages over sphinx - but I still think sphinx has some nice advantages over mkdocs. In my (quick, biased) opinion sphinx advantages outweigh mkdocs advantages, but I think the best way forward here would be to collaboratively come up with a table of advantages for each one, then we can come to a more considered consensus, and perhaps involve a wider set of folks in the decision?

[paddyroddy]

Can you link to a project that uses mkdocs so we can make a comparison with how complicated the setup/config is?

The problem with the ones I've used are ARC projects that are private, so you won't have access :(

[paddyroddy]

Also FYI I was being civil...

[samcunliffe]

Intersphinx is an important candidate for the sphinx pros list.

Anyone used mkdocstrings cross-references?

[matt-graham]

configuration from a python file rather than a YAML (only benefit I can see if having access to Python for updating some bits)

Being able to doing things like access the dynamically created version as recommeded by setuptools_scm is I think a good argument of why using a Python file for configuration can be helpful.

an empty _static folder to be committed

Now removed thanks to @dstansby's comment.

using RST (which many users won't know, i.e. me)

There is no reStructured Text required in the example in #318 - everything is configured to use Markdown.

a command so complex you're having to abstract it behind tox, rather than mkdocs serve

The reason for abstracting behind tox is the same reason you run tests in tox - so you are building documentation in an isolated environment and specifying dependencies that need to be installed.

I also wouldn't say

sphinx-build -W -b html docs docs/_build/html

is particularly complex and mkdocs serve is not the equivalent command - that would be mkdocs build which similarly has a range of options you can specify. Here actually we could just use

sphinx-build docs docs/_build/html

as -b html is equivalent to the current default behaviour and -W is just to get stricter checking by having warning enforced as errors.

I do also fear that #318 being worked one will mean it gets merged. For a fair comparison, an mkdocs PR could also be made.

Definitely no expectation that because I've worked on this we go with Sphinx, as I've said already here and on the PR.

[samcunliffe]

Just to close the loop (sorry for the noise)

the need to use plugins to get many features that we want by default,

What features are you thinking of that ship with mkdocs by default, but need a plugin in Sphinx?

I was thinking mostly of autodoc but might have misremembered and cede to Matt's comment.

[paddyroddy]

Anyone used mkdocstrings cross-references?

I've used

- mkdocstrings:
    handlers:

syntax before. But that was in the Heracles code that they've removed my access to. I don't know what I was using it for though, could well have been a different feature.

[paddyroddy]

FYI from SRR

I have this GitHub Action (which checks that all files are mentioned in the nav section

---
name: MkDocs

# yamllint disable-line rule:truthy
on:
  push:
    branches:
      - main
      - release
      - renovate/**
  pull_request:
    types:
      - opened
      - ready_for_review
      - reopened
      - synchronize

concurrency:
  cancel-in-progress: true
  group: >-
    ${{ github.workflow }}-
    ${{ github.event.pull_request.number || github.ref }}

jobs:
  documentation:
    if: github.event.pull_request.draft == false
    runs-on: ubuntu-latest
    steps:
      - name: Checkout source
        # yamllint disable-line rule:line-length
        uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11 # v4

      - name: Set up python
        # yamllint disable-line rule:line-length
        uses: actions/setup-python@0a5c61591373683505ea898e09a3ea4f39ef2b9c # v5
        with:
          python-version: 3.x
          cache: pip
          cache-dependency-path: requirements.txt

      - name: Install dependencies
        run: python -m pip install --requirement requirements.txt

      - name: Run MkDocs
        run: mkdocs build --strict

and

This is the config file, using various features. The nav section is horrible due to how many pages that project has.

---
copyright: Copyright © 2023 - 2024 ARC
edit_uri: edit/main/docs
repo_name: UCL-MIRSG/UCLH-MPBE-SRR-XNAT
repo_url: https://github.com/UCL-MIRSG/UCLH-MPBE-SRR-XNAT
site_author: UCL ARC
site_name: UCLH MPBE SRR XNAT

extra:
  generator: false
  icon:
    edit: material/pencil
    repo: fontawesome/brands/github
    view: material/eye

markdown_extensions:
  - pymdownx.superfences:
      custom_fences:
        - name: mermaid
          class: mermaid
          format: "!!python/name:pymdownx.superfences.fence_code_format"

nav:
  - Home: index.md
  - Background:
      - background/index.md
      - Existing Pipeline:
          - background/existing_pipeline/index.md
          - SRR Processing Flowchart: >-
              background/existing_pipeline/SRR_pipeline_overview.md
  - Development:
      - development/index.md
      - Ansible: development/ansible.md
      - Docker: development/docker.md
  - Quality Management System Documentation:
      - qms/index.md
      - Software Development Plan:
          - qms/software_development_plan/index.md
          - System Overview: qms/software_development_plan/system_overview.md
          - Software Development Overview: >-
              qms/software_development_plan/software_dev_overview.md
      - Software Requirement Analysis:
          - qms/software_requirement_analysis/index.md
          - Overview of Product Use: >-
              qms/software_requirement_analysis/overview_of_product_use.md
          - User Requirements Process: >-
              qms/software_requirement_analysis/user_requirements_process.md
          - Verification and Validation: >-
              qms/software_requirement_analysis/verification_and_validation.md
          - Appendices: qms/software_requirement_analysis/appendices.md
      - Software Architecture Design:
          - qms/software_architecture_design/index.md
          - Software Architecture Document: >-
              qms/software_architecture_design/software_architecture_overview.md
      - Software Detailed Design:
          - qms/software_detailed_design/index.md
      - Software Unit and Integration Testing:
          - qms/software_unit_and_integration_testing/index.md
          - qms/software_unit_and_integration_testing/srr_testing.md
          - qms/software_unit_and_integration_testing/unit_test_protocols.md
      - Software System Testing and Validation:
          - qms/software_system_testing_and_validation/index.md
      - Software Risk Management:
          - qms/software_risk_management/index.md
      - Software Change Control and Problem Resolution Processes:
          - qms/software_change_and_problem_processes/index.md
      - Software Release Process:
          - qms/software_release_process/index.md
      - Post Market Surveillance and Software Maintenance:
          - qms/software_maintenance/index.md

plugins:
  - include-markdown
  - search

theme:
  features:
    - announce.dismiss
    - content.action.edit
    - content.action.view
    - content.code.copy
    - header.autohide
    - navigation.expand
    - navigation.footer
    - navigation.indexes
    - navigation.tabs
    - navigation.top
    - navigation.tracking
    - search.highlight
    - search.suggest
    - toc.follow
  name: material
  palette:
    # palette toggle for dark mode
    - scheme: slate
      toggle:
        icon: material/brightness-4
        name: Switch to light mode
    # palette toggle for light mode
    - scheme: default
      toggle:
        icon: material/brightness-7
        name: Switch to dark mode

validation:
  absolute_links: warn
  omitted_files: warn
  unrecognized_links: warn

These are the only files. The rest are markdown files in a docs folder. I use mkdocs serve locally to visualise, and mkdocs build in CI.

[matt-graham]

Now added an equivalent PR #319 adding MkDocs support to template

Overall I would say similar level of complexity for the user. In both cases if they wanted add a new page they would create a Markdown file in docs directory, and reference it in the top-level navigation / table of contents. I'm not entirely sure if mkdocstrings would automatically create new entries in API reference for each sub-module in the package or only for things directly exposed in top-level __init__.py.

MkDocs supports Google, NumPy and Sphinx style docstrings - I've currently used the Google style.

The intersphinx equivalent seems to work well.

[matt-graham]

My summary of points for and against Sphinx / MkDoc, partly based on my attempts of adding each to template

For Sphinx

• In wider use in large open source scientific Python projects (for example, NumPy, SciPy, Matplotlib, Pandas, JAX, Dask) and possibly in existing ARC projects (?)
• Support for generating API documentation maintained by core developers (but this requires using reStructuredText docstrings)
• Supports backlinking to GitHub source files in API documentation

Against Sphinx

• Defaults to reStructuredText for formatting documentation and docstrings with Markdown support requiring using external plugins
• sphinx-autodoc2 plug-in which seems best current approach for using Markdown in docstrings is a relatively new project with only one main contributor
• sphinx-autodoc2 doesn't currently support NumPy / Google style docstrings (Support for NumPy-style docstrings sphinx-extensions2/sphinx-autodoc2#33)
• conf.py file used for configuration requires a few Ruff rule exceptions to be added (specifically INP001 and A001)

For MkDocs

• Natively supports Markdown
• Support a range of docstrings style (Google, NumPy, Sphinx) with Markdown formatting
• YAML based configuration plays nicer with Ruff without having to add specific exceptions
• Extension used for generating Python API documentation, mkdocstrings-python (https://github.com/mkdocstrings/python) seems to being actively developed by multiple contributors

Against MkDocs

• Currently no support backilinking to GitHub source files in API documentation (https://stackoverflow.com/a/76627612)

I personally did not find MkDocs any easier to set up than Sphinx - I am slightly more familiar with Sphinx, but in both cases my main pain point was having to hunt out things across both main documentation and plug-in / extension specific documentation, and for both the resulting structure is similar - a configuration file (docs/conf.py for Sphinx, mkdocs.yml for MkDocs) plus a series of Markdown pages in docs directory, one of which points to a directive for incuding generated API documentation.

The lack of support for NumPy / Google docstrings and low bus-factor in sphinx-autodoc2 would at the moment make me probably err towards favouring MkDocs as 🟢 overall, if we're assuming we want to use Markdown in docstrings.

[dstansby]

For automatically generating API docs in sphinx I've used https://numpydoc.readthedocs.io/en/latest/index.html + https://github.com/astropy/sphinx-automodapi with NumPy style docstrings.

I'm now coming round to the list of MkDocs features outweighing the wide usage of sphinx, so I think makes sense for it to go 🟢 and the MkDocs PR to be merged given that now seems to be the consensus 👍

I think worth having a discussion about which docstring style to recommend/use, but that's definitely for another issue 😄

[matt-graham]

As we do seem to have coalesced on MkDocs → 🟢 I've closed #318 and marked #319 as ready for review - thanks @paddyroddy for having already given it a once over.