Document Phoebus preferences in Markdown

[minijackson]

This is a follow up of #3567.

This PR migrates the preference listing documentation to Markdown (using the MyST Sphinx plugin) instead of RST, as Markdown is more well-known.

I've also gone through the various properties files, and make the documentation of each property valid Markdown, so that it renders nicely.

And lastly, in various places in the documentation, I've linkified mentions of preference to the preference on the listing page, using the prefs:pref and prefs:pack roles created in #3567.

Checklist

• Testing:

  • The feature has automated tests
  • Tests were run
  • If not, explain how you tested your changes

• Documentation:

  • The feature is documented
  • The documentation is up to date
  • Release notes:
    • Added an entry if the change is breaking or significant
    • Added an entry when adding a new feature

[minijackson]

• The newly rendered preference listing with this PR: https://control-system-studio--3588.org.readthedocs.build/en/3588/preference_properties.html
• An example of linkified preferences: https://control-system-studio--3588.org.readthedocs.build/en/3588/app/logbook/olog/ui/doc/index.html#preferences

[minijackson]

Note: MyST currently has an issue with titles inside objects documentation, which I've reported here: executablebooks/MyST-Parser#1050

This is why I've used the rubric Sphinx directive here and there, instead of using normal Markdown titles.

[georgweiss]

Even after installing the myst_parser Python module, I get:

Original exception:
Traceback (most recent call last):
  File "sphinx/registry.py", line 417, in load_extension
  File "importlib/__init__.py", line 127, in import_module
  File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
  File "<frozen importlib._bootstrap>", line 983, in _find_and_load
  File "<frozen importlib._bootstrap>", line 965, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'myst_parser'


Traceback (most recent call last):
  File "sphinx/registry.py", line 417, in load_extension
  File "importlib/__init__.py", line 127, in import_module
  File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
  File "<frozen importlib._bootstrap>", line 983, in _find_and_load
  File "<frozen importlib._bootstrap>", line 965, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'myst_parser'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "sphinx/cmd/build.py", line 279, in build_main
  File "sphinx/application.py", line 245, in __init__
  File "sphinx/application.py", line 402, in setup_extension
  File "sphinx/registry.py", line 421, in load_extension
sphinx.errors.ExtensionError: Could not import extension myst_parser (exception: No module named 'myst_parser')

On the master branch I get a different Python related error:

Traceback (most recent call last):
  File "sphinx/registry.py", line 417, in load_extension
  File "importlib/__init__.py", line 127, in import_module
  File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
  File "<frozen importlib._bootstrap>", line 983, in _find_and_load
  File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 728, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/Users/georgweiss/projects/phoebus/docs/source/_ext/preferences_listing/__init__.py", line 10, in <module>
    from sphinx.util.typing import ExtensionMetadata
ImportError: cannot import name 'ExtensionMetadata' from 'sphinx.util.typing' (/var/folders/4_/5qrr1nkn0_957tp3n0ltdn502f2zw4/T/_MEIpXVIxR/sphinx/util/typing.pyc)


Traceback (most recent call last):
  File "sphinx/registry.py", line 417, in load_extension
  File "importlib/__init__.py", line 127, in import_module
  File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
  File "<frozen importlib._bootstrap>", line 983, in _find_and_load
  File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 728, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/Users/georgweiss/projects/phoebus/docs/source/_ext/preferences_listing/__init__.py", line 10, in <module>
    from sphinx.util.typing import ExtensionMetadata
ImportError: cannot import name 'ExtensionMetadata' from 'sphinx.util.typing' (/var/folders/4_/5qrr1nkn0_957tp3n0ltdn502f2zw4/T/_MEIpXVIxR/sphinx/util/typing.pyc)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "sphinx/cmd/build.py", line 279, in build_main
  File "sphinx/application.py", line 245, in __init__
  File "sphinx/application.py", line 402, in setup_extension
  File "sphinx/registry.py", line 421, in load_extension
sphinx.errors.ExtensionError: Could not import extension preferences_listing (exception: cannot import name 'ExtensionMetadata' from 'sphinx.util.typing' (/var/folders/4_/5qrr1nkn0_957tp3n0ltdn502f2zw4/T/_MEIpXVIxR/sphinx/util/typing.pyc))

Extension error:
Could not import extension preferences_listing (exception: cannot import name 'ExtensionMetadata' from 'sphinx.util.typing' (/var/folders/4_/5qrr1nkn0_957tp3n0ltdn502f2zw4/T/_MEIpXVIxR/sphinx/util/typing.pyc))

In general I have concerns about the documentation build process if it depends on a specific Python environment and its module dependencies.

[georgweiss]

Also need to express doubts as to the value of supporting Markdown in preferences files, in particular if it requires additional Python modules. In my view preferences are in most cases not that complex in nature, so it should be sufficient to document them in plain text.

[minijackson]

The way it's built right now, there is no "plain text". Currently the preferences doc-comments are rendered as RST, but they were not written with that with that in mind.

I'd argue that any documentation benefits from having a markup language. When reading the preference doc-comments, you can see traces of markup, for example:

# Opacity of 'area'
#   0%: Area totally transparent (invisible)
#  20%: Area quite transparent
# 100%: Area uses  solid color
opacity=40

# Sites that prefer to use the EPICS_CA_... environment variables
# thus need to add
#
#      -Djca.use_env=true
#
# to their launcher script.

When reading these doc-comments, I see a definition list, and a block of code.

Having markup makes the documentation much more readable for users, and is a tool for developers to communicate to the user. It may be which part of the documentation is more important, which part to absolutely keep in mind, etc.

As for the extra Sphinx plugin, I understand the doubt. My reasoning was that Markdown is by far more well known than RST, and so is easier to contribute. I think that making the documentation easier to contribute to will lead to better documentation.

But at the end of the day, choosing between RST and Markown is your choice to make. This PR is only a proposition.

In general I have concerns about the documentation build process if it depends on a specific Python environment and its module dependencies.

It's already the case today, the documentation needs a Python environment with Sphinx installed, and the Sphinx ReadTheDocs theme, which is another Python package.

[minijackson]

On the master branch I get a different Python related error:

I think it comes from your Sphinx version being older than 7.2.7, where ExtensionMetadata was introduced. Make sure you're using the requirements.txt when installing the Python dependencies.

If you think the required Sphinx version is too recent, could you tag me in a new issue?

[shroffk]

Also,
The maven plugin which can be used to build the sphinx documentation also does not work since it uses an older version which does not support extensions.

[shroffk]

While my knee jerk reaction is to go back to what was working... I do also see value in staying up to date
:(

The maven plugin has had no activity since 4 years

[georgweiss]

Revealing my very limited experience in Python here, but I have serious issues with the Python setup.
Installing Sphinx 8.x required newer version of Python. Removing what I think is all traces of Sphinx 3.x does not make any difference: the doc build procedure seems to reference Sphinx 3.x.

[minijackson]

The maven plugin has had no activity since 4 years

That's what I was going to say. No matter how the preference properties page is implemented, there's going to be an issue between the 4-year old Sphinx version of the Maven sphinx plugin, and the Sphinx version used by ReadTheDocs.

The maven plugin seems to only support a few old extensions, which… seems very limiting.

@georgweiss can you make a new issue detailing which version of Python you're targeting? I'll make my Sphinx extension compatible with the corresponding Sphinx version.

While my knee jerk reaction is to go back to what was working... I do also see value in staying up to date

That's sad, I thought you would see value in having formatted and cross-referenceable properties :(

[shroffk]

While my knee jerk reaction is to go back to what was working... I do also see value in staying up to date

That's sad, I thought you would see value in having formatted and cross-referenceable properties :(

I am actually agreeing with you... what I was saying is that the knee jerk reaction is not correct and that we should try to stay up to date.

[shroffk]

@georgweiss here is my first attempt to simplify and improve the docs build...could you try to follow the instructions and let me know if it works

I am abandoning the maven build since it is not reasonable to depend on unsupported maven plugins.

minijackson#1

[kasemir]

I am abandoning the maven build since it is not reasonable

👍

[shroffk]

I should clarify this is this just the maven sphinx build... otherwise maven rocks :)

[kasemir]

Your inner voice had it right, go with it!