The instructions for generation of MOM6 documentation is divided into three sections.
To assist with writing of this documentation, we use several abbreviations.
DOX: Doxygen refers to the markdown language used by doxygen or a doxygen file.
RST: reStructuredText; a markdown language leveraged by sphinx via docutils.
RTD: Read the Docs
An API and user manual are the main forms of the MOM6 documentation produced.
API Manual
The API (reference) manual contains descriptions of all the modules, subroutines, functions, arguments and members that form data structures for variables. This version of the html documentation is hosted here: APIs
This version of the manual utilizes the html output of doxygen.
User Manual
A user manual format of the manual contains more descriptive materials. This version of the manual may also contain the APIs. RTD is a popular hosting service for documentation. A copy of the may be found here: MOM6 User Manual.
Please see the software operation section for details on how to produce various portions of this documentation locally and on your own RTD site as desired.
Source code with embedded documentation is processed by doxygen.
Doxygen in run in two modes. For the API manual, it is instructed to generate only the html version of the documentation. For the user manual, it is instructed to generate intermediate files for use with sphinx and RTD. For purposes of this documentation the API manual is referred to as generated using html(doxygen). The html generated by sphinx is referred to as html(sphinx). The same references may be applied to the pdf versions: pdf(doxygen) and pdf(sphinx).
If the software is installed, local copies of html and pdf instead of those found at RTD. The html and pdf versions produced by doxygen will not have any of the additional documentation from the sphinx portion of the pipeline.
RTD by default runs html(sphinx) pipeline and adds a bit of styling. The RTD can be instructed to also produce a PDF. However, the current pipeline exceeds 900 seconds execution time and fails if RTD is instructed to generate the html and PDF in one session. By default, the PDF generation is currently turned off.
We define SRC as the root of the source directory assuming the $(SRC) directory was created from downloading the tree from github using git clone.
The starting state of directories and files:
SRC/
config_src/
drivers/
FMS_cap
ice_solo_driver
mct_cap
nuopc_cap
solo_driver
unit_drivers
external
infra/
FMS1
memory/
dynamic
dynamic_symmetric
pkg/
CVMix-src
GSW-Fortran
src/
ALE
core
diagnostics
equation_of_state
framework
ice_shelf
initialization
ocean_data_assim
parameterizations
tracer
user
docs/
_static/
_templates/
*.rst
api/*.rst
images/*
The Doxygen package is used to scan the source code for embedded documentation. Doxygen also automatically produces API documentation from arguments to subroutines, functions and member of types. The guide for using doxygen in MOM6 is hosted on the MOM6 developer's wiki.
The majority of documentation is embedded in the comments of the source code. Additional documents have been added in the form of doxygen files (*.dox). The additional documents can be identified as those prefixed with an underscore (_) and have the dox extension.
The content of the additional documents and source code should conform to usage as defined by the doxygen user and reference manuals. Further guidance is provided on the MOM6 developer's wiki. Troubleshooting guides are provided with most commonly reported problems with MOM6 documentation.
NOTE: Not all doxygen commands are supported through the sphinx documentation processor. Support can be added by adding an issue to the github repository.
For the API documentation, the resultant tree will look like this:
SRC/
docs/
_build/
doxygen_warn_rtd_log.txt
[html|latex]
_static
MOM6.tags
The main driver for doxygen is a configuration file. The content on MOM6 developer's wiki uses the Doxyfile_nortd configuration file.
By default, the html directory is only available after processing the documentation. Please see software operation on how to generate the pdf companion of the documentation.
For the user manual documentation run, this is the first phase of the process. This process produces the following directories and files:
SRC/
docs/
_build/
doxygen_warn_rtd_log.txt
xml/
MOM6.tags
The documentation generated by Sphinx and RTD uses the Doxyfile_rtd. These options can be overridden at the command line. See software operation for more details.
The _build/doxygen_warn_rtd_log.txt should be reviewed for warnings and errors.
The Sphinx package renders final versions of html and pdf documents. The provided RST files (*.rst) provide the organization structure for the html and pdf documents. It is recommended that any significant content be pushed into the source code or additional dox files.
During a complete documentation run, this is the second phase of the process. Files and directories created:
SRC/
docs/
_build/
html_log.txt
doctrees
[html|latex]
The directories, html and or latex, will be present depending on how Sphinx was called. The doctrees directory contains processed documents saved as python pickles (binary data structures).
The html_log.txt should be reviewed for warnings and errors.
The RTD site can be configured to watch for updates on a github repository. A documentation update may be triggered when an update is pushed to the repository. The entire documentation process is run twice. The first run produces html. The second run produces a pdf (if so instructed).
NOTE: There is a rough execution time limit of about 900 seconds. Trying to do more than that will cause a "timeout" error.
NOTE: To post process equations within RTD, see UPDATEHTMLEQS below for more information.
Please see the details for special cases to the documentation syntax not found in any of the user manuals for Doxygen or Sphinx.
The entire documentation processing pipeline can be operated using the provided Makefile. Running make or make help will provide some helpful information.
Otherwise, savvy users can run individual pieces on the command line using provided configuration files.
Certain environment variables may be set prior or during the make command to influence the documentation build process. Explanations on how to run various parts of the pipeline are given below.
For generation of the API portion of the documentation, see the MOM6 developer's wiki.
The configuration file used for doxygen is Doxyfile_nortd.
From the Makefile, the API portion of the pipeline may be run using:
make nortd
NOTE: make nortd will attempt to install doxygen if a binary is not found.
See below, the following Makefile options are honored:
- BUILDDIR
- DOXYGEN_BIN
- DOXYGEN_CONF
- DOXYGEN_RELEASE
- UPDATEHTMLEQS
- UPDATEHTMLEQSVERBOSE
- PAPER
The full documentation can be generated locally with
# NOTE: make sure _build directory exists
make html >& _build/html_log.txtwhich will generate html in docs/_build/html/. Start at docs/_build/html/index.html. This will also place log information
into the _build tree that is viewable. Warning messages from doxygen is written to _build/doxygen_warn_rtd_log.txt.
To fix equation targets and references the full update to the RTD/Sphinx documentation requires:
make html UPDATEHTMLEQS=YMany options can be passed to sphinx with make on the command line after the last argument. e.g. make html SPHINXOPTS=-v
Multiple arguments may be passed using a space as separation between arguments.
Useful combinations:
- make clean (Cleans the working directory of most intermediate and final documentation)
- make Clean (Removes the
_buildandxmldirectory and MOM6.tags file) - make html (Builds html based documentation)
- make latex (Builds latex based documentation; does not compile a pdf)
- make latexpdf (Attempts to build a final pdf of the documentation)
Run make or make help for other options.
The doxygen generated HTML can be obtained locally (and slightly more quickly) with
make nortdwhich will generate html in docs/APIs/. Start at docs/APIs/index.html.
If doxygen is not already available this will install a local copy of doxygen.
This option is passed to sphinx-build. See sphinx-build --help for more information.
The default output directory is _build. The default doxygen configuration files are expecting an output
directory of _build.
If UPDATEHTMLEQS is set to Y, postProcessEquations.py will be run
to attempt to update equation numbers in the generated html. This environment variable has to be
added to the RTD administrative interface if you want to trigger post processing of equations.
If UPDATEHTMLEQSVERBOSE is set to -v this will turn on verbose printing for the post processor.
NOTE: These options affect solo doxygen html processing only for the make nortd option. For sphinx, they are
utilized in the sphinx python module to handle post processing and are not part of the Makefile. This was done
as RTD runs sphinx processing directly using sphinx-build and not the Makefile.
The default value is empty (PAPER=). There are two options currently available in the Makefile (a4 or letter).
The PAPER argument has no impact on html rendered content.
Other options are read by conf.py and can be passed on the command line when running make. These options may
also be set in the shell environment prior to running make. These can be set on RTD too.
Using this option, you can specify a different doxygen binary for processing the documentation. Otherwise, a doxygen executable needs to be available in the system path.
Use this option to specify a custom configuration file to doxygen.
This option allows selection of a doxygen version from https://github.com/doxygen.
The default is Release_1_8_13.
NOTE: This command does not have any impact if an existing binary is found.
This option activates the NCAR configuration file: ncar/Doxyfile_ncar_rtd
On a relatively bare system, you can install a fairly stable documentation pipeline.
If you do not have doxygen, to build a local version of the doxygen processor you will need the following packages:
- cmake
- g++ (or a c++ compiler)
- flex
- bison
- graphviz
- pdf2svg
(e.g. apt-get install cmake g++ flex bison graphviz)
If you are building the full generated sphinx documentation you will need the following packages in addition to those for doxygen above:
- libxml2-dev
- libxslt-dev (may also be called libxslt1-dev)
(e.g. apt-get install libxml2-dev libxslt-dev)
We strongly recommend using python3 with its virtual environment and pip3.
(e.g. apt-get install python3 python3-venv python3-pip)
Before running sphinx (make html) you will need to issue:
pip install -r requirements.txtYou may need to use pip3 to install requirements for python3.
Requirements:
- sphinx
- sphinx-rtd-theme
- sphinx-bibtex
- sphinx-fortran
- sphinxcontrib_autodox-doxygen
- flint
- lxml
- numpy
- future
For machines that need to build future, numpy or lxml, these packages are required:
- Cython
- wheel
PDF generation requires the following packages
- texlive-latex-base
- texlive-latex-recommended
- texlive-latex-extra
- latexmk
You may choose to download the source.
Latest is doxygen-1.8.20.src.tar.gz.
tar xzf doxygen-1.8.20.src.tar.gz
cd doxygen-1.8.20
mkdir build
cd build
cmake -G "Unix Makefiles" ..
make
sudo make installThe makefile for doxygen attempts to install the compiled version into /usr/local/bin.
You can link to a specific executable within the virtual environment. At this point we
also recommend renaming doxygen to doxygen-1.8.20 within /usr/local/bin.
NOTE: The makefile for the documentation framework will attempt to compile a local doxygen
binary of version 1.8.13 if a binary cannot be found in the $PATH.
A lot of manual testing has been completed using the following versions:
- 1.8.13
- 1.8.14
- 1.8.19
- 1.8.20
The Read the Docs (RTD) site uses a virtual machine (VM) for processing documentation. The VM architecture is type x86_64. The default version for doxygen is 1.8.13 on the RTD VM.
NOTE: Using modified python modules on RTD is possible through careful crafting of the requirements.txt file. It is impossible to replace system binaries or compile code on RTD. It is possible to ship replacement binaries that can be run from the repo. For security reasons, a binary cannot be included in the MOM6 repository.
See Sphinx run options below. We capture up to three logfiles for RTD. The main default logfile is LOGS/doxygen_warn_rtd_log.txt.
Logfiles were renamed to *.txt to allow easier access and viewing from most websites.
Most websites force download of *.log files.
The documentation pipeline was upgraded by Rob Cermak and Marshall Ward. Four modified python modules are required
to process the MOM6 documentation. The versions are tagged and placed into the production version of requirements.txt. Development versions may be found in the respective dev branches.
| Source | Modified | Version | Development |
|---|---|---|---|
| sphinx | sphinx-3.2.1mom6.4 | B:3.2.1mom6.4 | B:dev |
| sphinxcontrib-autodoc-doxygen | sphinxcontrib-autodoc-doxygen | T:0.7.13 | B:dev |
| sphinx-fortran | sphinx-fortran | T:1.2.2 | B:dev |
| flint | flint | T:0.0.1 | B:dev |
| MOM6 | esmg-docs | esmg-docs | B:esmg-test |
T: tag B: branch
The sphinx documentation of MOM6 is made possible by modifications by Angus Gibson to two packages, sphinx-fortran and autodoc_doxygen.