Understanding how the Ansible docs build

Understanding the Ansible Docs Build

There are many components to creating the Ansible documentation. This article attempts to start documenting some of these parts and pieces. It's a work in progress for sure, so don't take this all as proven fact just yet :-)

Sphinx - The workhorse of it all

Ansible uses Sphinx to convert our rst files to html. Fairly straightforward, but the main file it uses to configure it all is conf.py. We won't repeat the Sphinx conf.py documentation, but we do make use of a few items that we describe later in order to create two docsites from the ansible/ansible Github repository.

make webdocs - What does it do?

Whether you are building locally, or triggering the internal to Red Hat build script that publishes to docs.ansible.com, the most common command you use is make webdocs.

Prior to Ansible 2.10, make webdocs was the main command to make all the documentation. In 2.10 and later, there are two commands:

• make webdocs ANSIBLE_VERSION=[2.10,3,4] - this chooses between building all the docs (for 2.10) or building the Ansible package docs for 3 or later. The main difference is 3 or later does not include ansible-core roadmap or porting guides. If you don't select a version, it chooses the latest available from ansible-build-data.
• make coredocs - Excludes network and scenario guides, and any Ansible package roadmap/porting guides.

make webdocs - How does it do it?

This level of detail is mostly for the curious or anyone who has to change what and how these commands work. Feel free to ignore.

Not ignoring? Okay, let's take a walk through Makefiles and some python code.

Makefiles

In docs/docsite/ you'll see two makefiles:

• Makefile - sets up what make webdocs and make coredocs do at the highest level.
• Makefile.sphinx - sets up what Sphinx (the tool that converts our rst files to html) does.

There is also a section in the main Ansible Makefile make webdocs etc that allows you to run the make commands from the root directory instead of from docs/docsite. If you add some new variant on making the docs, consider if you should also add that variant to the main Ansible Makefile as well. Otherwise your new variant only works from `docs/docsite/

Docs Makefile

There's quite a lot in this Makefile, but let's start with the basics. Starting in Ansible 2.10, we need to create two separate docsites effectively from the same set of .rst files. To do this, we use a few Sphinx and Makefile tricks:

• Use the sphinx -c <path> option for a separate conf.py file to control the docsites and some other parameters.
• Use exclude_pattern[] to exclude network, galaxy, and scenario guides and the individual porting guides and roadmap files from the appropriate sites.
• Use symlinks in the Makefile to link index.rst from either ansible_index file for ansible or core_index.rst for core.

docs_build.py and the fun it has!

running notes from a session today... will need to make these more coherent later

docs-build from the Makefilegets saved into a variable PLUGIN_FORMATTER=../../hacking/build-ansible.py docs-build

The build-ansible.py script is a bit non-standard. The structure is that it's a lite wrapper script which sets up the location that the script can find its python files : https://github.com/ansible/ansible/blob/devel/hacking/build-ansible.py#L40-L41 : hacking/build-ansible.py:40-41 sys.path.insert(0, ansible_lib_path()) sys.path.insert(0, build_lib_path())

And then it loads the parts of the script that do actual work as plugins. So instead having a literal import docs_build somewhere, the loading of the code is done programmatically in a loop: https://github.com/ansible/ansible/blob/devel/hacking/build-ansible.py#L64
hacking/build-ansible.py:64 subcommands = load('build_ansible.command_plugins', subclasses=commands.Command)

Then the plugin we want (docs_build) is selected in this for loop: https://github.com/ansible/ansible/blob/devel/hacking/build-ansible.py#L82 hacking/build-ansible.py:82 for subcommand in subcommands:

And we execute the plugin's main() function here: https://github.com/ansible/ansible/blob/devel/hacking/build-ansible.py#L92 (edited) hacking/build-ansible.py:92 retval = command.main(args)

Explaining load('build_ansible.command_plugins', subclasses=commands.Command)

Okay, so load() comes from the plugin library we're using (straight.plugin). It takes a python package as its first argument and looks inside of that location for any python files . The second argument, subclasses=commands.Command, tells load () that valid plugins are commands.Command classes within the files its examining. So in our case, we added the hacking/build_library directory to sys.path (the directories that python checks for python libraries) on Line 41. So load finds the python package we specified in https://github.com/ansible/ansible/tree/devel/hacking/build_library/build_ansible/command_plugins

Each of those files should have a class inside of them that inherits from build_ansible.commands.Command (example: https://github.com/ansible/ansible/blob/devel/hacking/build_library/build_ansible/command_plugins/docs_build.py#L123 ) (edited) hacking/build_library/build_ansible/command_plugins/docs_build.py:123 class CollectionPluginDocs(Command):

load() finds that class in each of the files, and returns a list of all of them.

So make coredocs builds ansible-core docs. make webdocs ANSIBLE_VERSION=[2.10,3,4] builds ansible package docs. The latter even works in devel because the package builder is selecting what the version of the collection docs are.

We will Autodetect the latest collection *.deps file and use that.

(future ) Implement building the docs for the latest version of collections on galaxy?

Jenkins builds - where the magic happens

The Jenkins builds internal to Red Hat are what finally publishes all these documents. There are:

• Nightly builds -for ansible-core and Ansible (in the future)
• Manual builds - mostly for tests, but also to republish docs for a given minor release of ansible-core or Ansible.

When a new release happens, We need to update the Ansible jenkins build to recognize the new release number, and update redirects (see below).

Much Ado about Redirects

Starting with Ansible 2.10, we have server side redirects in place to allow the version switcher to switch between a module in 2.9 (say https://docs.ansible.com/ansible/2.9/modules/docker_image_module.html) to the same module now in 2.10 (https://docs.ansible.com/ansible/2.10/collections/community/general/docker_image_module.html). A significant (aka thousands) batch of redirects were generated for 2.9 and 2.10 to allow for the version switcher (and user bookmarks) to go to the correct new location for modules migrated to collections.

• For future releases, we need to create a release based folder at https://github.com/ansible/docsite/tree/master/ansible (private repo) and copy the 2.10 redirects into that new folder (for example 3/).