docs: Use MyST and sphinx-external-toc

README.md

@@ -1,4 +1,4 @@
-## [NVIDIA Merlin](https://github.com/NVIDIA-Merlin)
+# [NVIDIA Merlin](https://github.com/NVIDIA-Merlin)
 
 ![GitHub tag (latest SemVer)](https://img.shields.io/github/v/tag/NVIDIA-Merlin/Merlin?sort=semver)
 ![GitHub License](https://img.shields.io/github/license/NVIDIA-Merlin/Merlin)
@@ -13,7 +13,7 @@ of data, which is all accessible through easy-to-use APIs. For more information,
 see [NVIDIA Merlin](https://developer.nvidia.com/nvidia-merlin) on the NVIDIA
 developer web site.
 
-### Benefits
+## Benefits
 
 NVIDIA Merlin is a scalable and GPU-accelerated solution, making it easy to
 build recommender systems from end to end. With NVIDIA Merlin, you can:
@@ -26,7 +26,7 @@ build recommender systems from end to end. With NVIDIA Merlin, you can:
 - Deploy data transformations and trained models to production with only a few
   lines of code.
 
-### Components of NVIDIA Merlin
+## Components of NVIDIA Merlin
 
 NVIDIA Merlin consists of the following open source libraries:
 
@@ -97,22 +97,20 @@ With Merlin Core, you can:
 - Simplify your code by using a shared API for constructing graphs of data
   transformation operators.
 
-### Example Notebooks and Tutorials
+## Example Notebooks and Tutorials
 
-A collection of [end-to-end examples](./examples/) is available within this
-repository in the form of Jupyter notebooks. The example notebooks demonstrate
-how to:
+A collection of [end-to-end examples](./examples/) are available in the form of Jupyter notebooks.
+The example notebooks demonstrate how to:
 
-- download and prepare the dataset.
-- use preprocessing and engineering features.
-- train deep learning recommendation models with TensorFlow, PyTorch, FastAI, or
-  HugeCTR.
-- deploy the models to production.
+- Download and prepare a dataset.
+- Use preprocessing and engineering features.
+- Train deep-learning recommendation models with TensorFlow, PyTorch, FastAI, or HugeCTR.
+- Deploy the models to production with Triton Inference Server.
 
 These examples are based on different datasets and provide a wide range of
 real-world use cases.
 
-### Merlin Is Built On
+## Merlin Is Built On
 
 **[cuDF](https://github.com/rapidsai/cudf)**<br> Merlin relies on cuDF for
 GPU-accelerated DataFrame operations used in feature engineering.
@@ -125,7 +123,7 @@ dataloading in Merlin Models and HugeCTR.
 Merlin leverages Triton Inference Server to provide GPU-accelerated serving for
 recommender system pipelines.
 
-### Feedback and Support
+## Feedback and Support
 
 To report bugs or get help, please
 [open an issue](https://github.com/NVIDIA-Merlin/Merlin/issues/new/choose).

docs/requirements-doc.txt

@@ -6,9 +6,12 @@ markupsafe==2.0.1
 natsort==8.1.0
 sphinx_rtd_theme
 sphinx_markdown_tables
-sphinx-multiversion
-recommonmark>=0.6
-nbsphinx>=0.6
+recommonmark
+sphinx-multiversion@git+https://github.com/mikemckiernan/sphinx-multiversion.git@v0.3.0
+sphinxcontrib-copydirs@git+https://github.com/mikemckiernan/sphinxcontrib-copydirs.git@v0.3.1
+sphinx-external-toc==0.2.4
+myst-nb==0.13.2
+linkify-it-py==1.0.3
 
 # smx
 mergedeep==1.3.4

docs/source/conf.py

@@ -1,12 +1,8 @@
 # Configuration file for the Sphinx documentation builder.
-import errno
 import os
-import shutil
 import subprocess
-import sys
 
 from natsort import natsorted
-from recommonmark.parser import CommonMarkParser
 
 docs_dir = os.path.dirname(__file__)
 repodir = os.path.abspath(os.path.join(__file__, r"../../.."))
@@ -24,19 +20,36 @@
 # extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
 # ones.
 extensions = [
+    "myst_nb",
     "sphinx_multiversion",
     "sphinx_rtd_theme",
-    "recommonmark",
     "sphinx_markdown_tables",
-    "nbsphinx",
     "sphinx.ext.autodoc",
     "sphinx.ext.coverage",
     "sphinx.ext.githubpages",
     "sphinx.ext.napoleon",
     "sphinx.ext.viewcode",
     "sphinx.ext.intersphinx",
+    "sphinx_external_toc",
+    "sphinxcontrib.copydirs",
 ]
 
+# MyST configuration settings
+external_toc_path = "toc.yaml"
+myst_enable_extensions = [
+    "deflist",
+    "html_image",
+    "linkify",
+    "replacements",
+    "tasklist",
+]
+myst_linkify_fuzzy_links = False
+myst_heading_anchors = 3
+jupyter_execute_notebooks = "off"
+
+# Some documents are RST and include `.. toctree::` directives.
+suppress_warnings = ["etoc.toctree", "myst.header", "misc.highlighting_failure"]
+
 # Add any paths that contain custom static files (such as style sheets) here,
 # relative to this directory. They are copied after the builtin static files,
 # so a file named "default.css" will overwrite the builtin "default.css".
@@ -48,14 +61,19 @@
 # List of patterns, relative to source directory, that match files and
 # directories to ignore when looking for source files.
 # This pattern also affects html_static_path and html_extra_path.
-exclude_patterns = []
+exclude_patterns = [
+    "generated",
+]
 
 # -- Options for HTML output -------------------------------------------------
 
 # The theme to use for HTML and HTML Help pages.  See the documentation for
 # a list of builtin themes.
 #
 html_theme = "sphinx_rtd_theme"
+html_theme_options = {
+    "navigation_depth": 3,
+}
 html_show_sourcelink = False
 
 # Whitelist pattern for tags (set to None to ignore all tags)
@@ -69,23 +87,24 @@
     tag_refs = natsorted(tag_refs)[-6:]
     smv_tag_whitelist = r"^(" + r"|".join(tag_refs) + r")$"
 else:
-    # SMV is reading conf.py from a Git archive of the repo at a specific commit.
+    # SMV is reading conf.py from a Git archive of the repo at a
+    # specific commit.
     smv_tag_whitelist = r"^v.*$"
 
 # Only include main branch for now
 smv_branch_whitelist = "^main$"
 
+smv_refs_override_suffix = "-docs"
+
 html_sidebars = {"**": ["versions.html"]}
 
 # Add any paths that contain custom static files (such as style sheets) here,
 # relative to this directory. They are copied after the builtin static files,
 # so a file named "default.css" will overwrite the builtin "default.css".
 
-source_parsers = {".md": CommonMarkParser}
 source_suffix = [".rst", ".md"]
 
 nbsphinx_allow_errors = True
-html_show_sourcelink = False
 
 autodoc_inherit_docstrings = False
 autodoc_default_options = {
@@ -97,39 +116,8 @@
 
 autosummary_generate = True
 
+copydirs_additional_dirs = ["../../examples/", "../../README.md"]
 
-def copy_files(src: str):
-    """
-    src_dir: A path, specified as relative to the
-             docs/source directory in the repository.
-             The source can be a directory or a file.
-             Sphinx considers all directories as relative
-             to the docs/source directory.
-
-             TIP: Add these paths to the .gitignore file.
-    """
-    src_path = os.path.abspath(src)
-    if not os.path.exists(src_path):
-        raise FileNotFoundError(errno.ENOENT, os.strerror(errno.ENOENT), src_path)
-    out_path = os.path.basename(src_path)
-    out_path = os.path.abspath("{}/".format(out_path))
-
-    print(
-        r"Copying source documentation from: {}".format(src_path),
-        file=sys.stderr,
-    )
-    print(r"  ...to destination: {}".format(out_path), file=sys.stderr)
-
-    if os.path.exists(out_path) and os.path.isdir(out_path):
-        shutil.rmtree(out_path, ignore_errors=True)
-    if os.path.exists(out_path) and os.path.isfile(out_path):
-        os.unlink(out_path)
-
-    if os.path.isdir(src_path):
-        shutil.copytree(src_path, out_path)
-    else:
-        shutil.copyfile(src_path, out_path)
-
-
-copy_files(r"../../README.md")
-copy_files(r"../../examples/")
+copydirs_file_rename = {
+    "README.md": "index.md",
+}

docs/source/index.rst

@@ -1,17 +1,19 @@
 Merlin
 ======
 
-.. toctree::
-   :maxdepth: 2
+NVIDIA Merlin is an open source library that accelerates recommender
+systems on NVIDIA GPUs.
+The library brings together several component libraries that simplifies
+the process of building high-performing recommenders at scale.
 
-   Introduction <README.md>
-   Example Notebooks <examples/index>
-   Merlin Containers <containers>
-   Support Matrix <support_matrix/index>
+For more information, see the `Introduction <README.html>`_.
 
-Indices and tables
-==================
+Related Resources
+-----------------
 
-* :ref:`genindex`
-* :ref:`modindex`
-* :ref:`search`
+Merlin GitHub Repository
+  `<https://github.com/NVIDIA-Merlin/Merlin>`_
+
+Developer website for Merlin
+  More information about Merlin is available at our developer website:
+  `<https://developer.nvidia.com/nvidia-merlin>`_.

docs/source/support_matrix/index.rst

@@ -3,12 +3,3 @@ Merlin Support Matrix
 
 The following pages provide the software and model versions that Merlin supports for each container.
 
-.. toctree::
-   :maxdepth: 2
-
-   support_matrix_merlin_training
-   support_matrix_merlin_tensorflow_training
-   support_matrix_merlin_pytorch_training
-   support_matrix_merlin_inference
-   support_matrix_merlin_tensorflow_inference
-   support_matrix_merlin_pytorch_inference

docs/source/toc.yaml

@@ -0,0 +1,59 @@
+root: index
+subtrees:
+  - caption: Contents
+    entries:
+      - file: README.md
+        title: Introduction
+      - file: examples/index
+        title: Example Notebooks
+        subtrees:
+          - titlesonly: True
+            entries:
+              - file: examples/getting-started-movielens/index
+                title: Getting Started using the MovieLens Dataset
+                entries:
+                  - file: examples/getting-started-movielens/01-Download-Convert.ipynb
+                    title: MovieLens Download and Convert
+                  - file: examples/getting-started-movielens/02-ETL-with-NVTabular.ipynb
+                    title: Feature Engineering with NVTabular
+                  - file: examples/getting-started-movielens/03-Training-with-HugeCTR.ipynb
+                    title: Training with HugeCTR
+                  - file: examples/getting-started-movielens/03-Training-with-TF.ipynb
+                    title: Training with TensorFlow
+                  - file: examples/getting-started-movielens/03-Training-with-PyTorch.ipynb
+                    title: Training with PyTorch
+                  - file: examples/getting-started-movielens/04-Triton-Inference-with-HugeCTR.ipynb
+                    title: Serving the HugeCTR Model with Triton
+                  - file: examples/getting-started-movielens/04-Triton-Inference-with-TF.ipynb
+                    title: Serving the TensorFlow Model with Triton
+              - file: examples/Building-and-deploying-multi-stage-RecSys/index.md
+                entries:
+                  - file: examples/Building-and-deploying-multi-stage-RecSys/01-Building-Recommender-Systems-with-Merlin.ipynb
+                    title: Building the Recommender System
+                  - file: examples/Building-and-deploying-multi-stage-RecSys/02-Deploying-multi-stage-RecSys-with-Merlin-Systems.ipynb
+                    title: Deploying the Recommender System with Triton
+              - file: examples/scaling-criteo/index.md
+                entries:
+                  - file: examples/scaling-criteo/01-Download-Convert.ipynb
+                    title: Criteo Download and Convert
+                  - file: examples/scaling-criteo/02-ETL-with-NVTabular.ipynb
+                    title: Feature Engineering with NVTabular
+                  - file: examples/scaling-criteo/03-Training-with-FastAI.ipynb
+                    title: Training with FastAI
+                  - file: examples/scaling-criteo/03-Training-with-HugeCTR.ipynb
+                    title: Training with HugeCTR
+                  - file: examples/scaling-criteo/03-Training-with-TF.ipynb
+                    title: Training with TensorFlow
+                  - file: examples/scaling-criteo/04-Triton-Inference-with-HugeCTR.ipynb
+                    title: Serving the HugeCTR Model with Triton
+                  - file: examples/scaling-criteo/04-Triton-Inference-with-TF.ipynb
+                    title: Serving the TensorFlow Model with Triton
+      - file: containers.rst
+      - file: support_matrix/index.rst
+        entries:
+          - file: support_matrix/support_matrix_merlin_training.rst
+          - file: support_matrix/support_matrix_merlin_tensorflow_training.rst
+          - file: support_matrix/support_matrix_merlin_pytorch_training.rst
+          - file: support_matrix/support_matrix_merlin_inference.rst
+          - file: support_matrix/support_matrix_merlin_tensorflow_inference.rst
+          - file: support_matrix/support_matrix_merlin_pytorch_inference.rst

examples/Building-and-deploying-multi-stage-RecSys/README.md

@@ -1,18 +1,20 @@
-# NVIDIA Merlin Example Notebooks
+# Deploying a Multi-Stage Recommender System
 
-We created two jupyter notebook examples based on different stages of a Recommender Systems to demonstrate how to deploy a multi-stage Recommender Systems on Triton Inference Server. The notebooks demonstrate how to use NVTabular, Merlin Models and Merlin Systems libraries for ETL, training, and inference, respectively.
+We created two Jupyter notebooks that demonstrate two different stages of a Recommender Systems.
+The goal of the notebooks is to show how to deploy a multi-stage Recommender System and serve recommendations with Triton Inference Server.
+The notebooks demonstrate how to use the NVTabular, Merlin Models, and Merlin Systems libraries for feature engineering, training, and then inference.
 
-## Structure
+The two example notebooks are structured as follows:
 
-Two example notebooks are structured as follows:
-- 01-Building-Recommender-Systems-with-Merlin: 
-    - Execute the preprocessing and feature engineering pipeline (ETL) with NVTabular on the GPU
-    - train a ranking and retrieval model with TensorFlow based on the ETL output
-    - export saved models, user and item features and item embeddings.
-- 02-Deploying-multi-stage-RecSys-with-Merlin-Systems: 
-    - set up Feast feature store for feature storing and Faiss index for similarity search
-    - build multi-stage recommender systems ensemble pipeline with Merlin Systems operators
-    - perform Inference with the Triton Inference Server using Merlin Systems library.
+- [Building the Recommender System](01-Building-Recommender-Systems-with-Merlin.ipynb): 
+  - Execute the preprocessing and feature engineering pipeline (ETL) with NVTabular on the GPU.
+  - Train a ranking and retrieval model with TensorFlow based on the ETL output.
+  - Export the saved models, user and item features, and item embeddings.
+
+- [Deploying the Recommender System with Triton](02-Deploying-multi-stage-RecSys-with-Merlin-Systems.ipynb): 
+  - Set up a Feast feature store for feature storing and a Faiss index for similarity search.
+  - Build a multi-stage recommender system ensemble pipeline with Merlin Systems operators.
+  - Perform inference with the Triton Inference Server using the Merlin Systems library.
 
 ## Running the Example Notebooks