Docs revamp

.pre-commit-config.yaml

@@ -9,6 +9,20 @@ repos:
  - id: check-yaml
  - id: double-quote-string-fixer
 
+ - repo: https://github.com/executablebooks/mdformat
+ rev: 0.7.16
+ hooks:
+ - id: mdformat
+ additional_dependencies:
+ - mdformat-myst
+
+ - repo: https://github.com/adamchainz/blacken-docs
+ rev: "1.13.0"
+ hooks:
+ - id: blacken-docs
+ additional_dependencies:
+ - black==23.1.0
+
  - repo: https://github.com/psf/black
  rev: 23.3.0
  hooks:
@@ -33,3 +47,5 @@ repos:
  hooks:
  - id: prettier
  language_version: system
+ exclude_types:
+ - markdown # managed by mdformat

README.rst

@@ -37,10 +37,10 @@ Here is an example of directly accessing the data from within Python:
  from fsspec.implementations.http import HTTPFileSystem
 
  fs = HTTPFileSystem()
- http_map = fs.get_mapper('http://0.0.0.0:9000')
+ http_map = fs.get_mapper("http://0.0.0.0:9000")
 
  # open as a zarr group
- zg = zarr.open_consolidated(http_map, mode='r')
+ zg = zarr.open_consolidated(http_map, mode="r")
 
  # or open as another Xarray Dataset
  ds = xr.open_zarr(http_map, consolidated=True)

docs/requirements.txt

@@ -3,3 +3,5 @@ sphinx-autosummary-accessors
 pydata-sphinx-theme
 sphinx-autodoc-typehints
 autodoc_pydantic
+myst-nb
+sphinx-design

docs/source/api.rst → docs/source/api.md

@@ -1,17 +1,17 @@
+```{eval-rst}
 .. currentmodule:: xpublish
+```
 
-#############
-API reference
-#############
+# API reference
 
-Top-level Rest class
-====================
+## Top-level Rest class
 
-The :class:`~xpublish.Rest` class can be used for publishing a
-:class:`xarray.Dataset` object or a collection of Dataset objects.
+The {class}`~xpublish.Rest` class can be used for publishing a
+{class}`xarray.Dataset` object or a collection of Dataset objects.
 
 The main interfaces to Xpublish that many users may use.
 
+```{eval-rst}
 .. autosummary::
  :toctree: generated/
 
@@ -22,11 +22,13 @@ The main interfaces to Xpublish that many users may use.
  Rest.serve
  Rest.register_plugin
  Rest.dependencies
+```
 
 There are also a handful of methods that are more likely to be used
 when subclassing `xpublish.Rest` to modify functionality, or are used
 by plugin dependencies.
 
+```{eval-rst}
 .. autosummary::
  :toctree: generated/
 
@@ -37,78 +39,92 @@ by plugin dependencies.
  Rest.init_cache_kwargs
  Rest.init_app_kwargs
  Rest.plugin_routers
+```
 
-There is also a specialized version of :class:`xpublish.Rest` for use
+There is also a specialized version of {class}`xpublish.Rest` for use
 when only a single dataset is being served, instead of a collection
 of datasets.
 
+```{eval-rst}
 .. autosummary::
  :toctree: generated/
 
  SingleDatasetRest
  SingleDatasetRest.setup_datasets
+```
 
-For serving a single dataset the :class:`~xpublish.SingleDatasetRest` is used instead.
+For serving a single dataset the {class}`~xpublish.SingleDatasetRest` is used instead.
 
+```{eval-rst}
 .. autosummary::
  :toctree: generated/
 
  SingleDatasetRest
+```
 
-Dataset.rest (xarray accessor)
-==============================
+## Dataset.rest (xarray accessor)
 
-This accessor extends :py:class:`xarray.Dataset` with the same interface than
-:class:`~xpublish.SingleDatasetRest`. It is a convenient method for publishing one single
+This accessor extends {py:class}`xarray.Dataset` with the same interface than
+{class}`~xpublish.SingleDatasetRest`. It is a convenient method for publishing one single
 dataset. Proper use of this accessor should be like:
 
-.. code-block:: python
-
- >>> import xarray as xr # first import xarray
- >>> import xpublish   # import xpublish (the dataset 'rest' accessor is registered)
- >>> ds = xr.Dataset() # create or load an xarray Dataset
- >>> ds.rest(...)  # call the 'rest' accessor on the dataset
- >>> ds.rest.<meth_or_prop> # access to the methods and properties listed below
+```
+>>> import xarray as xr # first import xarray
+>>> import xpublish   # import xpublish (the dataset 'rest' accessor is registered)
+>>> ds = xr.Dataset() # create or load an xarray Dataset
+>>> ds.rest(...)  # call the 'rest' accessor on the dataset
+>>> ds.rest.<meth_or_prop> # access to the methods and properties listed below
+```
 
+```{eval-rst}
 .. currentmodule:: xarray
+```
 
 **Calling the accessor**
 
+```{eval-rst}
 .. autosummary::
  :toctree: generated/
  :template: autosummary/accessor_callable.rst
 
  Dataset.rest
+```
 
 **Properties**
 
+```{eval-rst}
 .. autosummary::
  :toctree: generated/
  :template: autosummary/accessor_attribute.rst
 
  Dataset.rest.app
  Dataset.rest.cache
+```
 
 **Methods**
 
+```{eval-rst}
 .. autosummary::
  :toctree: generated/
  :template: autosummary/accessor_method.rst
 
  Dataset.rest.serve
+```
 
-FastAPI dependencies
-====================
+## FastAPI dependencies
 
-The functions below are defined in module ``xpublish.dependencies`` and can
-be used as `FastAPI dependencies <https://fastapi.tiangolo.com/tutorial/dependencies>`_
+The functions below are defined in module `xpublish.dependencies` and can
+be used as [FastAPI dependencies](https://fastapi.tiangolo.com/tutorial/dependencies)
 when creating custom API endpoints directly.
 
-When creating routers with plugins, instead use ``xpublish.Dependency`` that will be
-passed in to the ``Plugin.app_router`` or ``Plugin.dataset_router`` method.
+When creating routers with plugins, instead use `xpublish.Dependency` that will be
+passed in to the `Plugin.app_router` or `Plugin.dataset_router` method.
 
+```{eval-rst}
 .. currentmodule:: xpublish.dependencies
+```
 
+```{eval-rst}
 .. autosummary::
  :toctree: generated/
 
@@ -119,13 +135,17 @@ passed in to the ``Plugin.app_router`` or ``Plugin.dataset_router`` method.
  get_zmetadata
  get_plugins
  get_plugin_manager
+```
 
-Plugins
+## Plugins
 
-Plugins are inherit from the :class:`~xpublish.Plugin` class, and implement various hooks.
+Plugins are inherit from the {class}`~xpublish.Plugin` class, and implement various hooks.
 
+```{eval-rst}
 .. currentmodule:: xpublish
+```
 
+```{eval-rst}
 .. autosummary::
  :toctree: generated/
 
@@ -137,3 +157,4 @@ Plugins are inherit from the :class:`~xpublish.Plugin` class, and implement vari
  plugins.manage.find_default_plugins
  plugins.manage.load_default_plugins
  plugins.manage.configure_plugins
+```

docs/source/conf.py

@@ -46,11 +46,16 @@
  'sphinxcontrib.autodoc_pydantic',
  'sphinx_autosummary_accessors',
  'sphinx_autodoc_typehints',
+ 'sphinx_design',
+ 'myst_parser',
 ]
 
+myst_enable_extensions = []
+myst_heading_anchors = 6
+
 extlinks = {
- 'issue': ('https://github.com/xarray-contrib/xpublish/issues/%s', 'GH#'),
- 'pr': ('https://github.com/xarray-contrib/xpublish/pull/%s', 'GH#'),
+ 'issue': ('https://github.com/xpublish-community/xpublish/issues/%s', 'GH#'),
+ 'pr': ('https://github.com/xpublish-community/xpublish/pull/%s', 'GH#'),
 }
 # Add any paths that contain templates here, relative to this directory.
 templates_path = ['_templates', sphinx_autosummary_accessors.templates_path]
@@ -140,7 +145,7 @@
  'icon_links': [
  {
  'name': 'GitHub',
- 'url': 'https://github.com/xarray-contrib/xpublish', # required
+ 'url': 'https://github.com/xpublish-community/xpublish', # required
  'icon': 'fa-brands fa-square-github',
  'type': 'fontawesome',
  }
@@ -149,7 +154,7 @@
 }
 
 html_context = {
- 'github_user': 'xarray-contrib',
+ 'github_user': 'xpublish-community',
  'github_repo': 'xpublish',
  'github_version': 'main',
  'doc_path': 'docs/source/',

docs/source/contributing.rst

@@ -16,7 +16,7 @@ Feature requests and feedback
 Do you like Xpublish? Share some love on Twitter or in your blog posts!
 
 We'd also like to hear about your propositions and suggestions. Feel free to
-`submit them as issues <https://github.com/xarray-contrib/xpublish>`_ and:
+`submit them as issues <https://github.com/xpublish-community/xpublish>`_ and:
 
 * Explain in detail how they should work.
 * Keep the scope as narrow as possible. This will make it easier to implement.
@@ -26,7 +26,7 @@ We'd also like to hear about your propositions and suggestions. Feel free to
 Report bugs
 -----------
 
-Report bugs for Xpublish in the `issue tracker <https://github.com/xarray-contrib/xpublish>`_.
+Report bugs for Xpublish in the `issue tracker <https://github.com/xpublish-community/xpublish>`_.
 
 If you are reporting a bug, please include:
 
@@ -45,7 +45,7 @@ fix the bug itself.
 Fix bugs
 --------
 
-Look through the `GitHub issues for bugs <https://github.com/xarray-contrib/xpublish/labels/type:%20bug>`_.
+Look through the `GitHub issues for bugs <https://github.com/xpublish-community/xpublish/labels/type:%20bug>`_.
 
 Talk to developers to find out how you can fix specific bugs.
 
@@ -81,7 +81,7 @@ Preparing Pull Requests
 -----------------------
 
 #. Fork the
- `xpublish GitHub repository <https://github.com/xarray-contrib/xpublish>`__. It's
+ `xpublish GitHub repository <https://github.com/xpublish-community/xpublish>`__. It's
  fine to use ``xpublish`` as your fork repository name because it will live
  under your user.
 
@@ -134,5 +134,5 @@ Preparing Pull Requests
  head-fork: YOUR_GITHUB_USERNAME/xpublish
  compare: your-branch-name
 
- base-fork: xarray-contrib/xpublish
+ base-fork: xpublish-community/xpublish
  base: main

docs/source/ecosystem/index.md

@@ -0,0 +1,31 @@
+# Ecosystem
+
+Xpublish's ecosystem is made up of plugins and servers, and the folks who build and run them.
+
+## Connect
+
+We have two main venues for discussing Xpublish and it's ecosystem, Github Discussions and Slack.
+
+### Github Discussions
+
+For longer form discussions, we can be found in [Github Discussions](https://github.com/xpublish-community/community/discussions?discussions_q=).
+
+### Slack
+
+Xpublish has a channel (`#xpublish`) on [ESIP](https://www.esipfed.org/)'s (Earth Science Information Partners) Slack. (Insert Rich's justification here about ESIP being the biggest unbrella that he can find) [Join here](https://join.slack.com/t/esip-all/shared_invite/zt-1omjufm9z-iH8Gf7gmmsm2SiS5Xh6BlQ)
+
+## Server distributions
+
+- [XREDS](https://github.com/asascience-open/xreds) from RPS
+- [Xpublish Host](https://github.com/axiom-data-science/xpublish-host) from Axiom Data Science
+
+## Plugins
+
+- [OGC EDR](https://github.com/xpublish-community/xpublish-edr/)
+- [OpenDAP](https://github.com/xpublish-community/xpublish-opendap/)
+- [WMS](https://github.com/xpublish-community/xpublish-wms)
+- [Intake](https://github.com/axiom-data-science/xpublish-intake)
+
+## Experiments
+
+We have a separate Github organization ([xpublish-experiments](https://github.com/xpublish-experiments)) for things that you've tried and want to share with the community, but don't necessarily want to maintain.

docs/source/getting-started/index.md

@@ -0,0 +1,13 @@
+# Getting started
+
+The getting started guide aims to get you using Xpublish productively as quickly as possible.
+It is designed as an entry point for new users, and it provided an introduction to Xpublish’s main concepts.
+
+```{toctree}
+---
+hidden:
+---
+why-xpublish
+installation
+tutorial/index
+```

docs/source/getting-started/tutorial/dataset-provider-plugin.md

@@ -0,0 +1,14 @@
+# Building a dataset provider plugin
+
+So far, we've been eagerly loading datasets for Xpublish to serve, but this tends not to scale well between memory needs and slow startup. Xpublish plugins can also be __Dataset Providers__ and handle loading of datasets on request.
+
+This also allows organizations to quickly be able to adapt Xpublish to work in their own environment, rather than needing Xpublish to explicitly support it.
+
+```{literalinclude} dataset-provider-plugin.py
+```
+
+With this plugin, Xpublish can serve the same datasets as we explictly defined and loaded in [serving multiple datasets](./serving-multiple-datasets.md), as well as any others supported by [`xr.tutorial`](https://github.com/pydata/xarray/blob/main/xarray/tutorial.py)
+
+```{note}
+For more details on building dataset provider plugins, please see the [plugin user guide](../../user-guide/plugins.md#dataset-provider-plugins)
+```

docs/source/getting-started/tutorial/dataset-provider-plugin.py

@@ -0,0 +1,24 @@
+import xarray as xr
+from requests import HTTPError
+
+from xpublish import Plugin, Rest, hookimpl
+
+
+class TutorialDataset(Plugin):
+ name = 'xarray-tutorial-dataset'
+
+ @hookimpl
+ def get_datasets(self):
+ return list(xr.tutorial.file_formats)
+
+ @hookimpl
+ def get_dataset(self, dataset_id: str):
+ try:
+ return xr.tutorial.open_dataset(dataset_id)
+ except HTTPError:
+ return None
+
+
+rest = Rest({})
+rest.register_plugin(TutorialDataset())
+rest.serve()