Merge pull request #790 from NCAR/rda_refs

Updated RDA addresses and dataset names

conf.py

@@ -10,7 +10,7 @@
 # add these directories to sys.path here. If the directory is relative to the
 # documentation root, use os.path.abspath to make it absolute, like shown here.
 #
-# import os
+import os
 # import sys
 # sys.path.insert(0, os.path.abspath('.'))
 
@@ -30,6 +30,7 @@
 # extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
 # ones.
 extensions = [
+     'sphinx_copybutton',
      'sphinx_rtd_theme',
      'sphinx.ext.autodoc',
      'sphinx.ext.mathjax'
@@ -86,3 +87,9 @@
 html_static_path = ['guide/_static']
 
 html_css_files = ['css/custom.css']
+
+# include references
+with open(os.path.join(os.path.dirname(__file__), 'guide/references.rst')) as f:
+    references_content = f.read()
+
+rst_prolog = references_content

guide/references.rst

@@ -0,0 +1,26 @@
+.. These should be referenced with the syntax
+   `text to appear in doc <LinkNameFoundHere_>`_
+
+.. CAM6 reanalysis data
+.. _CAM6rean_rda : https://rda.ucar.edu/datasets/d345000/
+
+.. CAM4 reanalysis data
+.. _CAM4rean_rda : https://rda.ucar.edu/datasets/d199001/
+
+.. CAM6 reanalysis wiki
+.. _CAM6rean_wiki : https://github.com/NCAR/DART/wiki/1-degree,-CAM6,-ensemble-reanalysis-for-CESM-experiments-(2011-thru-2019):-DATM,-hindcasts,-model-evaluation
+
+.. CAM4 reanalysis wiki
+.. _CAM4rean_wiki : https://github.com/NCAR/DART/wiki/2-degree-DATM-ensemble-for-CESM-experiments-(1998-thru-2010)
+
+.. NCEP's PREPBUFR files (prepqm) in NSF NCAR's Research Data Archive
+.. _NCEP_obs : https://rda.ucar.edu/datasets/d337000/
+
+.. NCEP+NCAR Reanalysis observations dataset
+.. _NCEP+NCAR_obs : https://rda.ucar.edu/datasets/d090000/
+
+.. WOD (World Ocean Database) 
+.. _WOD_obs : https://rda.ucar.edu/datasets/d285000/
+
+.. Sample observation sequence files
+.. _DART_obs : http://www.image.ucar.edu/pub/DART/Obs_sets/

guide/requirements.txt

@@ -1,2 +1,3 @@
 Sphinx==6.2.1
 sphinx-rtd-theme==1.2.2
+sphinx_copybutton

index.rst

@@ -622,3 +622,4 @@ References
    :hidden:
 
    guide/404
+   guide/references

models/POP/readme.rst

@@ -62,7 +62,7 @@ making minor edits to set details that are specific to your project. The setup
 scripts create a CESM case in which POP is configured using a 1° horizontal
 grid, and uses the eddy parametrization of  Gent and McWilliams (1990). [2]_
 The CICE model is active and atmospheric forcing is provided by the `CAM6 DART
-Reanalysis <https://rda.ucar.edu/datasets/ds345.0/>`_.
+Reanalysis <CAM6rean_rda_>`_.
 
 The filesystem attached to NSF NCAR's supercomputer is known as the Globally
 Accessible Data Environment (GLADE). All filepaths on GLADE have the structure:
@@ -231,7 +231,7 @@ Observation sequence files
 When ``setup_CESM_hybrid_ensemble.csh`` is used to create an assimilation
 experiment, ``DART_params.csh`` configures the experiment to assimilate 
 observation sequence files from the World Ocean Database 2013 (WOD13; Boyer et
-al. 2013 [5]_).
+al. 2013 [5]_, `NSF NCAR's Research Data Archive <WOD_obs_>`_.).
 
 The WOD13 dataset comprises data from 2005-01-01 to 2016-12-31 and contains the
 following observation types:
@@ -262,7 +262,7 @@ the following directory on GLADE:
 
 .. code-block::
 
-   /glade/p/cisl/dares/Observations/WOD13
+   /glade/campaign/cisl/fs1_p_relocation/dares/Observations/WOD13
 
 The subdirectories are formatted in ``YYYYMM`` order.
 
@@ -272,7 +272,7 @@ are also stored in the following directory on GLADE:
 
 .. code-block::
 
-   /glade/p/cisl/dares/Observations/WOD09
+   /glade/campaign/cisl/fs1_p_relocation/dares/WOD09
 
 These observation sequence files can be assimilated by changing the
 ``BASEOBSDIR`` variable in ``DART_params.csh``.
@@ -285,7 +285,7 @@ Data atmosphere streams files
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
 The setup scripts configure the CESM case with atmospheric forcing from the 
-`CAM6 DART Reanalysis <https://rda.ucar.edu/datasets/ds345.0/>`_. The coupler 
+`CAM6 DART Reanalysis <CAM6rean_rda_>`_. The coupler 
 history files from this reanalysis are referenced in
 ``user_datm.streams*template`` files. These ``user_datm.streams*template``
 files are contained in the same directory as the setup scripts and are

models/POP/shell_scripts/cesm2_1/user_datm.streams.txt.CPLHISTForcing.Solar_template

@@ -1,11 +1,11 @@
 <dataSource>
-    The CAM6-DART Ensemble Reanalysis (NCAR RDA ds345.0) contains
+    The CAM6-DART Ensemble Reanalysis (NCAR RDA d345000) contains
     DATM forcing files particularly appropriate for CESM experiments.
-    https://rda.ucar.edu/datasets/ds345.0
+    https://rda.ucar.edu/datasets/d345000
 
     They are available on spinning disk (compressed) on
     the NCAR supercomputers at:
-    /glade/collections/rda/data/ds345.0/cpl_unzipped
+    /glade/campaign/collections/rda/data/d345000/cpl_unzipped
 </dataSource>
 <domainInfo>
     <variableNames>
@@ -16,7 +16,7 @@
         doma_mask     mask
     </variableNames>
     <filePath>
-        /glade/collections/rda/data/ds345.0/cpl_unzipped/NINST
+        /glade/campaign/collections/rda/data/d345000/cpl_unzipped/NINST
     </filePath>
     <fileNames>
         f.e21.FHIST_BGC.f09_025.CAM6assim.011.cpl_NINST.ha2x3h.2011.nc
@@ -30,7 +30,7 @@
         a2x3h_Faxa_swvdf     swvdf
     </variableNames>
     <filePath>
-        /glade/collections/rda/data/ds345.0/cpl_unzipped/NINST
+        /glade/campaign/collections/rda/data/d345000/cpl_unzipped/NINST
     </filePath>
     <fileNames>
         f.e21.FHIST_BGC.f09_025.CAM6assim.011.cpl_NINST.ha2x3h.2011.nc

models/POP/shell_scripts/cesm2_1/user_datm.streams.txt.CPLHISTForcing.State1hr_template

@@ -1,11 +1,11 @@
 <dataSource>
-    The CAM6-DART Ensemble Reanalysis (NCAR RDA ds345.0) contains
+    The CAM6-DART Ensemble Reanalysis (NCAR RDA d345000) contains
     DATM forcing files particularly appropriate for CESM experiments.
-    https://rda.ucar.edu/datasets/ds345.0
+    https://rda.ucar.edu/datasets/d345000
 
     They are available on spinning disk (compressed) on
     the NCAR supercomputers at:
-    /glade/collections/rda/data/ds345.0/cpl_unzipped
+    /glade/campaign/collections/rda/data/d345000/cpl_unzipped
 </dataSource>
 <domainInfo>
     <variableNames>
@@ -16,7 +16,7 @@
         doma_mask     mask
     </variableNames>
     <filePath>
-        /glade/collections/rda/data/ds345.0/cpl_unzipped/NINST
+        /glade/campaign/collections/rda/data/d345000/cpl_unzipped/NINST
     </filePath>
     <fileNames>
         f.e21.FHIST_BGC.f09_025.CAM6assim.011.cpl_NINST.ha2x1h.2011.nc
@@ -28,7 +28,7 @@
         a2x1h_Sa_v           v
     </variableNames>
     <filePath>
-        /glade/collections/rda/data/ds345.0/cpl_unzipped/NINST
+        /glade/campaign/collections/rda/data/d345000/cpl_unzipped/NINST
     </filePath>
     <tInterpAlgo>
         linear

models/POP/shell_scripts/cesm2_1/user_datm.streams.txt.CPLHISTForcing.State3hr_template

@@ -1,11 +1,11 @@
 <dataSource>
-    The CAM6-DART Ensemble Reanalysis (NCAR RDA ds345.0) contains
+    The CAM6-DART Ensemble Reanalysis (NCAR RDA d345000) contains
     DATM forcing files particularly appropriate for CESM experiments.
-    https://rda.ucar.edu/datasets/ds345.0
+    https://rda.ucar.edu/datasets/d345000
 
     They are available on spinning disk (compressed) on
     the NCAR supercomputers at:
-    /glade/collections/rda/data/ds345.0/cpl_unzipped
+    /glade/campaign/collections/rda/data/d345000/cpl_unzipped
 </dataSource>
 <domainInfo>
     <variableNames>
@@ -16,7 +16,7 @@
         doma_mask     mask
     </variableNames>
     <filePath>
-        /glade/collections/rda/data/ds345.0/cpl_unzipped/NINST
+        /glade/campaign/collections/rda/data/d345000/cpl_unzipped/NINST
     </filePath>
     <fileNames>
         f.e21.FHIST_BGC.f09_025.CAM6assim.011.cpl_NINST.ha2x3h.2011.nc
@@ -34,7 +34,7 @@
         a2x3h_Sa_pslv        pslv
     </variableNames>
     <filePath>
-        /glade/collections/rda/data/ds345.0/cpl_unzipped/NINST
+        /glade/campaign/collections/rda/data/d345000/cpl_unzipped/NINST
     </filePath>
     <tInterpAlgo>
         linear

models/POP/shell_scripts/cesm2_1/user_datm.streams.txt.CPLHISTForcing.nonSolarFlux_template

@@ -1,11 +1,11 @@
 <dataSource>
-    The CAM6-DART Ensemble Reanalysis (NCAR RDA ds345.0) contains
+    The CAM6-DART Ensemble Reanalysis (NCAR RDA d345000) contains
     DATM forcing files particularly appropriate for CESM experiments.
-    https://rda.ucar.edu/datasets/ds345.0
+    https://rda.ucar.edu/datasets/d345000
 
     They are available on spinning disk (compressed) on
     the NCAR supercomputers at:
-    /glade/collections/rda/data/ds345.0/cpl_unzipped
+    /glade/campaign/collections/rda/data/d345000/cpl_unzipped
 </dataSource>
 <domainInfo>
     <variableNames>
@@ -16,7 +16,7 @@
         doma_mask     mask
     </variableNames>
     <filePath>
-         /glade/collections/rda/data/ds345.0/cpl_unzipped/NINST
+         /glade/campaign/collections/rda/data/d345000/cpl_unzipped/NINST
     </filePath>
     <fileNames>
         f.e21.FHIST_BGC.f09_025.CAM6assim.011.cpl_NINST.ha2x3h.2011.nc
@@ -31,7 +31,7 @@
         a2x3h_Faxa_lwdn      lwdn
     </variableNames>
     <filePath>
-        /glade/collections/rda/data/ds345.0/cpl_unzipped/NINST
+        /glade/campaign/collections/rda/data/d345000/cpl_unzipped/NINST
     </filePath>
     <fileNames>
         f.e21.FHIST_BGC.f09_025.CAM6assim.011.cpl_NINST.ha2x3h.2011.nc

models/cam-fv/readme.rst

@@ -5,8 +5,8 @@ Overview
 --------
 
 The larger context of the Community Earth System Model and DART interactions
-is described in the `CESM readme <../CESM/readme.html>`_
-This document focuses on the several `atmospheric models <http://www2.cesm.ucar.edu/models>`__
+is described in the :doc:`CESM readme <../CESM/readme>`
+This document focuses on the several `atmospheric models <http://www2.cesm.ucar.edu/models>`_
 that have been developed or adapted to run in the CESM environment. 
 They are named according to their dynamical core ("dycore").
 As of 2021 these include Finite Volume Community Atmosphere Model (CAM-FV), 
@@ -15,18 +15,16 @@ The DART system has supported data assimilation into CAM-FV continuously for man
 An interface to CAM-SE was added to DART in 2022.
 An interface to MPAS is being developed (contact us about the current status).
 
-.. |CAM6_Rean| replace:: 1 degree reanalysis wiki
-.. _CAM6_Rean: https://github.com/NCAR/DART/wiki/1-degree,-CAM6,-ensemble-reanalysis-for-CESM-experiments-(2011-thru-2019):-DATM,-hindcasts,-model-evaluation
-
 The flexibility of the DART environment has led to its use
 by graduate students, post-graduates, and scientists at universities and
 research labs to conduct data assimilation research. Others are using the
 products of data assimilation (analyses), which were produced at NSF NCAR
 using CESM+DART, to conduct related research. 
-The latest reanalysis is described in the DART |CAM6_Rean|_
+The latest reanalysis is described in the `CAM6 Reanalysis wiki <CAM6rean_wiki_>`_
+
 
 The variety of research can be sampled on the DART  
-`Publications <https://dart.ucar.edu/pages/Publications.html>`__ page.
+`Publications page <https://dart.ucar.edu/pages/Publications.html>`__ .
 
 Terminology
 ~~~~~~~~~~~
@@ -56,7 +54,7 @@ with two independent main characteristics. CESM labels these as:
      model domain vary with those choices. 
    - The combinations of parameterizations and vertical grids are named: CAM3.5,
      CAM5, CAM#, ... WACCM, WACCM#, WACCM-X, CAM-Chem.  
-   - The compset is specified as described in the `CESM readme <../CESM/readme.html>`_.
+   - The compset is specified as described in the :doc:`CESM readme <../CESM/readme>`.
 
 **ensemble, multi-instance, and multidriver**
 
@@ -116,10 +114,11 @@ Reanalyses
 ~~~~~~~~~~
 
 There have been two large-scale reanalysis efforts using CAM-FV and DART. 
+
 The **CAM6 Data Assimilation Research Testbed (DART) Reanalysis**
-is archived in the NSF NCAR Research Data Archive 
-`DS345.0 <https://rda.ucar.edu/datasets/ds345.0/#!description>`__ .
-(See the |CAM6_Rean|_ ).
+is archived in the NSF NCAR Research Data Archive (RDA) 
+`d345000 <CAM6rean_rda_>`_.
+(See the `wiki <CAM6rean_wiki_>`_ for details).
 It contains just under 120Tb (yes Tb) of data:
 
    These CAM6+DART Reanalysis data 
@@ -136,10 +135,10 @@ It contains just under 120Tb (yes Tb) of data:
 
 
 An earlier, more limited dataset can be found in the 
-`**Ensemble of Atmospheric Forcing Files from a CAM4-FV Reanalysis** 
-<https://github.com/NCAR/DART/wiki/2-degree-DATM-ensemble-for-CESM-experiments-(1998-thru-2010)>`__
+`Ensemble of Atmospheric Forcing Files from a CAM4-FV Reanalysis 
+<CAM4rean_wiki_>`_
 is archived in the NSF NCAR Research Data Archive 
-`DS199.1 <https://rda.ucar.edu/datasets/ds199.1/>`__ .
+`d199001 <CAM4rean_rda_>`_ .
 It contains about 1.5Tb of data:
 
    This dataset contains files that are an ensemble of 'coupler history' 
@@ -154,9 +153,6 @@ It contains about 1.5Tb of data:
    for CESM component sets that require a data atmosphere. Some example 
    stream text files are included in the RDA to illustrate how to use these data.
 
-..  * CAM4, 2 degree, 2000-2010, `Reanalysis <https://rda.ucar.edu/datasets/ds199.1>`__
-..  * files from the old "large file site": http://www.image.ucar.edu/pub/DART/CAM/
-
 Observations
 ------------
 
@@ -165,16 +161,16 @@ which contain the types of observations in the table below
 ("T" = temperature, "U" = zonal wind, "V" = meridional wind,
 "Q" = specific humidity, "refractivity" = the bending of light by density variations).
 These files are available on NSF NCAR's glade file system:
-/glade/p/cisl/dares/Observations/NCEP+ACARS+GPS+AIRS/Thinned_x9x10.
+/glade/campaign/cisl/fs1_p_relocation/dares/Observations/NCEP+ACARS+GPS+AIRS/Thinned_x9x10.
 Versions of these files, which also have the results of the reanalysis in them,
-are available from the RDA ds345.0 linked above.
+are available from the `RDA d345000 <CAM6rean_rda_>`_ .
 
 NCEP
    NCEP's PREPBUFR files (prepqm) in NSF NCAR's Research Data Archive:
-   (https://rda.ucar.edu/datasets/ds090.0/)
+   `newer <NCEP_obs_>`_ or `older <NCEP+NCAR_obs_>`_
 
 COSMIC
-   This site (http://www.cosmic.ucar.edu/) provides atmospheric refractivity 
+   `UCAR's COSMIC program <http://www.cosmic.ucar.edu/>`_ provides atmospheric refractivity 
    from a variety of satellites (including COSMIC), which receive Global Positioning System 
    radio occultation signals.
 
@@ -202,9 +198,8 @@ AIRS
 | altimeter            | Radiosondes, bouys                     | global surface           | NCEP   |
 +----------------------+----------------------------------------+--------------------------+--------+
 
-Sample sets of observations, which can be used with CAM+DART assimilations, can
-be found at http://www.image.ucar.edu/pub/DART/Obs_sets/ of which the NCEP BUFR
-observations are the most widely used.
+`Sample sets of observations <DART_obs_>`_ can be used with CAM+DART assimilations, 
+of which the NCEP BUFR observations are the most widely used.
 
 The CAM-FV DART Interface
 =========================
@@ -228,7 +223,7 @@ Setup Scripts
 Unlike pre-Manhattan versions of DART-CAM, CESM (CAM) runs using its normal scripts, 
 then stops and calls a DART script, which does the desired assimilation tasks, 
 then returns to the CESM run script for the next model advance. See the CESM
-interface documentation in the `CESM readme <../CESM/readme.html>`_
+interface documentation in the :doc:`CESM readme <../CESM/readme>`
 for more general information about
 running DART with CESM. Due to the complexity of the CESM software environment,
 the versions of CESM which can be used for assimilation are more restricted than
@@ -283,7 +278,7 @@ The scripts  in cesm#_# will handle, for that CESM version;
 Physics variants of other dycores are handled in other "model" interfaces,
 such as models/cam-se.
 
-.. _reanalysis: https://github.com/kdraeder/cesm}{github.com/kdraeder/cesm
+.. _reanalysis: https://github.com/kdraeder/cesm
 
 .. _SourceMods: 
 
@@ -886,7 +881,6 @@ before each hindcast by:
 
 -  ISSUE: Improve this page
     * Add links and references to this document.
-    * Publications web page.
     * CAM-chem; link?  More description?
 
 -  ISSUE?; ``model_interpolate`` assumes that obs with a vertical location have
@@ -905,7 +899,7 @@ before each hindcast by:
 References and Acknowledgements
 -------------------------------
 
--  `CESM homepage <https://www.cesm.ucar.edu/models/cesm1.3/>`__
+-  `CESM homepage <https://www.cesm.ucar.edu/models/cesm1.3/>`_
 
 Ave Arellano did the first work with CAM-Chem, assimilating MOPPITT CO
 observations into CAM-Chem. Jerome Barre and Benjamin Gaubert took up the