Update develop-ref after #2495 (#2499)

Co-authored-by: johnhg <johnhg@ucar.edu> Co-authored-by: Julie Prestopnik <jpresto@seneca.rap.ucar.edu> Co-authored-by: Seth Linden <linden@seneca.rap.ucar.edu> Co-authored-by: John Halley Gotway <johnhg@ucar.edu> Co-authored-by: Dave Albo <dave@seneca.rap.ucar.edu> Co-authored-by: MET Tools Test Account <met_test@seneca.rap.ucar.edu> Co-authored-by: Howard Soh <hsoh@seneca.rap.ucar.edu> Co-authored-by: Jonathan Vigh <jvigh@ucar.edu> Co-authored-by: jprestop <jpresto@ucar.edu> Co-authored-by: Seth Linden <linden@ucar.edu> Co-authored-by: hsoh-u <hsoh@ucar.edu> Co-authored-by: davidalbo <dave@ucar.edu> Co-authored-by: lisagoodrich <33230218+lisagoodrich@users.noreply.github.com> Co-authored-by: George McCabe <23407799+georgemccabe@users.noreply.github.com> Co-authored-by: Daniel Adriaansen <dadriaan@ucar.edu> Co-authored-by: Lisa Goodrich <lisag@ucar.edu> fix #2309 develop tcmpr (#2310) fix #2306 ascii2nc airnow hourly (#2314) fix_spread_md (#2335) fix #2389 develop flowchart (#2392) Fix Python environment issue (#2407) fix definitions of G172 and G220 based on comments in NOAA-EMC/NCEPLIBS-w3emc#157. (#2406) fix #2380 develop override (#2382) fix #2408 develop empty config (#2410) fix #2390 develop compile zlib (#2404) fix #2412 develop climo (#2422) fix #2437 develop convert (#2439) fix for develop, for #2437, forgot one reference to the search_parent for a dictionary lookup. fix #2452 develop airnow (#2454) fix #2449 develop pdf (#2464) fix #2402 develop sonarqube (#2468) fix #2426 develop buoy (#2475)
dtcenter · Mar 31, 2023 · fef0a54 · fef0a54
1 parent 86040a3
commit fef0a54
Show file tree

Hide file tree

Showing 22 changed files with 3,485 additions and 1,478 deletions.
diff --git a/.github/workflows/documentation.yml b/.github/workflows/documentation.yml
@@ -8,7 +8,7 @@ on:
       - main_*
       - bugfix_*
     paths:
-      - met/docs/**
+      - docs/**
   pull_request:
     types: [opened, reopened, synchronize]
   workflow_dispatch:
@@ -26,6 +26,7 @@ jobs:
         run: |
           python -m pip install --upgrade python-dateutil requests sphinx \
           sphinx-gallery Pillow sphinx_rtd_theme sphinx-panels
+          python -m pip install -r docs/requirements.txt
       - name: Build docs
         run: ./.github/jobs/build_documentation.sh
       - uses: actions/upload-artifact@v3

diff --git a/data/table_files/ndbc_stations.xml b/data/table_files/ndbc_stations.xml
diff --git a/docs/Users_Guide/appendixC.rst b/docs/Users_Guide/appendixC.rst
@@ -1101,7 +1101,7 @@ Called "IGN" in ECNT output :numref:`table_ES_header_info_es_out_ECNT`
 
 The ignorance score (IGN) is the negative logarithm of a predictive probability density function (:ref:`Gneiting et al., 2004 <Gneiting-2004>`). In MET, the IGN is calculated based on a normal approximation to the forecast distribution (i.e. a normal pdf is fit to the forecast values). This approximation may not be valid, especially for discontinuous forecasts like precipitation, and also for very skewed forecasts. For a single normal distribution **N** with parameters :math:`\mu \text{ and } \sigma`, the ignorance score is
 
-.. math:: \text{ign} (N( \mu, \sigma),y) = \frac{1}{2} \ln (2 \pi \sigma^2 ) + \frac{(y - \mu)^2}{\sigma^2}.
+.. math:: \text{ign} (N( \mu, \sigma),y) = \frac{1}{2} \ln (2 \pi \sigma^2 ) + \frac{(y - \mu)^2}{2\sigma^2}.
 
 Accumulation of the ignorance score for many forecasts is via the average of individual ignorance scores. This average ignorance score is the value output by the MET software. Like many error statistics, the IGN is negatively oriented, so smaller numbers indicate better forecasts.
 

diff --git a/docs/Users_Guide/appendixF.rst b/docs/Users_Guide/appendixF.rst
@@ -9,6 +9,8 @@ Introduction
 
 MET includes the ability to embed Python to a limited degree. Users may use Python scripts and whatever associated Python packages they wish in order to prepare 2D gridded data fields, point observations, and matched pairs as input to the MET tools. We fully expect that this degree of embedding will increase in the future. In addition, plans are in place to extend Python with MET in upcoming releases, allowing users to invoke MET tools directly from their Python script. While MET version 8.0 was built on Python 2.x, MET versions 9.0 and beyond are built on Python 3.6+.
 
+.. _compiling_python_support:
+
 Compiling Python Support
 ========================
 

diff --git a/docs/Users_Guide/config_options.rst b/docs/Users_Guide/config_options.rst
@@ -272,6 +272,33 @@ The default table can be found in the installed
 XML content for all stations that allows lookups of latitude, longitude,
 and, in some cases, elevation for all stations based on stationId.
 
+This set of stations comes from 2 online sources: the
+`active stations website <https://www.ndbc.noaa.gov/activestations.xml>`_
+and the `complete stations website <https://www.airnow.gov>`_.
+As these lists can change as a function of time, a script can be run to pull
+down the contents of both websites and merge any changes with the existing stations
+file content, creating an updated stations file locally.
+The MET_NDBC_STATIONS environment variable can be then set to refer to this newer
+stations file.  Also, the MET development team will periodically
+run this script and update *share/met/table_files/ndbc_stations.xml*.
+
+To run this utility:
+
+.. code-block:: none
+
+  build_ndbc_stations_from_web.py <-d> <-p> <-o OUTPUT_FILE>
+
+  Usage: build_ndbc_stations_from_web.py [options]
+  Options:
+    -h, --help            show this help message and exit
+    -d, --diagnostic      Rerun using downlaoded files, skipping download step (optional, default: False)
+    -p, --prune           Prune files that are no longer online (optional, default: False)
+    -o OUT_FILE, --out=OUT_FILE
+                          Save the text into the named file (optional, default: merged.txt)
+
+NOTE: The downloaded files are written to a subdirectory ndbc_temp_data which
+can be deleted once the final output file is created.
+
 MET_BASE
 ^^^^^^^^
 

diff --git a/docs/Users_Guide/release-notes.rst b/docs/Users_Guide/release-notes.rst
@@ -9,6 +9,44 @@ When applicable, release notes are followed by the GitHub issue number which des
 enhancement, or new feature (`MET GitHub issues <https://github.com/dtcenter/MET/issues>`_).
 Important issues are listed **in bold** for emphasis.
 
+MET Version 11.1.0-beta1 release notes (20230228)
+-------------------------------------------------
+
+  .. dropdown:: Repository, build, and test
+
+     * Add modulefiles for supported systems to the repository (`#2415 <https://github.com/dtcenter/MET/issues/2415>`_).
+     * Add LICENSE.md to the repository (`#2461 <https://github.com/dtcenter/MET/issues/2461>`_).
+     * Update the copyright year to 2023 and increase the version number to 11.1.0 (`#2469 <https://github.com/dtcenter/MET/issues/2469>`_).
+
+  .. dropdown:: Documentation 
+
+     * Enhance the Release Notes by adding dropdown menus (`#2146 <https://github.com/dtcenter/MET/issues/2146>`_).
+
+  .. dropdown:: Enhancements 
+
+     * Convert the python list to the numpy array for the python embedding at the base class (`#2386 <https://github.com/dtcenter/MET/issues/2386>`_).
+     * Refine Python runtime environment (`#2388 <https://github.com/dtcenter/MET/issues/2388>`_).
+     * Upgrade to using Python 3.10.4 (`#2421 <https://github.com/dtcenter/MET/issues/2421>`_).
+     * **Enhance TC-Pairs to disable the output of consensus track members** (`#2429 <https://github.com/dtcenter/MET/issues/2429>`_).
+
+  .. dropdown:: Bugfixes 
+
+     * Bugfix: Fix the MET CF-Compliant NetCDF library code to Polar Stereographic data from NSIDC Sea Ice Edge NetCDF files (`#2218 <https://github.com/dtcenter/MET/issues/2218>`_).
+     * Bugfix: Remove override keyword to avoid C++11 dependency (`#2380 <https://github.com/dtcenter/MET/issues/2380>`_).
+     * Bugfix: Fix ASCII2NC to not compute AOD 550 if other inputs are negative values (`#2383 <https://github.com/dtcenter/MET/issues/2383>`_).
+     * Bugfix: Fix PB2NC to report accurate total observation counts in log messages (`#2387 <https://github.com/dtcenter/MET/issues/2387>`_).
+     * Bugfix: Update the MET flowchart for version 11.0.0 (`#2389 <https://github.com/dtcenter/MET/issues/2389>`_).
+     * Bugfix: Fix issues with the met_compile_all.sh script and associated tar files (`#2390 <https://github.com/dtcenter/MET/issues/2390>`_).
+     * Bugfix: Correct definitions of NCEP grid numbers 172 and 220 (`#2399 <https://github.com/dtcenter/MET/issues/2399>`_).
+     * Bugfix: Address MET-11.0.0 SonarQube Blocker Bugs (`#2402 <https://github.com/dtcenter/MET/issues/2402>`_).
+     * Bugfix: Refine fix for handling empty configuration files (`#2408 <https://github.com/dtcenter/MET/issues/2408>`_).
+     * Bugfix: Fix time interpolation of monthly climatology data between December 15 and January 15 (`#2412 <https://github.com/dtcenter/MET/issues/2412>`_).
+     * Bugfix: Fix ASCII2NC to handle missing NDBC buoy location information (`#2426 <https://github.com/dtcenter/MET/issues/2426>`_).
+     * Bugfix: Fix the MET vx_pointdata_python library to handle MET_PYTHON_EXE for python embedding of point observations (`#2428 <https://github.com/dtcenter/MET/issues/2428>`_).
+     * Bugfix: Refine the regrid dictionary's data conversion and censoring operations and fix climo time matching logic for a single monthly climo file (`#2437 <https://github.com/dtcenter/MET/issues/2437>`_).
+     * Bugfix: Fix the creation of the MET User's Guide PDF (`#2449 <https://github.com/dtcenter/MET/issues/2449>`_).
+     * Bugfix: Fix inconsistent ASCII2NC AIRNOW location lookup logic (`#2452 <https://github.com/dtcenter/MET/issues/2452>`_).
+
 MET Version 11.0.0 release notes (20221209)
 -------------------------------------------
 
@@ -26,8 +64,6 @@ MET Version 11.0.0 release notes (20221209)
      * Fix GHA documentation workflow (`#2282 <https://github.com/dtcenter/MET/issues/2282>`_).
      * Fix GHA warnings and update the version of actions (i.e. actions/checkout@v3) (`#2297 <https://github.com/dtcenter/MET/issues/2297>`_).
 
-
-
   .. dropdown:: Documentation
 
      * Create outline for the MET Contributor's Guide (`#1774 <https://github.com/dtcenter/MET/issues/1774>`_).
@@ -143,4 +179,14 @@ MET Version 11.0.0 release notes (20221209)
 MET Upgrade Instructions
 ========================
 
+MET Version 11.1.0 upgrade instructions
+---------------------------------------
+
+* If compiling support for PYTHON (:numref:`compiling_python_support`), in addition to $MET_PYTHON_CC and $MET_PYTHON_LD, set **$MET_PYTHON_BIN_EXE** to specify the desired python executable to be used (`#2428 <https://github.com/dtcenter/MET/issues/2428>`_).
+
+* If running TC-Pairs to generate consensus tracks, update your TC-Pairs configuration file to include the new **write_members** option (`#2429 <https://github.com/dtcenter/MET/issues/2429>`_).
+
+MET Version 11.0.0 upgrade instructions
+---------------------------------------
+
 * Ensemble post-processing has been fully removed from Ensemble-Stat in version 11.0.0. It can be performed using the Gen-Ens-Prod tool.
diff --git a/docs/conf.py b/docs/conf.py
@@ -19,21 +19,20 @@
 
 project = 'MET'
 author = 'UCAR/NCAR, NOAA, CSU/CIRA, and CU/CIRES'
-author_list = 'Opatz, J., T. Jensen, J. Prestopnik, H. Soh, L. Goodrich, B. Brown, R. Bullock, J. Halley Gotway, K. Newman'
-version = '11.1.0'
+author_list = 'Jensen, T., J. Prestopnik, H. Soh, L. Goodrich, B. Brown, R. Bullock, J. Halley Gotway, K. Newman, J. Opatz'
+version = '11.1.0-beta1'
 verinfo = version
 release = f'{version}'
-release_year = '2022'
-release_date = f'{release_year}-12-09'
+release_year = '2023'
+release_date = f'{release_year}-02-28'
 copyright = f'{release_year}, {author}'
 
 # -- General configuration ---------------------------------------------------
 
 # Add any Sphinx extension module names here, as strings. They can be
 # extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
 # ones.
-# Adding 'sphinx_panels' to use drop-down menus in appendixA. 
-extensions = ['sphinx.ext.autodoc','sphinx.ext.intersphinx','sphinx_panels','sphinx_design',]
+extensions = ['sphinx.ext.autodoc','sphinx.ext.intersphinx','sphinx_design',]
 
 # settings for ReadTheDocs PDF creation
 latex_engine = 'pdflatex'

diff --git a/docs/requirements.txt b/docs/requirements.txt
@@ -1,4 +1,3 @@
 sphinx-gallery
 sphinxcontrib-bibtex
-sphinx-panels
 sphinx-design
diff --git a/internal/scripts/docker/Dockerfile b/internal/scripts/docker/Dockerfile
@@ -1,5 +1,5 @@
 ARG MET_BASE_REPO=met-base
-ARG MET_BASE_TAG=v1.1
+ARG MET_BASE_TAG=v2.0_debian10
 
 FROM dtcenter/${MET_BASE_REPO}:${MET_BASE_TAG}
 MAINTAINER John Halley Gotway <johnhg@ucar.edu>

diff --git a/internal/scripts/docker/Dockerfile.copy b/internal/scripts/docker/Dockerfile.copy
@@ -1,5 +1,5 @@
 ARG MET_BASE_REPO=met-base-unit-test
-ARG MET_BASE_TAG=v1.1
+ARG MET_BASE_TAG=v2.0_debian10
 
 FROM dtcenter/${MET_BASE_REPO}:${MET_BASE_TAG}
 MAINTAINER John Halley Gotway <johnhg@ucar.edu>

diff --git a/internal/scripts/installation/config/install_met_env.acorn_py3.10 b/internal/scripts/installation/config/install_met_env.acorn_py3.10
@@ -12,16 +12,16 @@ module load g2c/1.6.4
 
 export TEST_BASE=/apps/sw_review/emc/MET/11.0.1
 export LIB_DIR=${TEST_BASE}/external_libs
-export BIN_DIR_PATH=${TEST_BASE}/exec
 export COMPILER=intel_19.1.3.304
 export MET_SUBDIR=${TEST_BASE}
 export MET_TARBALL=v11.0.1.tar.gz
 export USE_MODULES=TRUE
 export ADDTL_DIR=/apps/spack/gettext/0.21/intel/19.1.3.304/at2kdo4edvuhyzrt5g6zhwrdb7bdui4s/lib64
 export PYTHON_MODULE=python_3.10.4
 export MET_PYTHON=/apps/spack/python/3.10.4/intel/19.1.3.304/xqft4d45h4dp4xnbz2ue3nbxv65i6bgp
+export MET_PYTHON_LIB=/apps/spack/python/3.10.4/intel/19.1.3.304/xqft4d45h4dp4xnbz2ue3nbxv65i6bgp/lib64
 export MET_PYTHON_CC=-I/apps/spack/python/3.10.4/intel/19.1.3.304/xqft4d45h4dp4xnbz2ue3nbxv65i6bgp/include/python3.10
-export MET_PYTHON_LD=-L/apps/spack/python/3.10.4/intel/19.1.3.304/xqft4d45h4dp4xnbz2ue3nbxv65i6bgp/lib/python3.10/config-3.10-x86_64-linux-gnu/\ -L/apps/spack/python/3.10.4/intel/19.1.3.304/xqft4d45h4dp4xnbz2ue3nbxv65i6bgp/lib64\ -lpython3.10\ -lintl\ -lcrypt\ -ldl\ -lutil\ -lm\ -lm
+export MET_PYTHON_LD=-L/apps/spack/python/3.10.4/intel/19.1.3.304/xqft4d45h4dp4xnbz2ue3nbxv65i6bgp/lib64\ -lpython3.10\ -lintl\ -lcrypt\ -ldl\ -lutil\ -lm\ -lm
 export MET_NETCDF=/apps/prod/hpc-stack/intel-19.1.3.304/netcdf/4.7.4
 export MET_HDF5=/apps/prod/hpc-stack/intel-19.1.3.304/hdf5/1.12.2
 export MET_BUFRLIB=/apps/ops/prod/libs/intel/19.1.3.304/bufr/11.5.0/lib64

diff --git a/internal/scripts/installation/config/install_met_env.generic b/internal/scripts/installation/config/install_met_env.generic
@@ -28,7 +28,7 @@ export USE_MODULES=FALSE
 export PYTHON_LOC="$(python3-config --prefix)"
 
 #Directory of your python executable
-export MET_PYTHON=${PYTHON_LOC}/bin
+export MET_PYTHON=${PYTHON_LOC}
 
 #Python ldflags created using python3-config
 export MET_PYTHON_LD="$(python3-config --ldflags)"

diff --git a/internal/scripts/installation/config/install_met_env.wcoss2_py3.10 b/internal/scripts/installation/config/install_met_env.wcoss2_py3.10
@@ -25,8 +25,9 @@ export USE_MODULES=TRUE
 export ADDTL_DIR=/apps/spack/gettext/0.21/intel/19.1.3.304/at2kdo4edvuhyzrt5g6zhwrdb7bdui4s/lib64
 export PYTHON_MODULE=python_3.10.4
 export MET_PYTHON=/apps/spack/python/3.10.4/intel/19.1.3.304/xqft4d45h4dp4xnbz2ue3nbxv65i6bgp
+export MET_PYTHON_LIB=/apps/spack/python/3.10.4/intel/19.1.3.304/xqft4d45h4dp4xnbz2ue3nbxv65i6bgp/lib64
 export MET_PYTHON_CC=-I/apps/spack/python/3.10.4/intel/19.1.3.304/xqft4d45h4dp4xnbz2ue3nbxv65i6bgp/include/python3.10
-export MET_PYTHON_LD=-L/apps/spack/python/3.10.4/intel/19.1.3.304/xqft4d45h4dp4xnbz2ue3nbxv65i6bgp/lib\ -lpython3.10\ -lpthread\ -ldl\ -lutil\ -lm\ -Xlinker\ -export-dynamic
+export MET_PYTHON_LD=-L/apps/spack/python/3.10.4/intel/19.1.3.304/xqft4d45h4dp4xnbz2ue3nbxv65i6bgp/lib64\ -lpython3.10\ -lcrypt\ -lintl\ -ldl\ -lutil\ -lm\ -lm
 export MET_NETCDF=/apps/prod/hpc-stack/intel-19.1.3.304/netcdf/4.7.4
 # JY export MET_HDF5=/apps/prod/hpc-stack/intel-19.1.3.304/hdf5/1.12.2
 export MET_HDF5=${HDF5_ROOT}

diff --git a/internal/scripts/sonarqube/python.sonar-project.properties b/internal/scripts/sonarqube/python.sonar-project.properties
@@ -1,4 +1,4 @@
-sonar.projectKey=org.sonarqube:MET_python_NB
+sonar.projectKey=MET_python_NB
 sonar.projectName=MET python Nightly Build
 sonar.projectVersion=1.0
 
@@ -13,5 +13,5 @@ sonar.sourceEncoding=UTF-8
 #sonar.host.url=http://localhost:9000
 sonar.host.url=http://mandan:9000
 
-sonar.login=met
-sonar.password=met@sonar.ucar
+sonar.login=squ_e965ad7d3c3abed1326573f25262212dc969798b
+sonar.branch.name=develop
diff --git a/internal/scripts/sonarqube/sonar-project.properties b/internal/scripts/sonarqube/sonar-project.properties
@@ -1,4 +1,4 @@
-sonar.projectKey=org.sonarqube:MET_develop_NB
+sonar.projectKey=MET_develop_NB
 sonar.projectName=MET Nightly Build
 sonar.projectVersion=1.0
 
@@ -14,6 +14,5 @@ sonar.sourceEncoding=UTF-8
 #sonar.host.url=http://localhost:9000
 sonar.host.url=http://mandan:9000
 
-sonar.login=met
-sonar.password=met@sonar.ucar
+sonar.login=squ_e965ad7d3c3abed1326573f25262212dc969798b
 sonar.branch.name=develop
diff --git a/scripts/utility/Makefile.am b/scripts/utility/Makefile.am
@@ -26,7 +26,8 @@
 pythonutilitydir = $(pkgdatadir)/utility
 
 pythonutility_DATA = \
-	print_pointnc2ascii.py
+	print_pointnc2ascii.py \
+	build_ndbc_stations_from_web.py
 
 EXTRA_DIST = ${pythonutility_DATA}
 

diff --git a/scripts/utility/Makefile.in b/scripts/utility/Makefile.in
@@ -298,7 +298,8 @@ top_builddir = @top_builddir@
 top_srcdir = @top_srcdir@
 pythonutilitydir = $(pkgdatadir)/utility
 pythonutility_DATA = \
-	print_pointnc2ascii.py
+	print_pointnc2ascii.py \
+	build_ndbc_stations_from_web.py
 
 EXTRA_DIST = ${pythonutility_DATA}
 MAINTAINERCLEANFILES = Makefile.in