Including codes about reading, interpolating, masking NetCDF data of GCMs and calculating metrics to evaluate models' performance on droughts.
Users must have a conda environment named "_CMEC_drought_metrics" to run this module. A conda package manager such as Miniconda or Anaconda can be used to accomplish this. The environment requirements and two strategies for creating this environment are provided below.
Packages: Python 3.6 is recommended. The required packages are affine, basemap, cartopy, esmpy, geopandas, matplotlib, netcdf4, numpy, pandas, rasterio, scikit-learn, scipy, shapely, xarray, xesmf, and climate-indices.
Create from yml: A conda environment can be created using drought_metrics.yml on the command line. The resulting environment may not work on all systems:
conda env create -f drought_metrics.yml
Create via command line: It is recommended that conda and the conda-forge channel are used to install all packages except climate-indices, geopandas, rasterio, and scikit-learn. Use pip to install these four packages afterwards. For example:
conda create -n _CMEC_drought_metrics --no-default-packages python=3.6
conda activate _CMEC_drought_metrics
conda install -c conda-forge affine basemap cartopy esmpy matplotlib netcdf4 numpy pandas pip scipy shapely xarray xesmf
pip install climate-indices rasterio geopandas scikit-learn
Use git to clone this repository. CMEC driver will run the module from this codebase.
Follow the instructions here to install cmec-driver: https://github.com/cmecmetrics/cmec-driver.
CMEC driver version must be "20210617" or later. Please pull the latest version if you have an existing cmec-driver repo.
Set-up steps in the cmec-driver directory:
- Activate any Python 3 environment.
- Create the following directories: "obs", "model", and "output".
- Copy (or link) your model files into model/.
- Copy (or link) your observational file into obs/.
- (Only for first-time cmec-driver users) Register your conda environment information. Your conda executable is the file that gets sourced to activate conda (usually conda.sh) and your environment root is the folder containing your '_CMEC_drought_metrics' environment:
python cmec-driver.py setup -conda_source <path_to_conda_executable> -env_root <path_to_environment_root> - Register the module:
python cmec-driver.py register <path to Drought Metrics repository>
Running Drought Metrics:
- (Optional) Edit settings in config/cmec.json under "Drought_Metrics"
- Run the module:
python cmec-driver.py run -obs obs/ model/ output/ Drought_Metrics
For testing, users may use the model files from Drought_Metrics/data/test. The default observations can be downloaded from the source described in the Observations section below.
Drought Metrics results will be written to cmec-driver/output/Drought_Metrics. Open cmec-driver/output/Drought_Metrics/index.html with your browser to view the generated metrics files and plots.
Results will be overwritten the next time cmec-driver is run with the same output path. To save results, either copy the output/Drought_Metrics folder to a new location or provide a unique output folder when running cmec-driver.
The user settings can be modified in cmec-driver/config/cmec.json after the package is registered.
The directory names "obs", "model", and "output" are recommended with cmec-driver but not required. The observations file should be located in the directory passed to cmec-driver via the "obs" flag, and the model files should be located in the directory passed to cmec-driver as a required positional argument after "run".
The following files and settings can be changed by the user in cmec.json:
- obs_file_name: Observation file name (default: 'precip.V1.0.mon.mean.nc')
- hu_name: Name of evaluation region (default: 'New England Region')
- shp_path: Watershed boundary file path (default: 'HU/WBDHU2.shp')
- wgt_path: netcdf file with grid for interpolation (default 'data/weightfile/interpolated_pr_Amon_E3SM-1-1_historical_r1i1p1f1_gr_187001-200912.nc')
- interpolation: true to interpolate inputs to same grid (default true)
- pfa: Path to existing principal metrics results (default 'output_principal_metrics_column_defined')
- run_pfa: Set to true to generate new PFA analysis (default false)
This analysis was designed to use the CPC Unified Gauge-Based Analysis of Daily Precipitation over CONUS. This dataset is available from NOAA PSL (download the monthly mean file "precip.V1.0.mon.mean.nc"). For best results, at least 50 years of data should be used.
Monthly precipitation output should be in CF-compliant netCDF files that conform to the standards for the CMIP6 project. Required dimensions are latitude, longitude, time, and precipitation flux. The Drought Metrics package assumes that there will be a single precipitation variable in each file and that its variable name contains the letters "pr".
The user has the option to reuse the results of a previous Principal Features Analysis or to generate new principal metrics.
To use the results of an old PFA analysis, set the optional pfa key to the absolute path for those PFA results.
By default, Drought Metrics uses the principal metrics file 'output_principal_metrics_column_defined' found in the Drought Metrics code folder.
This analysis requires a shapefile containing watershed boundaries. The boundary features must contain the fields "Name" and "geometry".
The example boundaries provided in Drought_Metrics/HU are for watersheds in the US at the 2-digit level, based on data obtained from the U.S. Geological Survey and the U.S. Department of Agriculture, Natural Resources Conservation Service. More information here.
evaluation.py: Evaluation class which computes drought metrics.
drought_metrics.py: Driver for evaluation.py.
dm_cmec_outputs.py: Functions for creating CMEC outputs.
cmec_drought_metrics.sh: CMEC driver script.
settings.json: CMEC settings file.
drought_metrics.yml: Environment file.
Tempelates of observational data, model data, weightfile used in interpolation, shapefile defining the evaluation regions and plots are provided. More information can be found in settings.json.
The folder 'example_output' contains the results from running the Drought Metrics code via cmec-driver while reusing existing principal metrics.
This work was performed under the auspices of the U.S. Department of Energy by Lawrence Livermore National Laboratory under Contract DE-AC52-07NA27344. Lawrence Livermore National Laboratory is operated by Lawrence Livermore National Security, LLC, for the U.S. Department of Energy, National Nuclear Security Administration under Contract DE-AC52-07NA27344.
This document was prepared as an account of work sponsored by an agency of the United States government. Neither the United States government nor Lawrence Livermore National Security, LLC, nor any of their employees makes any warranty, expressed or implied, or assumes any legal liability or responsibility for the accuracy, completeness, or usefulness of any information, apparatus, product, or process disclosed, or represents that its use would not infringe privately owned rights. Reference herein to any specific commercial product, process, or service by trade name, trademark, manufacturer, or otherwise does not necessarily constitute or imply its endorsement, recommendation, or favoring by the United States government or Lawrence Livermore National Security, LLC. The views and opinions of authors expressed herein do not necessarily state or reflect those of the United States government or Lawrence Livermore National Security, LLC, and shall not be used for advertising or product endorsement purposes.