Skip to content

Latest commit

 

History

History
146 lines (92 loc) · 5.14 KB

README_pyCECT.rst

File metadata and controls

146 lines (92 loc) · 5.14 KB
Raw

README.pyCECT

CESM Ensemble Consistency Test

The CESM-ECT package is used to compare the results of a set of new (modified) CESM simulations against the accepted ensemble. Current functionality includes CAM-ECT, which examines history files from CAM (atmosphere) or POP-ECT which examines files from POP (ocean). CAM-ECT requires a summary file generated by pyEnsSum.py (see README.pyEnsSum), and POP-ECT requires a summary file generated by pyEnsSumPop.py (see README.pyEnsSumPop). An overall pass or fail is designated.

For details on CAM-ECT, see:

A.H. Baker, D.M. Hammerling, M.N. Levy, H. Xu, J.M. Dennis, B.E. Eaton, J. Edwards, C. Hannay, S.A. Mickelson, R.B. Neale, D. Nychka, J. Shollenberger, J. Tribbia, M. Vertenstein, and D. Williamson,"A new ensemble-based consistency test for the community earth system model."Geoscientific Model Development, 8, pp. 2829-2840, doi:10.5194/gmd-8-2829-2015, 2015.

For details on POP-ECT, see:

A.H.Baker, Y.Hu, D.M. Hammerling, H. Xu, Y.Tseng, X. Huang, and F.O. Byan "Evaluating Consistency in the Ocean Model Component of the Community Earth System Model", submitted to Geoscientific Model Development Discussions, Dec. 2015

AUTHORS:Haiying Xu, Allison Baker
VERSION:2.0.0
COPYRIGHT:See the document entitled LICENSE.txt

Send questions and comments to Haiying Xu (haiyingx@ucar.edu).

Overview

This package includes:

pyCECT.py
A script which compares the new CAM or POP results to the accepted ensemble and issues an overall "pass" or "fail" (runs CAM-ECT by default or POP-ECT if specified)

pyEnsLib.py Library python script used by pyCECT.py

Before you start to use the package, you need to load the following modules:

  • module load python
  • module load numpy
  • module load scipy
  • module load pynio

To see all options (and defaults):

python pyCECT.py -h

Notes and examples:

  1. Options for all CESM-ECT approaches:

    Required:

    To specify the summary file generated by pyEnsSum.py:
    -sumfile

    ens.summary.nc
    To specifying the directory path that contains the run(s) to be evaluated:
    -indir

    /glade/u/tdd/asap/verification/cesm1_3_beta11/mira
    Optional:
    Verbose information:

    --verbose

  2. CAM-ECT specific options (and summary file generated by pyEnsSum.py)

    Note that CAM-ECT is the default test.

    Note that the parameters setting the pass/fail criteria are all set by default (ie. sigMul, minPCFail, minRunFail, numRunFile, and nPC). But if the specified indir contains more files than the number (num) specified by "--numRunFile <num>" (default= 3), then <num> files will be chosen at random from that directory.

    To enable printing of extra variable information:

    --printVarTest

    By default, CAM-ECT looks at annual averages which is indictated by

    --timeslice 1 (For monthly files, --timeslice 0. Note that this should correspond to what has been collected in the summary file.)

    Example using the default settings:

    python pyCECT.py --sumfile CAM.ens.summary.nc --indir /glade/u/tdd/asap/verification/cesm1_3_beta11/mira

  1. POP-ECT specific options (and summary file generated by pyEnsSumPop.py)

    To use POP-ECT, you MUST add the following to enable this test (which disables CAM-ECT):

    --popens

    Be sure to use a POP-ECT summary file:

    --sumfile /glade/u/tdd/asap/pop_verification/ens_sum/pop.40ens.openseas.nc

    Directory path that contains the run(s) to be evaluated.

    --indir /glade/u/tdd/asap/pop_verification/testcases/

    The above directory may contain many POP history files that following the standard CESM-POP naming convention. To specific with file or files you wish to test, you simply specifying the test case file prefix. For example, to compare against all months in year 3 from the input directory above:

    --input_glob cvtd100000.0.pop.h.0003

    To compare only against month 6 in year 3:

    --input_glob cvtd100000.0.pop.h.0003-06

    Be sure to specify the json file that includes the variables which will be run the test on:

    --jsonfile pop_ensemble.json

Note that the parameters setting the pass/fail criteria are all set by default (ie. pop_tol, pop_threshold) but may be modified:

Specifying test tolerance (the minimum Z-score threshold):

--pop_tol 3.0

Specifying pop threshold (fraction of points that must satisfy the Z-score tolerance):

--pop_threshold 0.9

To run in parallel (recommended if you are analyzing more than one month - one core per month):

--mpi_enable

Example:

python pyCECT.py --sumfile /glade/u/tdd/asap/pop_verification/ens_sum/pop.40ens.openseas.nc --indir /glade/u/tdd/asap/pop_verification/testcases/ --input_glob cvtd100000.0.pop.h.0003 --popens --jsonfile pop_ensemble.json --pop_tol 3.0 --pop_threshold 0.9