Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature 627 coherence spectra #891

Merged
merged 52 commits into from
May 3, 2021
Merged
Show file tree
Hide file tree
Changes from 5 commits
Commits
Show all changes
52 commits
Select commit Hold shift + click to select a range
76e2363
Initial checkin for feature 626.
hankenstein2 Apr 19, 2021
479d2a7
Added environment variables and removed hard coded stuff
hankenstein2 Apr 20, 2021
020262a
Added comments and reading in log level, cleaned up conf file
Apr 21, 2021
0926ad5
added new use case to list of cases to run for tests
georgemccabe Apr 21, 2021
6436c4f
added option to log failure to read time info as a warning since it i…
georgemccabe Apr 21, 2021
6bca3f7
Changed file to full path
Apr 21, 2021
e66ed08
also warn in get_time_obj if specified in calling function
georgemccabe Apr 21, 2021
0bc77c3
removed unused env var METplus_DATA_DIR and update path to METPLOTPY_…
georgemccabe Apr 21, 2021
4788b3d
Merge branch 'develop' into feature_626_hovmoeller_diagram
georgemccabe Apr 21, 2021
a5478c3
Changed hovmeoller.conf to hovmoeller.py
hankenstein2 Apr 21, 2021
07f3159
Initial checkin for code
hankenstein2 Apr 21, 2021
2a53dfa
specify cartopy version 0.18.0 to prevent failure
georgemccabe Apr 22, 2021
f90d5ab
Added a stub for documentation
hankenstein2 Apr 22, 2021
c4634a0
Changed filenames in conf and file name handling in python
Apr 22, 2021
27a8922
Changed the name to fit conventions
Apr 22, 2021
6615e9f
Changed names and added a thumbnail
Apr 22, 2021
1ac76a5
Cleaned up cut and paste issues with Hovmoeller documentation
Apr 23, 2021
a14f9c7
Updated documentation with image filename, added thumbnail image, rem…
Apr 26, 2021
8b0ab79
fixed issues with automated testing
Apr 26, 2021
14cb854
Merge remote-tracking branch 'origin/develop' into feature_626_hovmoe…
Apr 26, 2021
5f237ca
removed extra slash
georgemccabe Apr 27, 2021
660aac5
changed use case name to match name of conf file
georgemccabe Apr 27, 2021
7d94239
obtain METcalcpy and METplotpy code one level above METplus repo in t…
georgemccabe Apr 27, 2021
da828c1
add missing python package psutil that caused no images to be produced
georgemccabe Apr 27, 2021
c1541e0
added requests package that is needed to generate images
georgemccabe Apr 27, 2021
5d8806b
Merge branch 'develop' into feature_627_coherence_spectra
georgemccabe Apr 28, 2021
a5afbb8
removed duplicate files
georgemccabe Apr 28, 2021
8df2a12
added new use case to list of cases to run for tests
georgemccabe Apr 28, 2021
86dd7f3
fixed warning in doc
georgemccabe Apr 28, 2021
94c2663
fixed incorrect file path in include and added user python script to …
georgemccabe Apr 28, 2021
9b1422f
Merge branch 'develop' into feature_627_coherence_spectra
georgemccabe Apr 28, 2021
f3811c3
added plotpy==4.9.0 requirement for METplotpy
georgemccabe Apr 28, 2021
938946a
Added kaleido package for all of metplotpy scripts
hankenstein2 Apr 30, 2021
a0dd7ca
clone METplotpy/calcpy in level above METplus
georgemccabe Apr 30, 2021
d25ad0c
add argument to miniconda script to specify python version
georgemccabe Apr 30, 2021
2b49a3a
added spacetime environment setup script and added it to use case
georgemccabe Apr 30, 2021
3040507
Merge branch 'feature_627_coherence_spectra' of https://github.com/dt…
georgemccabe Apr 30, 2021
8e1acda
Reading custom yaml with environment variables and explicitly
Apr 30, 2021
10c6de3
Merge branch 'feature_626_hovmoeller_diagram' of https://github.com/d…
Apr 30, 2021
74ba59f
Added METcalcpy to the Hovmoeller requirements
Apr 30, 2021
9d57a26
Change OUTPUT_BASE to OUPUT_DIR and added log message
Apr 30, 2021
d56c568
Added a yaml config file to get output directory
Apr 30, 2021
95f9d70
added pyyaml package
georgemccabe Apr 30, 2021
06e339e
removed unused variable
georgemccabe May 3, 2021
30d0f03
create output directory if it does not exist to prevent failure
georgemccabe May 3, 2021
8e65538
skip create output dir to test that error is reported if the output f…
georgemccabe May 3, 2021
d62a77d
added back env var that is used by METcalcpy
georgemccabe May 3, 2021
82d9000
create output directory so file can be created
georgemccabe May 3, 2021
96069fc
merged 626 and resolved conflicts
georgemccabe May 3, 2021
09e4591
create the output directory if it doesn't exist and check if file is …
georgemccabe May 3, 2021
1c5651c
changed log messages to print statements
georgemccabe May 3, 2021
75af9a5
merged develop and resolved conflicts
georgemccabe May 3, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
133 changes: 133 additions & 0 deletions docs/use_cases/model_applications/s2s/SpaceTimeCrossSpectra.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,133 @@
"""
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks like this filename should also change to match the use case name.

UserScript: Make a Hovmoeller plot
========================================================================

s2s/SpaceTimeCrossSpectra
cross-spectra.py

"""

#################################################################################
# Scientific Objective
# --------------------
#
# This use case calls the METplotpy space time plot to create a sample cross
# spectra diagram
# using sample data created by METcalcpy hovmoeller functions
#
# The space time plot and cross spectra calculations were created by Maria Gehne
# at the
# Physical Sciences Labratory in NOAA

#################################################################################
# Datasets
# --------
#

##############################################################################
# METplus Components
# ------------------
#
# This use case runs the UserScript wrapper tool to run a user provided script,
# in this case, cross-spectra.py.
#

##############################################################################
# METplus Workflow
# ----------------
#
# This use case does not loop but plots the entire time period of data
#
# UserScript
#
#

##############################################################################
# METplus Configuration
# ---------------------
#
# METplus first loads all of the configuration files found in parm/metplus_config,
# then it loads any configuration files passed to METplus via the command line
# with the -c option, i.e. -c parm/use_cases/model_applications/s2s/hovmoeller_diagram.conf
#
# .. highlight:: bash
# .. literalinclude:: ../../../../parm/use_cases/model_applications/s2s/SpaceTimeCrossSpectra/cross-spectra_diagram.conf
#

#############################################################################
# MET Configuration
# ---------------------
#
# There are no MET tools used in this use case.
#

##############################################################################
# Python Embedding
# ----------------
#
# There is no python embedding in this use case
#

##############################################################################
# Running METplus
# ---------------
#
# This use case can be run two ways:
#
# 1) Passing in hovmoeller_diagram.conf,
# then a user-specific system configuration file::
#
# run_metplus.py \
# -c /path/to/METplus/parm/use_cases/model_applications/s2s/hovmoeller_diagram.conf \
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this be hovmoeller conf file or the UserScript_obsPrecip_obsOnly_CrossSpectraPlot.conf?

# -c /path/to/user_system.conf
#
# 2) Modifying the configurations in parm/metplus_config, then passing in hovmoeller_diagram.conf::
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same here, should this be hovmoeller?

#
# run_metplus.py \
# -c /path/to/METplus/parm/use_cases/model_applications/s2s/SpaceTimeSpectra.conf
#
# The former method is recommended. Whether you add them to a user-specific configuration file or modify the metplus_config files, the following variables must be set correctly:
#
# * **INPUT_BASE** - Path to directory where sample data tarballs are unpacked (See Datasets section to obtain tarballs). This is not required to run METplus, but it is required to run the examples in parm/use_cases
# * **OUTPUT_BASE** - Path where METplus output will be written. This must be in a location where you have write permissions
# * **MET_INSTALL_DIR** - Path to location where MET is installed locally
#
# and for the [exe] section, you will need to define the location of NON-MET executables.
# If the executable is in the user's path, METplus will find it from the name.
# If the executable is not in the path, specify the full path to the executable here (i.e. RM = /bin/rm)
# The following executables are required for performing series analysis use cases:
#
# Example User Configuration File::
#
# [dir]
# INPUT_BASE = /path/to/sample/input/data
# OUTPUT_BASE = /path/to/output/dir
# MET_INSTALL_DIR = /path/to/met-X.Y
#
# [exe]
# RM = /path/to/rm
# CUT = /path/to/cut
# TR = /path/to/tr
# NCAP2 = /path/to/ncap2
# CONVERT = /path/to/convert
# NCDUMP = /path/to/ncdump
#

##############################################################################
# Expected Output
# ---------------
#
# A successful run will output the following both to the screen and to the logfile::
#
# INFO: METplus has successfully finished running.
#

##############################################################################
# Keywords
# --------
#
# .. note::
# `UserScriptUseCase <https://dtcenter.github.io/METplus/develop/search.html?q=UserScriptUseCase&check_keywords=yes&area=default>`_,
# `S2SAppUseCase <https://dtcenter.github.io/METplus/search.html?q=S2SAppUseCase&check_keywords=yes&area=default>`_,
#
# sphinx_gallery_thumbnail_path = '_static/s2s-UserScript_Space_Time_Spectra.png'
Original file line number Diff line number Diff line change
@@ -0,0 +1,134 @@
"""
UserScript: Make a Cross Spectra plot
========================================================================

model_applications/
s2s/
UserScript_obsPrecip_obsOnly_CrossSpectraPlot.py

"""

#################################################################################
# Scientific Objective
# --------------------
#
# This use case calls the METplotpy space time plot to create a sample cross
# spectra diagram
# using sample data created by METcalcpy cross spectra functions
#
# The space time plot and cross spectra calculations were created by Maria Gehne
# at the
# Physical Sciences Labratory in NOAA

#################################################################################
# Datasets
# --------
#

##############################################################################
# METplus Components
# ------------------
#
# This use case runs the UserScript wrapper tool to run a user provided script,
# in this case, cross-spectra.py.
#

##############################################################################
# METplus Workflow
# ----------------
#
# This use case does not loop but plots the entire time period of data
#
# UserScript
#
#

##############################################################################
# METplus Configuration
# ---------------------
#
# METplus first loads all of the configuration files found in parm/metplus_config,
# then it loads any configuration files passed to METplus via the command line
# with the -c option, i.e. -c parm/use_cases/model_applications/s2s/UserScript_obsPrecip_obsOnly_CrossSpectraPlot.conf
#
# .. highlight:: bash
# .. literalinclude:: ../../../../parm/use_cases/model_applications/s2s/UserScript_obsPrecip_obsOnly_CrossSpectraPlot/cross-spectra_diagram.conf
#

#############################################################################
# MET Configuration
# ---------------------
#
# There are no MET tools used in this use case.
#

##############################################################################
# Python Embedding
# ----------------
#
# There is no python embedding in this use case
#

##############################################################################
# Running METplus
# ---------------
#
# This use case can be run two ways:
#
# 1) Passing in UserScript_obsPrecip_obsOnly_CrossSpectraPlot.conf,
# then a user-specific system configuration file::
#
# run_metplus.py \
# -c /path/to/METplus/parm/use_cases/model_applications/s2s/UserScript_obsPrecip_obsOnly_CrossSpectraPlot.conf \
# -c /path/to/user_system.conf
#
# 2) Modifying the configurations in parm/metplus_config, then passing in UserScript_obsPrecip_obsOnly_CrossSpectraPlot.conf::
#
# run_metplus.py \
# -c /path/to/METplus/parm/use_cases/model_applications/s2s/UserScript_obsPrecip_obsOnly_CrossSpectraPlot.conf
#
# The former method is recommended. Whether you add them to a user-specific configuration file or modify the metplus_config files, the following variables must be set correctly:
#
# * **INPUT_BASE** - Path to directory where sample data tarballs are unpacked (See Datasets section to obtain tarballs). This is not required to run METplus, but it is required to run the examples in parm/use_cases
# * **OUTPUT_BASE** - Path where METplus output will be written. This must be in a location where you have write permissions
# * **MET_INSTALL_DIR** - Path to location where MET is installed locally
#
# and for the [exe] section, you will need to define the location of NON-MET executables.
# If the executable is in the user's path, METplus will find it from the name.
# If the executable is not in the path, specify the full path to the executable here (i.e. RM = /bin/rm)
# The following executables are required for performing series analysis use cases:
#
# Example User Configuration File::
#
# [dir]
# INPUT_BASE = /path/to/sample/input/data
# OUTPUT_BASE = /path/to/output/dir
# MET_INSTALL_DIR = /path/to/met-X.Y
#
# [exe]
# RM = /path/to/rm
# CUT = /path/to/cut
# TR = /path/to/tr
# NCAP2 = /path/to/ncap2
# CONVERT = /path/to/convert
# NCDUMP = /path/to/ncdump
#

##############################################################################
# Expected Output
# ---------------
#
# A successful run will output the following both to the screen and to the logfile::
#
# INFO: METplus has successfully finished running.
#

##############################################################################
# Keywords
# --------
#
# .. note::
# `UserScriptUseCase <https://dtcenter.github.io/METplus/develop/search.html?q=UserScriptUseCase&check_keywords=yes&area=default>`_,
# `S2SAppUseCase <https://dtcenter.github.io/METplus/search.html?q=S2SAppUseCase&check_keywords=yes&area=default>`_,
#
# sphinx_gallery_thumbnail_path = '_static/UserScript_obsPrecip_obsOnly_CrossSpectraPlot.png'
56 changes: 56 additions & 0 deletions parm/use_cases/model_applications/s2s/SpaceTimeSpectraDiagram.conf
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
[config]

# time looping - options are INIT, VALID, RETRO, and REALTIME
# If set to INIT or RETRO:
# INIT_TIME_FMT, INIT_BEG, INIT_END, and INIT_INCREMENT must also be set
# If set to VALID or REALTIME:
# VALID_TIME_FMT, VALID_BEG, VALID_END, and VALID_INCREMENT must also be set
LOOP_BY = REALTIME

# %Y = 4 digit year, %m = 2 digit month, %d = 2 digit day, etc.
# see www.strftime.org for more information
# %Y%m%d%H expands to YYYYMMDDHH
VALID_TIME_FMT = %Y%m%d%H

# BLank for this usecase but the parameter still needs to be there
VALID_BEG =

# BLank for this usecase but the parameter still needs to be there
VALID_END =

# BLank for this usecase but the parameter still needs to be there
VALID_INCREMENT =

# List of forecast leads to process for each run time (init or valid)
# In hours if units are not specified
# If unset, defaults to 0 (don't loop through forecast leads)
LEAD_SEQ =

# Order of loops to process data - Options are times, processes
# Not relevant if only one item is in the PROCESS_LIST
# times = run all wrappers in the PROCESS_LIST for a single run time, then
# increment the run time and run all wrappers again until all times have
# been evaluated.
# processes = run the first wrapper in the PROCESS_LIST for all times
# specified, then repeat for the next item in the PROCESS_LIST until all
# wrappers have been run
LOOP_ORDER = processes

PROCESS_LIST = UserScript

USER_SCRIPT_RUNTIME_FREQ = RUN_ONCE

USER_SCRIPT_COMMAND = {PARM_BASE}/use_cases/model_applications/s2s/SpaceTimeSpectraDiagram/cross_spectra_plot.py


[user_env_vars]

# Difficulty index specific variables

LOG_FILE = "cross_spectra_plot.log"

LOG_LEVEL = "INFO"

INPUT_FILE_NAMES = {INPUT_BASE}/model_applications/s2s/SpaceTimeSpectraDiagram/SpaceTimeSpectra_ERAI_P_D200_symm_2spd.nc,{INPUT_BASE}/model_applications/s2s/SpaceTimeSpectraDiagram/SpaceTimeSpectra_ERAI_TRMM_P_symm_2spd.nc,{INPUT_BASE}/model_applications/s2s/SpaceTimeSpectraDiagram/SpaceTimeSpectra_ERAI_P_D850_symm_2spd.nc

METPLOTPY_BASE = {METPLUS_BASE}/METplotpy/metplotpy/
Loading