-
Notifications
You must be signed in to change notification settings - Fork 38
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Feature 627 coherence spectra #891
Merged
Merged
Changes from 5 commits
Commits
Show all changes
52 commits
Select commit
Hold shift + click to select a range
76e2363
Initial checkin for feature 626.
hankenstein2 479d2a7
Added environment variables and removed hard coded stuff
hankenstein2 020262a
Added comments and reading in log level, cleaned up conf file
0926ad5
added new use case to list of cases to run for tests
georgemccabe 6436c4f
added option to log failure to read time info as a warning since it i…
georgemccabe 6bca3f7
Changed file to full path
e66ed08
also warn in get_time_obj if specified in calling function
georgemccabe 0bc77c3
removed unused env var METplus_DATA_DIR and update path to METPLOTPY_…
georgemccabe 4788b3d
Merge branch 'develop' into feature_626_hovmoeller_diagram
georgemccabe a5478c3
Changed hovmeoller.conf to hovmoeller.py
hankenstein2 07f3159
Initial checkin for code
hankenstein2 2a53dfa
specify cartopy version 0.18.0 to prevent failure
georgemccabe f90d5ab
Added a stub for documentation
hankenstein2 c4634a0
Changed filenames in conf and file name handling in python
27a8922
Changed the name to fit conventions
6615e9f
Changed names and added a thumbnail
1ac76a5
Cleaned up cut and paste issues with Hovmoeller documentation
a14f9c7
Updated documentation with image filename, added thumbnail image, rem…
8b0ab79
fixed issues with automated testing
14cb854
Merge remote-tracking branch 'origin/develop' into feature_626_hovmoe…
5f237ca
removed extra slash
georgemccabe 660aac5
changed use case name to match name of conf file
georgemccabe 7d94239
obtain METcalcpy and METplotpy code one level above METplus repo in t…
georgemccabe da828c1
add missing python package psutil that caused no images to be produced
georgemccabe c1541e0
added requests package that is needed to generate images
georgemccabe 5d8806b
Merge branch 'develop' into feature_627_coherence_spectra
georgemccabe a5afbb8
removed duplicate files
georgemccabe 8df2a12
added new use case to list of cases to run for tests
georgemccabe 86dd7f3
fixed warning in doc
georgemccabe 94c2663
fixed incorrect file path in include and added user python script to …
georgemccabe 9b1422f
Merge branch 'develop' into feature_627_coherence_spectra
georgemccabe f3811c3
added plotpy==4.9.0 requirement for METplotpy
georgemccabe 938946a
Added kaleido package for all of metplotpy scripts
hankenstein2 a0dd7ca
clone METplotpy/calcpy in level above METplus
georgemccabe d25ad0c
add argument to miniconda script to specify python version
georgemccabe 2b49a3a
added spacetime environment setup script and added it to use case
georgemccabe 3040507
Merge branch 'feature_627_coherence_spectra' of https://github.com/dt…
georgemccabe 8e1acda
Reading custom yaml with environment variables and explicitly
10c6de3
Merge branch 'feature_626_hovmoeller_diagram' of https://github.com/d…
74ba59f
Added METcalcpy to the Hovmoeller requirements
9d57a26
Change OUTPUT_BASE to OUPUT_DIR and added log message
d56c568
Added a yaml config file to get output directory
95f9d70
added pyyaml package
georgemccabe 06e339e
removed unused variable
georgemccabe 30d0f03
create output directory if it does not exist to prevent failure
georgemccabe 8e65538
skip create output dir to test that error is reported if the output f…
georgemccabe d62a77d
added back env var that is used by METcalcpy
georgemccabe 82d9000
create output directory so file can be created
georgemccabe 96069fc
merged 626 and resolved conflicts
georgemccabe 09e4591
create the output directory if it doesn't exist and check if file is …
georgemccabe 1c5651c
changed log messages to print statements
georgemccabe 75af9a5
merged develop and resolved conflicts
georgemccabe File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
133 changes: 133 additions & 0 deletions
133
docs/use_cases/model_applications/s2s/SpaceTimeCrossSpectra.py
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,133 @@ | ||
""" | ||
UserScript: Make a Hovmoeller plot | ||
======================================================================== | ||
|
||
s2s/SpaceTimeCrossSpectra | ||
cross-spectra.py | ||
|
||
""" | ||
|
||
################################################################################# | ||
# Scientific Objective | ||
# -------------------- | ||
# | ||
# This use case calls the METplotpy space time plot to create a sample cross | ||
# spectra diagram | ||
# using sample data created by METcalcpy hovmoeller functions | ||
# | ||
# The space time plot and cross spectra calculations were created by Maria Gehne | ||
# at the | ||
# Physical Sciences Labratory in NOAA | ||
|
||
################################################################################# | ||
# Datasets | ||
# -------- | ||
# | ||
|
||
############################################################################## | ||
# METplus Components | ||
# ------------------ | ||
# | ||
# This use case runs the UserScript wrapper tool to run a user provided script, | ||
# in this case, cross-spectra.py. | ||
# | ||
|
||
############################################################################## | ||
# METplus Workflow | ||
# ---------------- | ||
# | ||
# This use case does not loop but plots the entire time period of data | ||
# | ||
# UserScript | ||
# | ||
# | ||
|
||
############################################################################## | ||
# METplus Configuration | ||
# --------------------- | ||
# | ||
# METplus first loads all of the configuration files found in parm/metplus_config, | ||
# then it loads any configuration files passed to METplus via the command line | ||
# with the -c option, i.e. -c parm/use_cases/model_applications/s2s/hovmoeller_diagram.conf | ||
# | ||
# .. highlight:: bash | ||
# .. literalinclude:: ../../../../parm/use_cases/model_applications/s2s/SpaceTimeCrossSpectra/cross-spectra_diagram.conf | ||
# | ||
|
||
############################################################################# | ||
# MET Configuration | ||
# --------------------- | ||
# | ||
# There are no MET tools used in this use case. | ||
# | ||
|
||
############################################################################## | ||
# Python Embedding | ||
# ---------------- | ||
# | ||
# There is no python embedding in this use case | ||
# | ||
|
||
############################################################################## | ||
# Running METplus | ||
# --------------- | ||
# | ||
# This use case can be run two ways: | ||
# | ||
# 1) Passing in hovmoeller_diagram.conf, | ||
# then a user-specific system configuration file:: | ||
# | ||
# run_metplus.py \ | ||
# -c /path/to/METplus/parm/use_cases/model_applications/s2s/hovmoeller_diagram.conf \ | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Should this be hovmoeller conf file or the UserScript_obsPrecip_obsOnly_CrossSpectraPlot.conf? |
||
# -c /path/to/user_system.conf | ||
# | ||
# 2) Modifying the configurations in parm/metplus_config, then passing in hovmoeller_diagram.conf:: | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. same here, should this be hovmoeller? |
||
# | ||
# run_metplus.py \ | ||
# -c /path/to/METplus/parm/use_cases/model_applications/s2s/SpaceTimeSpectra.conf | ||
# | ||
# The former method is recommended. Whether you add them to a user-specific configuration file or modify the metplus_config files, the following variables must be set correctly: | ||
# | ||
# * **INPUT_BASE** - Path to directory where sample data tarballs are unpacked (See Datasets section to obtain tarballs). This is not required to run METplus, but it is required to run the examples in parm/use_cases | ||
# * **OUTPUT_BASE** - Path where METplus output will be written. This must be in a location where you have write permissions | ||
# * **MET_INSTALL_DIR** - Path to location where MET is installed locally | ||
# | ||
# and for the [exe] section, you will need to define the location of NON-MET executables. | ||
# If the executable is in the user's path, METplus will find it from the name. | ||
# If the executable is not in the path, specify the full path to the executable here (i.e. RM = /bin/rm) | ||
# The following executables are required for performing series analysis use cases: | ||
# | ||
# Example User Configuration File:: | ||
# | ||
# [dir] | ||
# INPUT_BASE = /path/to/sample/input/data | ||
# OUTPUT_BASE = /path/to/output/dir | ||
# MET_INSTALL_DIR = /path/to/met-X.Y | ||
# | ||
# [exe] | ||
# RM = /path/to/rm | ||
# CUT = /path/to/cut | ||
# TR = /path/to/tr | ||
# NCAP2 = /path/to/ncap2 | ||
# CONVERT = /path/to/convert | ||
# NCDUMP = /path/to/ncdump | ||
# | ||
|
||
############################################################################## | ||
# Expected Output | ||
# --------------- | ||
# | ||
# A successful run will output the following both to the screen and to the logfile:: | ||
# | ||
# INFO: METplus has successfully finished running. | ||
# | ||
|
||
############################################################################## | ||
# Keywords | ||
# -------- | ||
# | ||
# .. note:: | ||
# `UserScriptUseCase <https://dtcenter.github.io/METplus/develop/search.html?q=UserScriptUseCase&check_keywords=yes&area=default>`_, | ||
# `S2SAppUseCase <https://dtcenter.github.io/METplus/search.html?q=S2SAppUseCase&check_keywords=yes&area=default>`_, | ||
# | ||
# sphinx_gallery_thumbnail_path = '_static/s2s-UserScript_Space_Time_Spectra.png' |
134 changes: 134 additions & 0 deletions
134
docs/use_cases/model_applications/s2s/UserScript_obsPrecip_obsOnly_CrossSpectraPlot.py
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,134 @@ | ||
""" | ||
UserScript: Make a Cross Spectra plot | ||
======================================================================== | ||
|
||
model_applications/ | ||
s2s/ | ||
UserScript_obsPrecip_obsOnly_CrossSpectraPlot.py | ||
|
||
""" | ||
|
||
################################################################################# | ||
# Scientific Objective | ||
# -------------------- | ||
# | ||
# This use case calls the METplotpy space time plot to create a sample cross | ||
# spectra diagram | ||
# using sample data created by METcalcpy cross spectra functions | ||
# | ||
# The space time plot and cross spectra calculations were created by Maria Gehne | ||
# at the | ||
# Physical Sciences Labratory in NOAA | ||
|
||
################################################################################# | ||
# Datasets | ||
# -------- | ||
# | ||
|
||
############################################################################## | ||
# METplus Components | ||
# ------------------ | ||
# | ||
# This use case runs the UserScript wrapper tool to run a user provided script, | ||
# in this case, cross-spectra.py. | ||
# | ||
|
||
############################################################################## | ||
# METplus Workflow | ||
# ---------------- | ||
# | ||
# This use case does not loop but plots the entire time period of data | ||
# | ||
# UserScript | ||
# | ||
# | ||
|
||
############################################################################## | ||
# METplus Configuration | ||
# --------------------- | ||
# | ||
# METplus first loads all of the configuration files found in parm/metplus_config, | ||
# then it loads any configuration files passed to METplus via the command line | ||
# with the -c option, i.e. -c parm/use_cases/model_applications/s2s/UserScript_obsPrecip_obsOnly_CrossSpectraPlot.conf | ||
# | ||
# .. highlight:: bash | ||
# .. literalinclude:: ../../../../parm/use_cases/model_applications/s2s/UserScript_obsPrecip_obsOnly_CrossSpectraPlot/cross-spectra_diagram.conf | ||
# | ||
|
||
############################################################################# | ||
# MET Configuration | ||
# --------------------- | ||
# | ||
# There are no MET tools used in this use case. | ||
# | ||
|
||
############################################################################## | ||
# Python Embedding | ||
# ---------------- | ||
# | ||
# There is no python embedding in this use case | ||
# | ||
|
||
############################################################################## | ||
# Running METplus | ||
# --------------- | ||
# | ||
# This use case can be run two ways: | ||
# | ||
# 1) Passing in UserScript_obsPrecip_obsOnly_CrossSpectraPlot.conf, | ||
# then a user-specific system configuration file:: | ||
# | ||
# run_metplus.py \ | ||
# -c /path/to/METplus/parm/use_cases/model_applications/s2s/UserScript_obsPrecip_obsOnly_CrossSpectraPlot.conf \ | ||
# -c /path/to/user_system.conf | ||
# | ||
# 2) Modifying the configurations in parm/metplus_config, then passing in UserScript_obsPrecip_obsOnly_CrossSpectraPlot.conf:: | ||
# | ||
# run_metplus.py \ | ||
# -c /path/to/METplus/parm/use_cases/model_applications/s2s/UserScript_obsPrecip_obsOnly_CrossSpectraPlot.conf | ||
# | ||
# The former method is recommended. Whether you add them to a user-specific configuration file or modify the metplus_config files, the following variables must be set correctly: | ||
# | ||
# * **INPUT_BASE** - Path to directory where sample data tarballs are unpacked (See Datasets section to obtain tarballs). This is not required to run METplus, but it is required to run the examples in parm/use_cases | ||
# * **OUTPUT_BASE** - Path where METplus output will be written. This must be in a location where you have write permissions | ||
# * **MET_INSTALL_DIR** - Path to location where MET is installed locally | ||
# | ||
# and for the [exe] section, you will need to define the location of NON-MET executables. | ||
# If the executable is in the user's path, METplus will find it from the name. | ||
# If the executable is not in the path, specify the full path to the executable here (i.e. RM = /bin/rm) | ||
# The following executables are required for performing series analysis use cases: | ||
# | ||
# Example User Configuration File:: | ||
# | ||
# [dir] | ||
# INPUT_BASE = /path/to/sample/input/data | ||
# OUTPUT_BASE = /path/to/output/dir | ||
# MET_INSTALL_DIR = /path/to/met-X.Y | ||
# | ||
# [exe] | ||
# RM = /path/to/rm | ||
# CUT = /path/to/cut | ||
# TR = /path/to/tr | ||
# NCAP2 = /path/to/ncap2 | ||
# CONVERT = /path/to/convert | ||
# NCDUMP = /path/to/ncdump | ||
# | ||
|
||
############################################################################## | ||
# Expected Output | ||
# --------------- | ||
# | ||
# A successful run will output the following both to the screen and to the logfile:: | ||
# | ||
# INFO: METplus has successfully finished running. | ||
# | ||
|
||
############################################################################## | ||
# Keywords | ||
# -------- | ||
# | ||
# .. note:: | ||
# `UserScriptUseCase <https://dtcenter.github.io/METplus/develop/search.html?q=UserScriptUseCase&check_keywords=yes&area=default>`_, | ||
# `S2SAppUseCase <https://dtcenter.github.io/METplus/search.html?q=S2SAppUseCase&check_keywords=yes&area=default>`_, | ||
# | ||
# sphinx_gallery_thumbnail_path = '_static/UserScript_obsPrecip_obsOnly_CrossSpectraPlot.png' |
56 changes: 56 additions & 0 deletions
56
parm/use_cases/model_applications/s2s/SpaceTimeSpectraDiagram.conf
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,56 @@ | ||
[config] | ||
|
||
# time looping - options are INIT, VALID, RETRO, and REALTIME | ||
# If set to INIT or RETRO: | ||
# INIT_TIME_FMT, INIT_BEG, INIT_END, and INIT_INCREMENT must also be set | ||
# If set to VALID or REALTIME: | ||
# VALID_TIME_FMT, VALID_BEG, VALID_END, and VALID_INCREMENT must also be set | ||
LOOP_BY = REALTIME | ||
|
||
# %Y = 4 digit year, %m = 2 digit month, %d = 2 digit day, etc. | ||
# see www.strftime.org for more information | ||
# %Y%m%d%H expands to YYYYMMDDHH | ||
VALID_TIME_FMT = %Y%m%d%H | ||
|
||
# BLank for this usecase but the parameter still needs to be there | ||
VALID_BEG = | ||
|
||
# BLank for this usecase but the parameter still needs to be there | ||
VALID_END = | ||
|
||
# BLank for this usecase but the parameter still needs to be there | ||
VALID_INCREMENT = | ||
|
||
# List of forecast leads to process for each run time (init or valid) | ||
# In hours if units are not specified | ||
# If unset, defaults to 0 (don't loop through forecast leads) | ||
LEAD_SEQ = | ||
|
||
# Order of loops to process data - Options are times, processes | ||
# Not relevant if only one item is in the PROCESS_LIST | ||
# times = run all wrappers in the PROCESS_LIST for a single run time, then | ||
# increment the run time and run all wrappers again until all times have | ||
# been evaluated. | ||
# processes = run the first wrapper in the PROCESS_LIST for all times | ||
# specified, then repeat for the next item in the PROCESS_LIST until all | ||
# wrappers have been run | ||
LOOP_ORDER = processes | ||
|
||
PROCESS_LIST = UserScript | ||
|
||
USER_SCRIPT_RUNTIME_FREQ = RUN_ONCE | ||
|
||
USER_SCRIPT_COMMAND = {PARM_BASE}/use_cases/model_applications/s2s/SpaceTimeSpectraDiagram/cross_spectra_plot.py | ||
|
||
|
||
[user_env_vars] | ||
|
||
# Difficulty index specific variables | ||
|
||
LOG_FILE = "cross_spectra_plot.log" | ||
|
||
LOG_LEVEL = "INFO" | ||
|
||
INPUT_FILE_NAMES = {INPUT_BASE}/model_applications/s2s/SpaceTimeSpectraDiagram/SpaceTimeSpectra_ERAI_P_D200_symm_2spd.nc,{INPUT_BASE}/model_applications/s2s/SpaceTimeSpectraDiagram/SpaceTimeSpectra_ERAI_TRMM_P_symm_2spd.nc,{INPUT_BASE}/model_applications/s2s/SpaceTimeSpectraDiagram/SpaceTimeSpectra_ERAI_P_D850_symm_2spd.nc | ||
|
||
METPLOTPY_BASE = {METPLUS_BASE}/METplotpy/metplotpy/ |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It looks like this filename should also change to match the use case name.