Code

Feature #2550 compute_diagnostics#2728

Code

Merged

JohnHalleyGotway merged 79 commits intodevelopdtcenter/MET:developfrom feature_2550_compute_diagnosticsdtcenter/MET:feature_2550_compute_diagnosticsCopy head branch name to clipboardNov 17, 2023

+660,261-4,176

Commits

Commits on Sep 19, 2023

• Squashed 'scripts/python/tc_diag_driver/' content from commit 2faf117e3

  Show description for 61a7a3a
  JohnHalleyGotway

  committed
  61a7a3a

  Copy full SHA for 61a7a3a

• Merge commit '61a7a3ad8cc5b6c12012980e9745fa3551416439' as 'scripts/python/tc_diag_driver'

  JohnHalleyGotway

  committed
  4b93ed8

  Copy full SHA for 4b93ed8

Commits on Sep 28, 2023

• Merge remote-tracking branch 'origin/develop' into feature_2550_compute_diagnostics

  JohnHalleyGotway

  committed
  17c8705

  Copy full SHA for 17c8705

• Per #2550, restructuring the tc_diag python directory.

  JohnHalleyGotway

  committed
  807106e

  Copy full SHA for 807106e

• Per #2550, trying to install python directories but it isn't working well

  JohnHalleyGotway

  committed
  6a5ee58

  Copy full SHA for 6a5ee58

• Per #2550, more restructuring.

  JohnHalleyGotway

  committed
  b08b21a

  Copy full SHA for b08b21a

Commits on Sep 29, 2023

• Per #2550, update Makefile structure to install the python tc_diag computation code

  JohnHalleyGotway

  committed
  08a6b74

  Copy full SHA for 08a6b74

• Per #2550, renaming compute_tc_diagnostics.py to compute_tc_diag.py.

  JohnHalleyGotway

  committed
  d443c5c

  Copy full SHA for d443c5c

• Merge remote-tracking branch 'origin/develop' into feature_2550_compute_diagnostics

  JohnHalleyGotway

  committed
  f61fba9

  Copy full SHA for f61fba9

• Per #2550, update tc_diag to import the atcf_tools, diag_lib, and tc_diag_driver code

  JohnHalleyGotway

  committed
  dc879a0

  Copy full SHA for dc879a0

Commits on Oct 2, 2023

• Per #2550, only whitespace changes

  JohnHalleyGotway

  committed
  1f7efd0

  Copy full SHA for 1f7efd0

Commits on Oct 3, 2023

• Per #2550, update tc_diag_driver code to version 0.6.1 from https://github.com/robertdemariacira/tc_diag_driver to update the error handling logic.

  JohnHalleyGotway

  committed
  547ffa4

  Copy full SHA for 547ffa4

• Per #2550, add the global distance to land data file to the tc_data directory

  JohnHalleyGotway

  committed
  8858160

  Copy full SHA for 8858160

Commits on Oct 4, 2023

• Merge remote-tracking branch 'origin/develop' into feature_2550_compute_diagnostics

  JohnHalleyGotway

  committed
  2a16674

  Copy full SHA for 2a16674

• Per #2550, add scripts/python/tc_diag/config directory and stash the python diagnostics config files there.

  JohnHalleyGotway

  committed
  6350597

  Copy full SHA for 6350597

• Per #2550, update configure and Makefile.in for tc_diag/config after running bootstrap on seneca.

  JohnHalleyGotway

  committed
  a45d5d7

  Copy full SHA for a45d5d7

Commits on Oct 6, 2023

• Per #2550, update to version 0.7.0 of tc_diag_driver to make the land file optional.

  JohnHalleyGotway

  committed
  559fd12

  Copy full SHA for 559fd12

• Per #2550, work in progress. Updates to compute_tc_diag.yml to use tc_diag_driver version 0.7.0. Still need to update the TC-Diag C++ code accordingly and decide how to address issues in diagnostic…

  Show description for d0a9a8a
  JohnHalleyGotway

  committed
  d0a9a8a

  Copy full SHA for d0a9a8a

Commits on Oct 7, 2023

• Per #2550, work in progress. Getting closer but getting a segfault when the MET C++ parses the results.

  JohnHalleyGotway

  committed
  676231f

  Copy full SHA for 676231f

Commits on Oct 13, 2023

• Per #2550, add StringArray::serialize() member function.

  JohnHalleyGotway

  committed
  e1abc15

  Copy full SHA for e1abc15

• Per #2550, allow option to disable the alignment of decimals for AsciiTable objects since they are NOT aligned in CIRA diag output files.

  JohnHalleyGotway

  committed
  782f0c6

  Copy full SHA for 782f0c6

• Per #2550, enhance NumArray::sort_array(bool increasing) to support sorting in both increasing and decreasing order.

  JohnHalleyGotway

  committed
  b253946

  Copy full SHA for b253946

• Per #2550, ensure that at least one of the three output types is requested in the TC-Diag config file

  JohnHalleyGotway

  committed
  0b1b49d

  Copy full SHA for 0b1b49d

• Per #2550, update the unit test to write 3 output types and the xml to check for their creation.

  JohnHalleyGotway

  committed
  f0e52c9

  Copy full SHA for f0e52c9

• Per #2550, update tc_diag python embedding logic to keep track of the insertion order using a vector of map keys.

  JohnHalleyGotway

  committed
  596d262

  Copy full SHA for 596d262

• Per #2550, major upgrade to tc_diag to actually retrieve the diagnostics from the python script and write them to NetCDF and ASCII output files.

  JohnHalleyGotway

  committed
  56c4b14

  Copy full SHA for 56c4b14

Commits on Oct 18, 2023

• Per #2550, need to install python/tc_diag/config directory.

  JohnHalleyGotway

  committed
  7af67a8

  Copy full SHA for 7af67a8

• Per #2550, install python/tc_diag/config directory.

  JohnHalleyGotway

  committed
  92812b6

  Copy full SHA for 92812b6

• Per #2550, need to call Xarray.data with this python version to avoid runtime error

  JohnHalleyGotway

  committed
  3bfb46b

  Copy full SHA for 3bfb46b

• Per #2550, add SKIP_DATA_NAMES and make the script more pythonic

  JohnHalleyGotway

  committed
  e3db0c6

  Copy full SHA for e3db0c6

Commits on Oct 23, 2023

• Per #2550, just make ConcatString::replace() error message more informative.

  JohnHalleyGotway

  committed
  65e7e10

  Copy full SHA for 65e7e10

• Per #2550, add output_base_format and switch from nc_rng_azi_flag to nc_cyl_grid_flag.

  JohnHalleyGotway

  committed
  3caee17

  Copy full SHA for 3caee17

• Merge branch 'feature_2550_compute_diagnostics' of https://github.com/dtcenter/MET into feature_2550_compute_diagnostics

  JohnHalleyGotway

  committed
  7fe5ab0

  Copy full SHA for 7fe5ab0

• Per #2550, don't actually create the output file streams until we're ready to write the output in case we run into an error during the processing.

  JohnHalleyGotway

  committed
  caaa7aa

  Copy full SHA for caaa7aa

• Per #2550, the output dicts must be defined global

  JohnHalleyGotway

  committed
  8f38418

  Copy full SHA for 8f38418

• Per #2550, move the checking for non-zero diags further down in the logic.

  JohnHalleyGotway

  committed
  a2bc140

  Copy full SHA for a2bc140

• Per #2550, reverting back

  JohnHalleyGotway

  committed
  a8dd44d

  Copy full SHA for a8dd44d

• Per #2550, correct writing the number of custom diags

  JohnHalleyGotway

  committed
  fa7160c

  Copy full SHA for fa7160c

Commits on Oct 24, 2023

• Per #2550, incorporate version TC-Diag python driver code version 0.8.0 from https://github.com/robertdemariacira/tc_diag_driver

  JohnHalleyGotway

  committed
  730a27f

  Copy full SHA for 730a27f

• Per #2550, add new met_post_process.py file to the list of python scripts to be installed by the Makefile.

  JohnHalleyGotway

  committed
  e2458c1

  Copy full SHA for e2458c1

• Per #2550, fix list of diags to skip from 850DVRG to 850DVG

  JohnHalleyGotway

  committed
  c0f827e

  Copy full SHA for c0f827e

• Per #2550, pass the diagnostics comments through to the output cira diag file

  JohnHalleyGotway

  committed
  46b0640

  Copy full SHA for 46b0640

Commits on Oct 25, 2023

• Per #2550, update StringArray::serialize(const char *sep=" ") to take an argument to specify the separator.

  JohnHalleyGotway

  committed
  fa166f8

  Copy full SHA for fa166f8

• Per #2550, add support for the override_diags config option. Still need to document it.

  JohnHalleyGotway

  committed
  8e2ae72

  Copy full SHA for 8e2ae72

• Merge remote-tracking branch 'origin/develop' into feature_2550_compute_diagnostics

  JohnHalleyGotway

  committed
  054d2a0

  Copy full SHA for 054d2a0

• Per #2550, update the documentation about override_diags and add to the todo list.

  JohnHalleyGotway

  committed
  58beee3

  Copy full SHA for 58beee3

Commits on Oct 26, 2023

• Merge remote-tracking branch 'origin/develop' into feature_2550_compute_diagnostics

  JohnHalleyGotway

  committed
  f9e26c4

  Copy full SHA for f9e26c4

• Per #2550, update compute_tc_diag.py to serve up diagnostics in the expected order to match previous CIRA Diag output.

  JohnHalleyGotway

  committed
  289985d

  Copy full SHA for 289985d

• Per #2550, correct the naming convention from DVG to DVRG to match previous CIRA Diag outputs.

  JohnHalleyGotway

  committed
  a2a2b18

  Copy full SHA for a2a2b18

Commits on Oct 27, 2023

• Per #2550, add long_name and domain NetCDF variable attributes for each diagnostic

  JohnHalleyGotway

  committed
  6e1fa94

  Copy full SHA for 6e1fa94

• Per #2550, slight tweak to long names

  JohnHalleyGotway

  committed
  25037d4

  Copy full SHA for 25037d4

• Per #2550, enable separate regridding logic for each field and also write a domain variable to the NetCDF diagnostics output file

  JohnHalleyGotway

  committed
  9833cd3

  Copy full SHA for 9833cd3

Commits on Oct 30, 2023

• Per #2550, rename read/write_tmp_tc_diag.py to just read/write_tmp_diag.py to use consistent naming conventions.

  JohnHalleyGotway

  committed
  812067a

  Copy full SHA for 812067a

• Per #2550, update TC diag python script to group all the data into a single diag_data dictionary.

  JohnHalleyGotway

  committed
  62d2267

  Copy full SHA for 62d2267

• Per #2550, delete unneeded read_tmp_diag.py script since the functionality is provided through pyembed.

  JohnHalleyGotway

  committed
  3951610

  Copy full SHA for 3951610

• Per #2550, update compute_tc_diag.py to group all the diag data into a single diag_data dictionary.

  JohnHalleyGotway

  committed
  867c40b

  Copy full SHA for 867c40b

Commits on Oct 31, 2023

• Per #2550, fix divergence abbreviation as DVRG in the MET python script

  JohnHalleyGotway

  committed
  6d48512

  Copy full SHA for 6d48512

• Per #2550, update the pyembed logic to support MET_PYTHON_EXE for python embedding in TC-Diag

  JohnHalleyGotway

  committed
  4822197

  Copy full SHA for 4822197

• Per #2550, add support for MET_PYTHON_EXE for python embedding in TC-Diag

  JohnHalleyGotway

  committed
  c5ab032

  Copy full SHA for c5ab032

• Per #2550, make DEBUG level consistently 3 when reporting the python path updates for python embedding

  JohnHalleyGotway

  committed
  ad217f1

  Copy full SHA for ad217f1

• Per #2550, remove censor_thresh, censor_val, and convert(x) from the default config file since they don't actually need to exist there and just slow down the processing. Update the docs accordingly…

  Show description for 49884c3
  JohnHalleyGotway

  committed
  49884c3

  Copy full SHA for 49884c3

Commits on Nov 1, 2023

• Per #2550, move progress updates out of code comments and into GitHub issue comment #2550 (comment)

  JohnHalleyGotway

  committed
  b560cbd

  Copy full SHA for b560cbd

• Per #2550, update tc_diag_driver config files to report Z in units of DM instead of default GPM.

  JohnHalleyGotway

  committed
  d6ee8d6

  Copy full SHA for d6ee8d6

Commits on Nov 6, 2023

• Merge remote-tracking branch 'origin/develop' into feature_2550_compute_diagnostics

  JohnHalleyGotway

  committed
  62f8884

  Copy full SHA for 62f8884

• Per #2550, update long names to match those listed in https://docs.google.com/document/d/1bSXN0LvQoKyRMplwiFvERlhjjAG3pCCBu5myvNHtsl8/edit

  JohnHalleyGotway

  committed
  1265748

  Copy full SHA for 1265748

Commits on Nov 7, 2023

• Per #2550, add yaml to the list of required Python packages for MET listed in Appendix F.

  JohnHalleyGotway

  committed
  4e08f20

  Copy full SHA for 4e08f20

• Per #2550, update the MET Dockerfiles to build from METbaseimage version 3.1 to include the yaml Python package.

  JohnHalleyGotway

  committed
  0bde3c2

  Copy full SHA for 0bde3c2

• Per #2550, update the METbaseimage version number in 2 more spots. This is set in at least 4 spots... wish we could consolidate.

  JohnHalleyGotway

  committed
  678b4eb

  Copy full SHA for 678b4eb

• Per #2550, comment out the unused code from cyclindrical_grid.py to eliminate the dependency on scipy.

  JohnHalleyGotway

  committed
  3e0aa7d

  Copy full SHA for 3e0aa7d

Commits on Nov 9, 2023

• Per #2550, redirect standard error to standard out so that it can be captured in the make_install.log file.

  JohnHalleyGotway

  committed
  9b19b9b

  Copy full SHA for 9b19b9b

• Merge remote-tracking branch 'origin/develop' into feature_2550_compute_diagnostics

  JohnHalleyGotway

  committed
  208217e

  Copy full SHA for 208217e

Commits on Nov 10, 2023

• Per #2550, update the MET Dockerfiles to call to update the OS as needed at build time.

  JohnHalleyGotway

  committed
  9a18a9f

  Copy full SHA for 9a18a9f

Commits on Nov 14, 2023

• Per #2550, merge in changes for tc_diag_driver version 0.9.0 which includes updating diag_lib to version 0.5.0. This makes the SciPy Python package optional rather than required.

  JohnHalleyGotway

  committed
  e354b55

  Copy full SHA for e354b55

Commits on Nov 16, 2023

• Merge remote-tracking branch 'origin/develop' into feature_2550_compute_diagnostics

  JohnHalleyGotway

  committed
  b6cb0be

  Copy full SHA for b6cb0be

• Per #2550, update list of expected output file names for unit_tc_diag.xml.

  JohnHalleyGotway

  committed
  bb51db9

  Copy full SHA for bb51db9

Commits on Nov 17, 2023

• Per #2550, TC-Diag documentation updates recommended by @jvigh.

  JohnHalleyGotway

  committed
  b3b767c

  Copy full SHA for b3b767c

• Per #2550, remove note about missing functionality since its been added for MET-12.0.0-beta2

  JohnHalleyGotway

  committed
  ddc49b0

  Copy full SHA for ddc49b0

• Per #2550, fix typo

  JohnHalleyGotway

  committed
  d1cb7e6

  Copy full SHA for d1cb7e6

• Merge remote-tracking branch 'origin/develop' into feature_2550_compute_diagnostics

  JohnHalleyGotway

  committed
  769d442

  Copy full SHA for 769d442