Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Port UFS_UTILS to WCOSS2 #642

Merged
merged 81 commits into from
Jul 13, 2022
Merged

Port UFS_UTILS to WCOSS2 #642

merged 81 commits into from
Jul 13, 2022

Conversation

GeorgeGayno-NOAA
Copy link
Collaborator

@GeorgeGayno-NOAA GeorgeGayno-NOAA commented Apr 6, 2022 •
edited
Loading

DESCRIPTION OF CHANGES:

Add the following for WCOSS2

  • Build modules
  • Consistency test driver scripts.
  • Driver script for creating new grids.

TESTS CONDUCTED:

All testing is documented in #559.

DEPENDENCIES:

None.

DOCUMENTATION:

N/A

ISSUE:

#559

GeorgeGayno-NOAA and others added 30 commits September 7, 2021 21:13
@GeorgeGayno-NOAA
Update machine-setup.sh for wcoss2. Baseline initial
7e87be1 
build module for that machine.

Fixes ufs-community#559.
@GeorgeGayno-NOAA
Kyle's updates for finding the wgrib2 library.
47e3ea8 
Fixes ufs-community#559
@GeorgeGayno-NOAA
Update link_fixdirs.sh for wcoss2. For now, Kate will
51e9fbd 
host the fixed files in her own directory.

Fixes ufs-community#559.
@GeorgeGayno-NOAA
Load additional libraries from the stack instead of letting
f3588cf 
cmake find another version.

Fixes ufs-community#559
@GeorgeGayno-NOAA
Add grid generation driver script for WCOSS2.
7d946c2 
Fixes ufs-community#559.
@GeorgeGayno-NOAA
Update vcoord_gen utility for wcoss2.
a61adfb 
Fixes ufs-community#559
@GeorgeGayno-NOAA
Create ice_blend regression test driver script for wcoss2.
da9db59 
Fixes ufs-community#559.
@GeorgeGayno-NOAA
Create sfc_climo_gen utility script for wcoss2.
301f5d1 
Fixes ufs-community#559.
@GeorgeGayno-NOAA
Add snow2mdl regression test script for wcoss2.
8a25192 
Fixes ufs-community#559.
@GeorgeGayno-NOAA
Add global_cycle regression test script for wcoss2.
1812a0f 
Fixes ufs-community#559.
@GeorgeGayno-NOAA
Add grid_gen regression test driver script for wcoss2.
a4c80f3 
Fixes ufs-community#559.
@GeorgeGayno-NOAA
Minor updates to global_cycle reg test script.
ee51e7d 
Fixes ufs-community#559
@GeorgeGayno-NOAA
Add chgres_cube regression test driver script for wcoss2.
773ae5d 
Fixes ufs-community#559.
@GeorgeGayno-NOAA
Merge branch 'develop' into feature/wcoss2
4d3bf93 
Fixes ufs-community#559.
@GeorgeGayno-NOAA
Begin work on new driver script for gdas_init utility.
bafbef3 
Fixes ufs-community#559.
@GeorgeGayno-NOAA
Complete mods for data pull scripts. Start mods for chgres
1939031 
scripts.

Fixes ufs-community#559.
@GeorgeGayno-NOAA
Add qsub commands to remainder of ./util/gdas_init/driver.wcoss2.sh
7b7cdf0 
script.

Fixes ufs-community#559
@GeorgeGayno-NOAA
Hardware threading was turned off on WCOSS2. Start adjusting
7b0d1f4 
the scripts accordingly.

Fixes ufs-community#559.
@GeorgeGayno-NOAA
Update more scripts now that hardware threading is
b49c138 
turned off.

Fixes ufs-community#559.
@GeorgeGayno-NOAA
Update more scripts for new hardware configuration.
5b50294 
Fixes ufs-community#559.
@GeorgeGayno-NOAA
Update gdas_init driver script for new hardware
49675fd 
configuration.

Fixes ufs-community#559.
@GeorgeGayno-NOAA
Update ./util/gdas_init/driver.wcoss2.sh to process all
d0c7784 
80 enkf members.

Fixes ufs-community#559
@GeorgeGayno-NOAA
Update ice blend test to use v1.2.3 of grib_util. v1.2.2
52a5192 
gave wrong results when replacing the bitmap with the 1.57
flag value.

Fixes ufs-community#559.
@GeorgeGayno-NOAA
Update build module to point to latest esmf v8.
d71d386 
Fixes ufs-community#559
@GeorgeGayno-NOAA
Update consistency test driver scripts to point
4468002 
to my local copy of 'nccmp'. Hope this is temporary.

Fixes ufs-community#559.
@GeorgeGayno-NOAA
Merge branch 'develop' into feature/wcoss2
b0e1a14 
Fixes ufs-community#559
@GeorgeGayno-NOAA
Merge branch 'develop' into feature/wcoss2
3c6ceb4 
Fixes ufs-community#559.
@GeorgeGayno-NOAA
Update fix/link_fixdirs.sh to point to the correct 'fix'
8256d77 
directories.

Fixes ufs-community#559.
@GeorgeGayno-NOAA
Point to new location for data and utilities on wcoss2.
426ba04 
Fixes ufs-community#559.
@GeorgeGayno-NOAA
Merge branch 'develop' into feature/wcoss2
11fae19 
Fixes ufs-community#559.
@GeorgeGayno-NOAA
Merge branch 'develop' into feature/wcoss2
b9f1ad5 
Fixes ufs-community#559.
@GeorgeGayno-NOAA
Increase memory for the global_cycle regression test.
cc6327d 
Fixes ufs-community#559.
@GeorgeGayno-NOAA
Merge branch 'develop' into feature/wcoss2
ea32748 
Fixes ufs-community#559.
@GeorgeGayno-NOAA
Update to esmf v8.2.
8b948ba 
Fixes ufs-community#559.
@GeorgeGayno-NOAA
Update cpld_gridgen regression test for wcoss2.
e441ea0 
Fixes ufs-community#559.
@GeorgeGayno-NOAA
Update ./link_fixdirs.sh for new location of fix files.
8fe9166 
Fixes ufs-community#559.
@GeorgeGayno-NOAA
Update ./reg_tests/cpld_gridgen/rt.sh to run regression
34f578c 
tests on wcoss2.

Fixes ufs-community#559.
@GeorgeGayno-NOAA
Update ./reg_tests/rt.sh to run cpld_gridgen on wcoss2.
9a13a6b 
Fixes ufs-community#559.
@GeorgeGayno-NOAA
Merge branch 'develop' into feature/wcoss2
c137757 
Fixes ufs-community#559.
@GeorgeGayno-NOAA
Add new chgres_cube regression test to wcoss2 driver script.
340a50f 
Fixes ufs-community#559.
@GeorgeGayno-NOAA
Increase memory for global_cycle regression tests.
af63b99 
Fixes ufs-community#559.
@GeorgeGayno-NOAA
Increase memory in ./util/vcoord_gen/run.wcoss2.sh.
eadc5ca 
Fixes ufs-community#559.
@GeorgeGayno-NOAA
Boost requested memory for gdas init utility.
0268817 
Fixes ufs-community#559.
@GeorgeGayno-NOAA
Minor fix to rt.sh.
29e4daf 
Fixes ufs-community#559.
@GeorgeGayno-NOAA
Update README.md
b75f502 
Fixes ufs-community#559.
@GeorgeGayno-NOAA
Increase requested wall clock time in grid_gen consistency
b4b3570 
test driver script. Remove 'cpld_gridgen' test from
./reg_tests/rt.sh as that program does not run on
WCOSS2.

Fixes ufs-community#559.
@GeorgeGayno-NOAA GeorgeGayno-NOAA marked this pull request as ready for review July 7, 2022 18:59
@GeorgeGayno-NOAA
Copy link
Collaborator Author

Note - the cpld_gridgen program does not work on WCOSS2. But we can hold up this merge any longer. It will be fixed under #663.

@GeorgeGayno-NOAA
Copy link
Collaborator Author

@JeffBeck-NOAA and @LarissaReames-NOAA - Sorry for the short suspense, but can you please review this. WCOSS2 is already online. I should have completed this a month ago. Thanks.

@JeffBeck-NOAA
Copy link
Collaborator

@GeorgeGayno-NOAA, this looks good to me. Was the PR tested on other machines to make sure the build/regression tests still work there?

@GeorgeGayno-NOAA
Copy link
Collaborator Author

@GeorgeGayno-NOAA, this looks good to me. Was the PR tested on other machines to make sure the build/regression tests still work there?

I only have access to Hera, Orion, Jet, WCOSS1 (soon to be retired) and WCOSS2. I am not even sure who the focal points are for the other machines. They are welcome to test my branch before I merge.

@GeorgeGayno-NOAA
Copy link
Collaborator Author

#667 should be also tested on other machines as I am removing stuff, which is more dangerous than adding stuff.

@JeffBeck-NOAA
Copy link
Collaborator

JeffBeck-NOAA commented Jul 13, 2022 •
edited
Loading

Thanks, @GeorgeGayno-NOAA. I built things successfully on Cheyenne, but can't run the regression tests there since rt.sh isn't set up yet for that platform.

@GeorgeGayno-NOAA GeorgeGayno-NOAA merged commit 53f7d18 into ufs-community:develop Jul 13, 2022
@GeorgeGayno-NOAA GeorgeGayno-NOAA deleted the feature/wcoss2 branch July 13, 2022 17:26
@KateFriedman-NOAA KateFriedman-NOAA mentioned this pull request Aug 15, 2022
Closed
AnnetteGibbs-NOAA added a commit to AnnetteGibbs-NOAA/UFS_UTILS that referenced this pull request Jul 3, 2023
@AnnetteGibbs-NOAA
Merge remote-tracking branch 'origin/develop' into feature/dyn_dng
2286a16 
* origin/develop:
  Update ccpp submodule (ufs-community#827)
  Change length of character variable "mosaic_name" in "make_solo_mosaic.c" (ufs-community#824)
  Update GDAS INIT utility for GFS COM reorganization (ufs-community#820)
  Add backup calculation for orography and mask generation (ufs-community#713)
  Improve repository build script (ufs-community#819)
  Use latest wgrib2 executable on Jet. (ufs-community#816)
  chgres_cube - Minor update for high-resolution grids . (ufs-community#814)
  Update PR template (ufs-community#813)
  global_cycle - Add soil moisture nudging for NoahMP option (ufs-community#809)
  Release v1.10.0 (ufs-community#807)
  Run sfc_climo_gen utility on Hera (ufs-community#791)
  Fix bug in chgres_cube subroutine search_many (ufs-community#808)
  sfc_climo_gen - Output fractions of each vegetation/soil type category (ufs-community#748)
  Replace w3nco library with w3emc (ufs-community#802)
  Fix Debug compilation (ufs-community#800)
  Update WCOSS2 build module to use new ESMF v8.4.1 library. (ufs-community#799)
  Update chgres_cube documentation for duplicate grib2 records (ufs-community#795)
  Use latest Cray modules on WCOSS2 (ufs-community#792)
  Remove checksum attribute from global_cycle restart files (ufs-community#794)
  Remove goto statements from chgres_cube (ufs-community#775)
  FVCOM_TOOLS - Remove dependency on module_nwp_base.f90 (ufs-community#790)
  Remove "gcovr" step from the 'developer' workflow (ufs-community#785)
  Update build module for Cheyenne (ufs-community#783)
  GDAS Init utility - consolidate the copy of coldstart files (ufs-community#773)
  Remove support for ODIN machine (ufs-community#782)
  Update GDAS INIT utility to optionally use fracoro data (ufs-community#741)
  Option to build only application specific utilities (e.g. GFS) (ufs-community#777)
  Use combined IP/IP2 library (ufs-community#695)
  Initial updates to global_cycle for Noah-MP land model (ufs-community#774)
  New resolution options for the cpld_gridgen utility (ufs-community#769)
  Use new EPIC-maintained hpc-stack on Jet  (ufs-community#771)
  Update GDAS initialization scripts for the new ENKF directory (ufs-community#764)
  Reduce memory usage in chgres_cube (ufs-community#766)
  Detect duplicate vertical levels in chgres_cube. (ufs-community#767)
  Point to new input orography directory. (ufs-community#758)
  Release version 1.9 (ufs-community#754)
  HAFSv1 grid nesting updates (ufs-community#752)
  Remove compiler warnings from chgres_cube (ufs-community#747)
  chgres_cube: Split input_data module into to 3 separate modules (ufs-community#744)
  More work on CI, checking with different versions of ESMF (ufs-community#742)
  Add processing of soil color to sfc_climo_gen (ufs-community#721)
  more work on CI - added Linux_versions workflow (ufs-community#739)
  Add TEST_FILE_DIR option to CMake build, where test data files can be found instead of using FTP.  (ufs-community#732)
  adding developer workflow (ufs-community#724)
  Update GDAS INIT utility for v16.3. (ufs-community#723)
  OROG_GSL - Remove negative bias in orographic asymmetery (OA) fields (ufs-community#718)
  Incorporate BNU soil type data (ufs-community#717)
  High-resolution MODIS and STATSGO veg/soil data. (ufs-community#703)
  Run WCOSS2 consistency tests under role account (ufs-community#711)
  Fix macOS CI workflow (ufs-community#715)
  Add utility codes to create BNU soil texture data (ufs-community#707)
  Update chgres_cube to output netcdf4 file. (ufs-community#704)
  Incorporate high-res global VIIRS vegetation data. (ufs-community#699)
  Run grid_gen consistency tests in parallel. (ufs-community#697)
  Fix error handling in "link_fixdirs.sh". Add -L to the copy command. (ufs-community#701)
  Update for new fixed data directory structure (ufs-community#688)
  Add WCOSS2 support for tests (ufs-community#693)
  Increase test data pull timeout (ufs-community#692)
  Option to install binaries to any directory. (ufs-community#685)
  Update global_cycle to use the latest CCPP version of sfcsub.F (ufs-community#671)
  Incorporate weight_gen program. (ufs-community#677)
  Add install of jpeg-turbo to macos workflows (ufs-community#684)
  Improve logic in regression test driver script (ufs-community#681)
  Release v1.8 (ufs-community#679)
  Update link_fixdirs.sh for new data directories (ufs-community#672)
  Update default ice climatology in ./ush/global_cycle.sh. (ufs-community#664)
  Remove all references to WCOSS 1 from UFS_UTILS (ufs-community#667)
  Update workflows to use latest macOS and ubuntu (ufs-community#675)
  Update GDAS Initialization utility for recent HPSS tarball name change (ufs-community#666)
  Port UFS_UTILS to WCOSS2 (ufs-community#642)
  Fix chgres_cube to process GEFS GRIB2 data (ufs-community#658)
  Update global_cycle_driver.sh for GFS OPS directory convention  (ufs-community#655)
  Update documentation for SRW App (ufs-community#656)
  New coupled model utility (ufs-community#647)
  Update support for S4 and enable regression testing (ufs-community#654)
  global_cycle - Link to CCPP version of sfcsub.F (ufs-community#636)
  Add processing of new global AFWA snow data to emcsfc_snow2mdl. (ufs-community#648)
  Update build module on Cheyenne (ufs-community#646)
  Move to Intel 2022 on Jet, Hera and Orion (ufs-community#650)
  Host doxygen documentation for multiple releases (ufs-community#644)
  Download unit test data as part of the CMake build (ufs-community#630)
  chgres_cube - Complete removal of wgrib2 library (ufs-community#641)
  Eliminate circular dependency in chgres_cube
  Update workflow files to pull netcdf-c library from GitHub
  chgres_cube - Remove the wgrib2 library from the GRIB2 data read routines.
  Undefined symbols on macOS with Intel compiler (ufs-community#628)
  Update FVCOM code to handle sub-domain restart files using multiple cores. (ufs-community#624)
  chgres_cube - Run routine 'convert_omega' on all tasks. (ufs-community#627)
  Use ESMF 8.2.0 library
  Automate update of consistency test baseline data. (ufs-community#603)
  Update workflow files to use newer versions of ESMF and NCEPLIBS. (ufs-community#617)
  Update build modules to be lua compliant (ufs-community#614)
  Allow FVCOM tools to Update Ice Surface Roughness Length (ufs-community#604)
  Update the requested memory in the Orion chgres_cube consistency test script (ufs-community#611)
  Use copy of grib_util under Jet role account. (ufs-community#608)
  Run consistency tests on Orion using role account (ufs-community#606)
  Run consistency tests on Hera using role account (ufs-community#605)
  Run consistency tests on Jet using role account. (ufs-community#607)
  Update more documentation after move to ufs-community (ufs-community#597)
  fvcom_tools - Add option to process 'cold' or 'warm' restart files (ufs-community#595)
  Update documentation after move to ufs-community (ufs-community#594)
  chgres_cube - Eliminate segmentation fault in input_data.F90 (ufs-community#585)
  Update to language of unit test README to match that in unit test.
  chgres_cube - Simplify surface processing using field bundles (ufs-community#572)
  Add compiler flags for GNU Fortran v10 or newer compilers. (ufs-community#583)
  Move verbose output from example unit test to be commented to streamline test output.
  Updates to test README to add instructions for use of example unit test.
  Add Findwgrib2.cmake (ufs-community#578)
  Added unit test to be used as an instructional example for new users.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@JeffBeck-NOAA JeffBeck-NOAA JeffBeck-NOAA approved these changes

@LarissaReames-NOAA LarissaReames-NOAA Awaiting requested review from LarissaReames-NOAA

Assignees

@GeorgeGayno-NOAA GeorgeGayno-NOAA

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@GeorgeGayno-NOAA @JeffBeck-NOAA