Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
* add gfs gempak downstream jobs into def file * Issue #94 add waves-prdgen, ICE->ICEC, Sleep in gempak script * Issue #1: update name of DA jobs and scripts in accordance with WCOSS Implementation Standards * Increase walltime for new wavepost jobs * Issue #1: correct DA typos in sorc/link_fv3gfs.sh * deleted relocate_mv_nvortex.fd since storm relocation is no longer needed. modified build_tropcy_NEMS.sh to remove references to relocate_mv_nvortex * Restructured ecflow - up to post step * adding the line to go back a day for RTOFS for the if not NCO section because RTOFS will not be available until 06 cycle * Issue #1: add fhrgrp and fhrlst back to gfsawips in setup_workflow.py (bugfix) * updates for by hour post * Adding wafs wave and downstream jobs * Issue #131 added a path to DBNROOT * Issue #131 reduce scripts output to logfile * Small updates: - new UPP tag - new GLDAS tag - new WAFS tag - new module for WAFS - EE2 updates to awips scripts - added WAFS to archival - break downstream and WAFS archival into separate gfs_downstream tarball - update gfsarch dependencies to wait for all wavepost jobs to complete * Update WAFS tag to gfs_wafs.v6.0.6 * ecflow full day cycle included * Fix for running prep on Hera * Issue #1: update to UPP tag "upp_gfsv16_release.v1.0.15" * Issue #131 Unify dbn_alert path * Issue #1 - fix to link_fv3gfs.sh for new GLDAS tag * Issue #1 - update to UFS_UTILS ops-gfsv16.0.0 tag * Issue #1 - update SEND variables and add DBNROOT to base configs and add check to build_enkf_chgres_recenter_nc.sh for GSI build * updates for optimizing point jobs * renamed: scripts/exglobal_fcst_nemsfv3gfs.sh -> scripts/exglobal_forecast.sh and modified jobs/JGLOBAL_FORECAST parm/config/config.fcst * renamed: jobs/JGFS_POSTSND -> jobs/JGFS_ATMOS_POSTSND renamed: scripts/exgfs_postsnd.sh -> scripts/exgfs_atmos_postsnd.sh modified: docs/archive/README_bufr driver/product/run_postsnd.sh driver/product/run_postsnd.sh.cray driver/product/run_postsnd.sh.dell driver/product/run_postsnd.sh.hera driver/product/run_postsnd.sh.jet parm/config/config.postsnd * In anticipating changes from the GLDAS repo : renaming JGDAS_GLDAS to JGDAS_ATMOS_GLDAS, and exgdas_gldas.sh to exgdas_atmos_gldas.sh modified: driver/gdas/test_gdas_gldas.sh jobs/rocoto/gldas.sh parm/config/config.gldas sorc/link_fv3gfs.sh * Rename scripts to match ecf script naming convention. Add SENDDBN and DBNROOT. Update GLDAS tag. * Update EMC_verif-global tag to verif_global_v1.11.0 * Updated jobs name * 1)Update sorc/checkout.sh with new UPP tag upp_gfsv16_release.v1.0.16. 2)Update sorc/link_fv3gfs.sh with new file name convention for jjob and ex-script of post processing part. * Update Externals.cfg with new UPP tag upp_gfsv16_release.v1.0.16. * Updated scripts * Updated drivers and release notes * Fixing spelling mistake in config.gldas * Name change for tropcy scripts and update WAFS tag * Script name updates for sfc_prep and tracker * Update config.base.nco.static with config.base.emc.dyn changes * Modify each ecflow script with old j-job name for test. Modify the following in each definition file. - ecen family require new extern from previous cycle - Job jgfs_atmos_npoess_pgrb2_0p5deg trigger changed to be (requested from code manager): 1. jgfs_atmos_post_manager:release_post180 - Job jgdas_enkf_select_obs trigger changed to be: 1. previous cycle enkf post complete 2. current cycle jgdas_atmos_analysis_calc complete (new job) Tested as of 9/21/2020 before production switch with following condition: 1. Known issue in wafs gcip. Job failed. Waiting for code manager to fix. 2. obsproc testing still on going. 3. Code manager still updating j-job and ex-script names. Testing in hold until package is ready and WCOSS availability. * As requested from management, point ufs-weather-model to development: b8c5c22 git submodule f61416fef691d9ba39a40df1ce72aa574f54c390 FMS (2019.01.03) 6bc61df FV3 (heads/develop) 9d05172b711f4ab5d6f978dbe575bd67a681b55a NEMS (heads/develop) 96e3f3a8fa0389a4b110b0fa23e7a414f6d92038 WW3 (6.07.1-50-g96e3f3a8) ffdd19bc6c1df747394b7e9958a76238fcd44242 stochastic_physics (ufs-v1.0.0-70-gffdd19b) Remove aerosol from checkout * Update post.sh UPP JJOB script name to submit * Making J-Job naming change accourding to code manager. Remove temp files * modifications to python scripts remove aeroic task from setup_workflow_fcstonly.py delete setup_workflow_fcstonly_noaeroic.py add setup_workflow_fcstonly_aeroic.py * Update WAFS jobs/rocoto scripts to use new JJOB names * Update config.awips for newly named JJOB scripts * Issue #94 add /fakedbn to run DBN_alerts * Issue #1: update vrfy.sh to submit vsdb processing as separate job (only on WCOSS_DELL_P3) * Update gfswafs job to run with loop over fcsthrs * Update GLDAS tag to gldas_gfsv16_release.v1.10.0 * The EMC realtime parallel does not use operational job settings. Ecflow job card roll back the setting from module_used_gfs-16_job google sheet document. * Code manager indicated all wafs jobs wall clock is 30 mins. Code manager indicated job card for scripts/gfs/atmos/gempak/jgfs_atmos_pgrb2_spec_gempak.ecf need to be changed * Issue #1: update parm/config.vrfy to define VSDBJOBSH (used by jobs/rocoto/vrfy.sh) * Jobs were tested with PDY 20200925, code managers from post, gempak, wave, and post process certified the test run result. This merge included an update from high watermark testing (tested by gfs team using devonprod) results impact the following jobs: jgfs_atmos_analysis.ecf jgfs_forecast.ecf jgdas_atmos_analysis.ecf jgdas_enkf_update.ecf jgdas_enkf_ecen.ecf * cleaning up the rearranged scripts * Issue #1: update name of ncdiag executable and source code directory to be conistent * Issue #94 add native grids as default grids * - updated to use 28Sep develop ufs-weather-model, d021e7b f61416fef691d9ba39a40df1ce72aa574f54c390 FMS (2019.01.03) 9e1ba7c FV3 (heads/develop) 9d05172b711f4ab5d6f978dbe575bd67a681b55a NEMS (heads/develop) 96e3f3a8fa0389a4b110b0fa23e7a414f6d92038 WW3 (6.07.1-50-g96e3f3a8) ffdd19bc6c1df747394b7e9958a76238fcd44242 stochastic_physics (ufs-v1.0.0-70-gffdd19b) - changed compilation options in build_fv3.sh - removed fv3gfs.fd_jkh directory since changes are already in develop branch - updated getic script to retrieve files after 00Z 26Feb20 from mass store with prefix name of 'com' * Issue #1 - update WAFS tag to gfs_wafs.v6.0.8 * Issue #94 fix for failing silently * updates for EE2 from waves * last of EE2 changes * update resources and trigger from 192->180 * bug fix in exgfs_wave_post_pnt.sh * fix resource time estimates * reverting changes to configs that were not intented to be committed * Issue #1 - updates for modules and small fixes - fix to run ens_tracker without tclogg module in modulefiles/module_base.wcoss_dell_p3 - fix to sorc/syndat_qctropcy.fd/qctropcy.f for compile warning - fix to gfswaveawipsbulls dependency in setup scripts - update to bufr/11.3.0 from bufr/11.2.0 in modulefiles - add override ability for POE for prep jobs in env/WCOSS_DELL_P3.env * issue #145 change dev path to prod for parallel netcdf modules * issue #145 change dev path to prod for parallel netcdf modules * issue #142 add 6 bufr stations for Thailand TMD * issue #142 add 6 bufr station data * issue #142 generate station i,j grid * issue #142 generate station i,j grid * issue #142 generate station i,j grid * Issue #1 - adjust error handling in wave rocoto job scripts * Issue #1 - update component tags and modulefiles for nwtest lib updates, remove unneeded module load and modulefile from downstream wave job rocoto scripts * Issue #1 - update GSI tag to gfsda.v16.0.0 * Issue #1 - remove unneeded line in vrfy.sh and update link_fv3gfs.sh for UFS_UTILS execs * Issue #1 - config updates from real-time parallel * Issue #1 - update g2tmpl module load in modulefiles/module_base.wcoss_dell_p3 * Issue #1 - update prep job resources * Issue #1 - update config.fv3 based on real-time parallel * Issue #1 - move ABIBF, AHIBF, and HDOB pointers into RUN_ENVIR=emc block * Issue #1 - remove hardcoded POE and BACK values from config.prep * Issue #1 - return POE=YES and BACK=off setting for prep on WCOSS_DELL_P3 * for rocoto add a dependency to wavepostpnt job on wavepostbndpnt so that both jobs will not run at the same time which will slow both jobs down. This is the reason for the dependency, otherwise there is not a "true" dependency between the jobs * Issue #1 - remove unneeded DMPDIR and ICSDIR from config.base.nco.static * Issue #1 - remove POE/BACK block from config.prep and set POE=YES/BACK=off as defaults in env/WCOSS_DELL_P3.env prep section * Issue #1 - update WAFS tag to gfs_wafs.v6.0.9 * Temporarily peg GSI checkout to hash of release branch * add a dependency for the wavepostpnt on wavepostbndpnt for just gfs as this job does not exist for gdas * Modify module for each job to match implementation package change Modify two wafs jobs trigger Modify wall clock and resource for running jobs in NCO Modify obsproc package location * updates to parm to reduce the number of wave variables changes to the config so that wave models are interpolated to the multi_1 masked files for the regional output grids * Issue #1 - update to fbwndgfs modulefiles for WCOSS-Dell and WCOSS-Cray * Change config.resources for eobs for low resolution cases * Issue #1 - update gfs_util modulefiles * Issue #1 - update config.fv3 and config.resources with v16rt2 values * Issue #1 - update WAFS tag to gfs_wafs.v6.0.10 and change WAFS job dependencies * Modify two wafs jobs trigger as: jgfs_atmos_wafs_grib2 trigger ../../post/jgfs_atmos_post_f000 == complete jgfs_atmos_wafs_grib2_0p25 trigger ../../post/jgfs_atmos_post_f036 == complete * Issue #1 - update link_fv3gfs.sh to point to newly frozen fix_nco_gfsv16 FIX_DIR * Modify gfs/gdas post job to 20 minutes in wall clock. * updates to add glo_30m to the created grib files for waves for awips processing * Issue #94 producing awips files with masks and deleting wmo headers for arctic ocean * Issue #1 - adding release notes for GFSv16 * Issue #1 - adjust WAFS dependencies to wait for f036 post output * Add wave env updates into ORION.env, increase waveinit tasks, and update resources for analysis and efcs walltimes * Update .gitignore for script renaming and removed external files * Issue #1 - update FV3 tag to GFS.v16.0.14 for Hera/Orion build support * Issue #1 - new FV3 tag that builds on Hera/Orion and update for building/running high res system on Hera * Issue #1 - save GFSv16 hand-off resource settings into new static nco configs * Issue #1 - add FDATE to config.base * Issue #1 - add parm mon folder to ignore list and move parm section of ignore list up * Issue #1 - pull in corrected npe_eobs values in config.resources * Add UPP netcdf module library load to HERA.env to resolve runtime netcdf version mismatch * Issue #1 - increase C768 npe_eobs to 200 for Hera nodes with less memory * Issue #1 - resource updates from low res testing - set io_layout to "1,1" for low res gfs in config.fcst - set npe_wav[_gfs] to 140 for all resolutions, same wave grid - increase nth_fv3 to 4 for C192 and C384 - set smaller walltime for gfsfcst when resolutions less than C768 * Issue #1 - fix firstcyc queue bug in workflow_utils.py on WCOSS * Issue #1 - add wtime_fcst_gfs time of 4hrs for C384 * Fix wrong DATE in new if-block in config.anal for cold start checking * Issue #1 - increase wavepostbndpnt and wavepostpnt walltimes to give more time on Hera * Cycled warm start run using develop ufs-weather-model branch and feature/ccpp using merged branch on the 10/13 (4da0f) with feature/gfsv16b (31563). CPP_SUITE="FV3_GFS_v16beta" build_fv3 option: CCPP=Y 32BIT=Y SUITES=FV3_GFS_v15,FV3_GSD_v0,FV3_GSD_noah,FV3_GFS_v16beta wave is turned off for this test because it's for atmospheric physics and comparison. * Issue #1 - update WAFS tag to gfs_wafs.v6.0.12 for removal of in-cloud turbulence per AWC * Rename Release_Notes.gfs.v16.0.0.txt to Release_Notes.gfs.v16.0.0.md * Issue #1 - adjust epos groups for DOIAU/DOIAU_ENKF=NO * Issue #1 - update anal, eobs, and eupd resources for low res * Issue #1 - update WAFS tag to gfs_wafs.v6.0.13 * Revert epos change in setup_workflow.py * modified: jobs/JGFS_ATMOS_POSTSND and jobs/rocoto/postsnd.sh to remove redundant variables in the two scripts and make them work for both EMC and NCO parallels. * Issue #1 - update WAFS tag to gfs_wafs.v6.0.14 and update dumpjb version to 5.1.0 * Reduce C384 nth_fv3 to 1 in config.fv3 * Add FDATE calculation to setup scripts and change FDATE in to parsed value in config.base.emc.dyn * Set nth_fcst to 4 for C384 on Hera to handle less memory * Increase gfsfcst walltime for C192 * Tie DOIAU_ENKF to DOIAU and add DOIAU check for IAU_OFFSET and IAU_FHROT in config.base * Set nth_fcst to 4 for C384 deterministic * Hera updates for OUTPUT_FILETYPES and resources * Change wavepostbndpnt to wait for fcst to end * As of 11/12/2020, this package is tested with a C768 cycled (6 cycle) run on Dell system using FV3_GFS_v16beta (imp_physics=11). The wave components are turned off. Ready to merge back to the development. The expdir and log files are available on HPSS: /NCEPDEV/emc-global/1year/Lin.Gan/WCOSS_DELL_P3/feature_ccpp/FV3_GFS_v16beta/FV3_GFS_v16beta-LOG.tar /NCEPDEV/emc-global/1year/Lin.Gan/WCOSS_DELL_P3/feature_ccpp/FV3_GFS_v16beta/FV3_GFS_v16beta-EXPT.tar File parm/config/config.resources has been modified to fix EUPD job issue in job card resource. File sorc/build_fv3.sh and sorc/link_fv3gfs.sh changed to clear up development remark. Removed a developer directory that is not required for FV3_GFS_v16beta confiruration. * Changes to be committed: modified: jobs/rocoto/fcst.sh deleted: modulefiles/module_base.wcoss_dell_p3_fcst deleted: parm/config/config.aeroic modified: parm/config/config.base.emc.dyn modified: parm/config/config.resources modified: sorc/build_fv3.sh modified: sorc/checkout.sh modified: sorc/link_fv3gfs.sh deleted: ush/load_fv3gfs_modules_fcst.sh deleted: ush/rocoto/setup_workflow_fcstonly_aeroic.py * deleted: jobs/rocoto/aeroic.sh * Remove GSD suite file and aero IC extract util * sync config.base.emc.dyn and config.resources with development * modified: parm/config/config.base.emc.dyn parm/config/config.base.nco.static scripts/exglobal_forecast.sh sorc/build_all.sh sorc/build_fv3.sh sorc/checkout.sh to add the option to check out ufs_weather_model and run with CCPP phsyics * Merge with v16ccpp modified: parm/config/config.base.emc.dyn modified: parm/config/config.resources modified: scripts/exglobal_forecast.sh modified: sorc/build_fv3.sh modified: sorc/checkout.sh * update eupd resource * Remove development configuration from config.nsst * modified: ../scripts/exglobal_forecast.sh modified: build_all.sh build_fv3.sh to automatically detect model version, CCPP vs IPD * Modified build_all.sh checkout.sh partial_build.sh to allow switch turn on/off ccpp option Turn on CCPP usage: checkout.sh -c build_all.sh -c Turn off CCPP usage: checkout.sh build_all.sh * Clean up development code from build_all * modified: build_fv3.sh * modified: exglobal_forecast.sh to add min_lakeice = ${min_lakeice:-"0.15"} min_seaice = ${min_seaice:-"0.15"} * bug fix exglobal_forecast.sh * modified: ../parm/config/config.fcst and exglobal_forecast.sh 1. use iovr for model after NCAR/ccpp-physics#514 and iovr_sw and iovr_lw for older version of the model, controled by RUN_CCPP 2. remove the if block and use atmos_model_nml to turn on/off the CCPP option * modified checkout.sh to check out a hash instead of head of the develop branch * modified: ../scripts/exglobal_forecast.sh * Modify exglobal_forecast.sh to create namelist in runable sequence. * As requested, using 2e25df5 for ufs-weather-model. * scripts/exglobal_forecast.sh Merge with v16ccpp * Modify build_all.sh while getopts "oc" option to while getopts "c" option * Fix missing COMROOT setting on Orion * Adjust efcs walltime and eupd thread value on Orion * gfsv16 physics; script updates * update IC scripts * model configure changes; preserve $DATA in postdet * update /driver; /parm and /ecflow folder to match gfsv16 * Revert "update /driver; /parm and /ecflow folder to match gfsv16" This reverts commit 45341b6. * configuration update * preserve /atmos level COM directory * typo fix * Coupled/devmerge (#203) * update to waves scripts for merging with develop - synced with develop where necessary and left changes in prep and gridded post for coupled-crow updates that should remain * removing ecflow scripts that are not in develop branch * removing files from branch that do not appear in develop branch and were not added for coupling in the coupled-crow branch * updating sorc and ush files to match dev when these are not files changes intentionally in coupled-crow branch and therefore should match develop branch * updating files to match develop that were not matching through merge but should match develop branch * cleaning up evn files after merge with develop * cleaning up differences in jobs/rocoto folder from develop branch as it does not seem that the changes here were made intentionally in feature/coupled-crow so this brings us insync with the develop branch * resolving differences between develop and this branch in terms of wave variables for jobs/JGLOBAL_FORECAST * updating module file to match develop * updates for wave model so that the forecast runs -- atm post still does not work * fix typo * Fixes for issue #202 (FINDDATE) and issue #208 (postsnd.sh permissions) * coupled: updates for merging checkout, build and link with develop branch (#206) * update to waves scripts for merging with develop - synced with develop where necessary and left changes in prep and gridded post for coupled-crow updates that should remain * removing ecflow scripts that are not in develop branch * removing files from branch that do not appear in develop branch and were not added for coupling in the coupled-crow branch * updating sorc and ush files to match dev when these are not files changes intentionally in coupled-crow branch and therefore should match develop branch * updating files to match develop that were not matching through merge but should match develop branch * cleaning up evn files after merge with develop * cleaning up differences in jobs/rocoto folder from develop branch as it does not seem that the changes here were made intentionally in feature/coupled-crow so this brings us insync with the develop branch * resolving differences between develop and this branch in terms of wave variables for jobs/JGLOBAL_FORECAST * updating module file to match develop * updates for wave model so that the forecast runs -- atm post still does not work * fix typo * updating .gitignore to match develop a little better * cleaning up differences between checkout and link with merging to develop * updates for build and checkout scripts to unify with develop and simplify build_all to add an option for coupling * fix typo in build all script * updating .gitignore for linked files missing from list * add fix files for ocean post * changing -p to -c option and adding coupled to the ccpp -c * configuration and minor script update for inline post * bug fix for vrfy job in layout file; missing namelist variable fix * updates that are needed after merging * Issue #201 - workaround for failing post000 job before hpc-stack solution * Issue #189 - update ufs-weather-model hash * Coupled/devmerge 3 (#219) * update for making post work without inline post * add in linking capabilities * updating module file so that atm pos will work (#214) Co-authored-by: JessicaMeixner-NOAA <Jessica.Meixner@noaa.gov> * set inline_post off for default file dependencies for post job triggering updated * add new parameters into /default * clean up excessive logic clause in namelist parsing scripts * match default value of some namelist variables * name list update for CCPP suite "FV3_GFS_v16beta_coupled" update CCPP suite selection in test case file create prototype 5.1 case file with updated CCPP suite Making each CCPP suite a selectable option is beneficial, because this could handle future development when they start to differ from each other. * change CCPP suite back to FV3_GFS_v15p2_coupled in case file * bugfix for iovr variables Co-authored-by: lin.gan <lin.gan@noaa.gov> Co-authored-by: kate.friedman <kate.friedman@noaa.gov> Co-authored-by: wx21rph <Roberto.Padilla@m71a1.ncep.noaa.gov> Co-authored-by: russ.treadon <Russ.Treadon@noaa.gov> Co-authored-by: Jessica Meixner <jessica.meixner@noaa.gov> Co-authored-by: fanglin.yang <fanglin.yang@noaa.gov> Co-authored-by: Guang Ping Lou <guang.ping.lou@noaa.gov> Co-authored-by: BoiVuong-NOAA <Boi.Vuong@noaa.gov> Co-authored-by: Wen Meng <wen.meng@noaa.gov> Co-authored-by: Judy.K.Henderson <Judy.K.Henderson@noaa.gov> Co-authored-by: Roberto Padilla <Roberto.Padilla@noaa.gov> Co-authored-by: CoryMartin-NOAA <cory.r.martin@noaa.gov> Co-authored-by: wx21rph <Roberto.Padilla@m71a3.ncep.noaa.gov> Co-authored-by: lgannoaa <37596169+lgannoaa@users.noreply.github.com>
- Loading branch information
15 people
authored
Dec 18, 2020
1 parent
fbe9a2a
commit e13dafc