Add Early Cycle EnKF workflow #1022

XianwuXue-NOAA · 2022-09-19T13:26:49Z

Description
Updates workflow to allow early cycle EnKF to run.

Refs: #1021

Type of change

Bug fix (non-breaking change which fixes an issue)

How Has This Been Tested?

Cycled test on Hera

Checklist

My code follows the style guidelines of this project
I have performed a self-review of my own code

On branch feature/dev_early_enkf modified: workflow/applications.py Refs: NOAA-EMC#1021

workflow/applications.py

On branch feature/dev_early_enkf modified: workflow/applications.py modified: workflow/rocoto/workflow_tasks.py Refs: NOAA-EMC#1021

WalterKolczynski-NOAA · 2022-09-27T09:34:24Z

workflow/rocoto/workflow_tasks.py

@@ -1075,7 +1075,7 @@ def atmensanalpost(self):

    def ecen(self):

-        self._is_this_a_gdas_task(self.cdump, 'ecen')
+        #self._is_this_a_gdas_task(self.cdump, 'ecen')


Just delete these instead of commenting them out.

Thanks. Yes, I will delete them later.

On branch feature/dev_early_enkf modified: workflow/applications.py Refs: NOAA-EMC#1021

for cycled gfs On branch feature/dev_early_enkf modified: workflow/rocoto/workflow_tasks.py Refs: NOAA-EMC#1021

On branch feature/dev_early_enkf modified: rocoto/workflow_tasks.py Refs: NOAA-EMC#1021

WalterKolczynski-NOAA · 2022-10-04T12:40:46Z

@aerorahul Other than the commented out lines that need to be removed, is there anything holding this up?

XianwuXue-NOAA · 2022-10-04T13:12:10Z

@WalterKolczynski-NOAA Still working on it. We may change it back (see the last comment), when everything is done, I will delete all commented lines.

On branch feature/dev_early_enkf Changes to be committed: modified: jobs/JGDAS_ENKF_ECEN modified: jobs/JGDAS_ENKF_SFC modified: parm/config/config.anal modified: ush/calcanl_gfs.py Refs: NOAA-EMC#1021

XianwuXue-NOAA · 2022-10-11T18:03:54Z

@aerorahul I have submitted my changes to this PR

ush/calcanl_gfs.py

WalterKolczynski-NOAA

I'm not sure these changes leave late cycle functionality intact.

jobs/JGDAS_ENKF_ECEN

jobs/JGDAS_ENKF_SFC

ush/calcanl_gfs.py

workflow/rocoto/workflow_tasks.py

XianwuXue-NOAA · 2022-10-12T18:36:47Z

@CatherineThomas-NOAA

expdir on Hera: /scratch2/NCEPDEV/ensemble/noscrub/Xianwu.Xue/GEFS_v13_da/test_early_enkf_cold_BOTH_c96_test # my experiment is using C96/C48 with 20 members.
Output: /scratch1/NCEPDEV/stmp2/Xianwu.Xue/o/test_early_enkf_cold_BOTH_c96_test
ICs: /scratch2/NCEPDEV/ensemble/noscrub/Xianwu.Xue/GEFS_v13_da/z_WalterData/C96_cold_IC[C384_cold_IC]

On branch feature/dev_early_enkf Changes to be committed: modified: workflow/rocoto/workflow_tasks.py Refs: NOAA-EMC#1021

TravisElless-NOAA · 2022-10-20T20:52:33Z

jobs/JGDAS_ENKF_SFC

@@ -97,7 +97,7 @@ export COMIN="$ROTDIR/$CDUMP.$PDY/$cyc/$COMPONENT"
 export COMIN_GES="$ROTDIR/$CDUMP.$gPDY/$gcyc/$COMPONENT"


Make sure $CDUMP here in line 97 and in line 83 are also updated to $GDUMP to point the early-cycle to the correct directory

Changes to be committed: modified: jobs/JGDAS_ENKF_SFC Refs: NOAA-EMC#1021

Changes to be committed: modified: jobs/JGDAS_ENKF_FCST modified: ush/forecast_predet.sh Refs: NOAA-EMC#1021

ush/forecast_predet.sh

XianwuXue-NOAA · 2022-10-21T19:52:51Z

@WalterKolczynski-NOAA @TravisElless-NOAA I have completed one and half tests successfully. The log files are in : /scratch1/NCEPDEV/stmp2/Xianwu.Xue/o/test_early_enkf_cold_BOTH_c96/logs/2022012112. I am working on solving the issues which caused failure in lint.

On branch feature/dev_early_enkf Changes to be committed: modified: ush/forecast_predet.sh Refs: NOAA-EMC#1021

Prefer [ p ] && [ q ] as [ p -a q ] is not well defined. On branch feature/dev_early_enkf Changes to be committed: modified: ush/forecast_predet.sh Refs: NOAA-EMC#1021

XianwuXue-NOAA · 2022-10-22T01:17:48Z

I have fix the lint error.

XianwuXue-NOAA · 2022-10-24T22:32:36Z

@WalterKolczynski-NOAA @TravisElless-NOAA @CatherineThomas-NOAA @aerorahul Do you have any comments about this PR? If you have, please let me know. I will merge the latest develop branch and then do the final test.

WalterKolczynski-NOAA · 2022-10-26T15:28:21Z

@XianwuXue-NOAA If the DA group has signed off on the results, the only thing I'm waiting on is removal of the dead code.

XianwuXue-NOAA · 2022-10-26T15:31:55Z

@WalterKolczynski-NOAA Thanks. I will remove them today after the meeting this afternoon.

On branch feature/dev_early_enkf Changes to be committed: modified: workflow/rocoto/workflow_tasks.py Refs: NOAA-EMC#1021

On branch feature/dev_early_enkf Changes to be committed: modified: ush/calcanl_gfs.py Refs: NOAA-EMC#1021

XianwuXue-NOAA · 2022-10-26T18:15:06Z

@WalterKolczynski-NOAA I have merged the latest develop branch into my branch and also delete the dead codes. Travis will triple check the changes and results.

WalterKolczynski-NOAA · 2022-10-31T09:09:37Z

Waiting on @CatherineThomas-NOAA to give final approval before merging.

TravisElless-NOAA

Tests show the late cycle reproduces the develop branch results and the early cycle points to the appropriate directories. From my perspective these changes should be good to go.

CatherineThomas-NOAA

Code looks good to me. Thanks @XianwuXue-NOAA and @TravisElless-NOAA for testing. Approved.

XianwuXue-NOAA · 2022-10-31T14:10:04Z

@TravisElless-NOAA @CatherineThomas-NOAA and @WalterKolczynski-NOAA Thanks a lot for your approval.

* develop: Correct issue in linking final restart files (NOAA-EMC#1285) Remove execute permissions from config files (NOAA-EMC#1281) Make needed updates to run forecast from GEFS (NOAA-EMC#1203) Remove unnecessary variables which reference to nemsio (NOAA-EMC#1259) Create analysis files for early-cycle EnKF by default (NOAA-EMC#1237) Don't wipe $DATA before running ocean bmat (NOAA-EMC#1280) More marine DA j-jobs (NOAA-EMC#1270) Update UFS-DA atmospheric prep script to be consistent with GDASApp update (NOAA-EMC#1265) Add new jjob for ocean analysis bmat (NOAA-EMC#1239) Retire ecf/versions in develop (NOAA-EMC#1267) Deploy documentation to RTD (NOAA-EMC#1264) Temporarily disable failing pytest (NOAA-EMC#1263) Remove incorrect/misleading comments in config.base (NOAA-EMC#1261) Add initial Sphinx documentation (NOAA-EMC#1258) Remove nemsio support (NOAA-EMC#1255) Increase wallclock for diag jobs (NOAA-EMC#1216) Use correct resources for GFS gempak (NOAA-EMC#1214) Abstract common j-job tasks (NOAA-EMC#1230) Add missing mkgfsawps.x link (NOAA-EMC#1218) Fix post sounding job (NOAA-EMC#1212) Revert "Use fracoro data for all new UFS applications (NOAA-EMC#1182)" (NOAA-EMC#1240) Use fracoro data for all new UFS applications (NOAA-EMC#1182) Revert "Merge GFS v16.3 operational GSI changes into develop branch. (NOAA-EMC#1158)" (NOAA-EMC#1238) Add more user defined parameters for the marine DA (NOAA-EMC#1235) Update pytests action version and run sequentially (NOAA-EMC#1236) Add utility to compare Fortran namelists (NOAA-EMC#1234) Updates for pygw (NOAA-EMC#1231) Merge GFS v16.3 operational GSI changes into develop branch. (NOAA-EMC#1158) Move member up in directory hierarchy (NOAA-EMC#1201) Enable staging ics for cycled experiments. (NOAA-EMC#1199) Add tests for configuration.py (NOAA-EMC#1192) Replace ocnanal_${CDATE}} with ${RUN}ocnanal_${cyc} (NOAA-EMC#1191) define NET and RUN in the Rocoto XML to accurately mimic the ecf in ecflow (NOAA-EMC#1193) Fix checking for restart files (NOAA-EMC#1186) Fix 'DEBUG' option in build_ufs.sh (NOAA-EMC#1188) Update archive job memory request value for R&Ds (NOAA-EMC#1183) Reorder post so all flux files are generated when running offline (NOAA-EMC#1181) Stop checking for restarts on non-GFS CDUMPs (NOAA-EMC#1179) Add missing jobids in some pre-job scripts (NOAA-EMC#1176) Remove existing directory if it exists when getic runs (NOAA-EMC#1165) Add logging decorator, test and test for yaml_file (NOAA-EMC#1178) fix coding norm check in `hosts.py` (NOAA-EMC#1174) Fix some bugs and make other changes so ctest in GDASApp works (NOAA-EMC#1172) Support for the GDASApp testing in containers (NOAA-EMC#1151) ATM 3DVAR with and without IAU (NOAA-EMC#1113) Enable checking for python norms and fix violating code (NOAA-EMC#1168) Enforce decimal math in atmos post (NOAA-EMC#1171) Update marine DA j-jobs to new format (NOAA-EMC#1149) Add utility to manipulate files en masse (NOAA-EMC#1166) add action to run pytests (NOAA-EMC#1167) Pin `differential-shellcheck` to `v3` tag (NOAA-EMC#1162) Add a task base class and basic logger (NOAA-EMC#1160) Recursively convert dict to AttrDict when making an AttrDict (NOAA-EMC#1154) move configuration.py to pygw. Use it from there. return AttrDict after sourcing configs (NOAA-EMC#1153) JEDI based Marine DA tasks (NOAA-EMC#1134) Allow customizations based on user/configuration (NOAA-EMC#1146) First step towards making j-jobs consistent in use from ecflow and rocoto (NOAA-EMC#1120) enable APP=S2SWA on WCOSS2 (NOAA-EMC#1142) Fix typo in .shellcheckrc Remove prod_envir module load from WCOSS2 (NOAA-EMC#1138) Link staged GSI fix files instead of cloning them from gerrit (NOAA-EMC#1132) Address shellcheck warnings in env files (NOAA-EMC#1136) Adds group size and nmem for GEFS (NOAA-EMC#1127) Remove unnecessary sCDATE assignment in forecast_predet.sh (NOAA-EMC#1133) Convert archive jobs to proper j-jobs (NOAA-EMC#1115) Update C48 forecast to run with one thread (NOAA-EMC#1131) Improved error messages from atmos analysis (NOAA-EMC#1125) Update MODULEPATH for Orion (NOAA-EMC#1126) MPMD variable updates and fix (NOAA-EMC#1124) Introduce FHMAX_ENKF_GFS to extending ensemble forecast capabilities (NOAA-EMC#1122) Update R&D launcher commands for tasks and multi-prog (NOAA-EMC#1112) Correct crtm path in UFS DA atmospheric analysis scripts (NOAA-EMC#1111) Correct syntax in remaining sorc scripts (NOAA-EMC#1105) Add GSI background error covariance as an option for UFS DA variational assimilation (NOAA-EMC#1104) Add Early Cycle EnKF workflow (NOAA-EMC#1022) Correct errors with gdas and monitoring symlinks (NOAA-EMC#1101) Fixed gfs-utils links (NOAA-EMC#1099) Fix build scripts and bring into compliance (NOAA-EMC#1096) Feature/updates for gdas app (NOAA-EMC#1091) Change GLDAS USE_CFP to NO on Hera (NOAA-EMC#1094) Resource updates to support WCOSS2 (NOAA-EMC#1070) Set COMPILER in link for detect machine (NOAA-EMC#1092) gfs utils update (NOAA-EMC#1088) GFS-UTILS update for build and ush scripts (NOAA-EMC#1082) Update UFS version to 2022 Oct 19 (NOAA-EMC#1083) Use more cycledefs for task control (NOAA-EMC#1078) removing superfluous EFSOI-specific files from develop (NOAA-EMC#1079) Update UFS to Sept 9 version (NOAA-EMC#1073) Modify default file location for monitor data when using rocoto (NOAA-EMC#1065) Fix companion ocean resolution for C48 (NOAA-EMC#1066) Add trailing slash for gldas topo path (NOAA-EMC#1064) Limit number of CPU for post (NOAA-EMC#1061) Fix eupd trace (NOAA-EMC#1057) Port to S4 (NOAA-EMC#1023) Update to obsproc.v1.0.2 and prepobs.v1.0.1 (NOAA-EMC#1049) Add GDAS to the partial build list (NOAA-EMC#1050) Fix group number being treated as octal in gdas arch (NOAA-EMC#1053) Remove trace from link script (NOAA-EMC#1046) Update gfs-utils hash to 3a609ea (NOAA-EMC#1048) Fix link script usage statement (NOAA-EMC#1045) Replace preamble variable commands with functions (NOAA-EMC#1012) Implement fix reorg and remove gfs-utils code (NOAA-EMC#1009) Rename post scripts (NOAA-EMC#1038) Fix missing @ symbol with COMINsyn in config.base (NOAA-EMC#1039) WCOSS2 run support and script/config updates (NOAA-EMC#1030) Remove base_svn from Hera and Orion hosts files (NOAA-EMC#1036) initial commit for incoming yaml work (NOAA-EMC#1029) Fix radiance verification failing to find diag files (NOAA-EMC#1031) Supported resolutions on platforms and defaults for mode (NOAA-EMC#1026) Add GLDAS scripts & fix GLDAS job (NOAA-EMC#1018) Update GSI Monitor for radmon fix Correct shell linter config (NOAA-EMC#1013) Correct diagnostic file handling in ush/ozn_xtrct.sh (NOAA-EMC#1016) Add shell linter Github action for pull requests (NOAA-EMC#1007) Build updates for WCOSS2 (NOAA-EMC#1002) Update UFS_UTILS tag to `ufs_utils_1_8_0` (NOAA-EMC#1001) Fix preamble id (NOAA-EMC#996) Add missing "atmos" into job dependencies (NOAA-EMC#998) Bugfix in arch.sh to remove hardwired "htar" (NOAA-EMC#992) Add in stubs for aerosol DA tasks + bugfix for setup_expt where cycled and ATMA are used (NOAA-EMC#990) Add GSI monitor scripts (NOAA-EMC#969) Fix product generation at some fcst hrs (NOAA-EMC#988) Add initial config files for global aerosol DA (NOAA-EMC#986) Update diag table to remove wav-ocn coupling fields (NOAA-EMC#979) use a robust Findwgrib2.cmake to find wgrib2 built w/ native wgrib2 build (NOAA-EMC#970) Externals.cfg was stale and had drifted off (NOAA-EMC#965) Fix post comparison with zero-padded numbers (NOAA-EMC#964)

Fix a bug to exclude the unsued tasks for cycled gfs

fe6f07f

On branch feature/dev_early_enkf modified: workflow/applications.py Refs: NOAA-EMC#1021

XianwuXue-NOAA added the bug Something isn't working label Sep 19, 2022

XianwuXue-NOAA requested review from aerorahul and WalterKolczynski-NOAA September 19, 2022 13:26

WalterKolczynski-NOAA requested changes Sep 20, 2022

View reviewed changes

workflow/applications.py Show resolved Hide resolved

XianwuXue-NOAA added 3 commits September 25, 2022 22:09

Merge branch 'develop' into feature/dev_early_enkf

39ad8dd

Modify to add task for cycled gfs

fa542cb

On branch feature/dev_early_enkf modified: workflow/applications.py modified: workflow/rocoto/workflow_tasks.py Refs: NOAA-EMC#1021

Merge branch 'develop' into feature/dev_early_enkf

c5bcede

WalterKolczynski-NOAA requested changes Sep 27, 2022

View reviewed changes

XianwuXue-NOAA added 3 commits September 28, 2022 10:29

Remove echgres for cycled gfs

f4b0732

On branch feature/dev_early_enkf modified: workflow/applications.py Refs: NOAA-EMC#1021

Change eupd_cdump to cdump and change hard coded gdas to cdump

837931d

for cycled gfs On branch feature/dev_early_enkf modified: workflow/rocoto/workflow_tasks.py Refs: NOAA-EMC#1021

Change back for echgres funtion because cycled gfs doesn't run it

05b362a

On branch feature/dev_early_enkf modified: rocoto/workflow_tasks.py Refs: NOAA-EMC#1021

XianwuXue-NOAA added 3 commits October 5, 2022 22:39

Merge branch 'develop' into feature/dev_early_enkf

3d43242

Merge branch 'develop' into feature/dev_early_enkf

0a29772

Improve to run early enkf

201fa98

On branch feature/dev_early_enkf Changes to be committed: modified: jobs/JGDAS_ENKF_ECEN modified: jobs/JGDAS_ENKF_SFC modified: parm/config/config.anal modified: ush/calcanl_gfs.py Refs: NOAA-EMC#1021

XianwuXue-NOAA changed the title ~~Fix a bug to avoid adding the unusued tasks for cycled gfs~~ Add Early Cycle EnKF workflow Oct 11, 2022

XianwuXue-NOAA commented Oct 11, 2022

View reviewed changes

ush/calcanl_gfs.py Outdated Show resolved Hide resolved

WalterKolczynski-NOAA requested changes Oct 11, 2022

View reviewed changes

Merge branch 'develop' into feature/dev_early_enkf

3bc3245

XianwuXue-NOAA requested a review from CatherineThomas-NOAA October 12, 2022 18:09

XianwuXue-NOAA added feature New feature or request and removed bug Something isn't working labels Oct 12, 2022

XianwuXue-NOAA added 2 commits October 18, 2022 17:40

Merge branch 'develop' into feature/dev_early_enkf

830b374

Delete the unused statements

cc81ee2

On branch feature/dev_early_enkf Changes to be committed: modified: workflow/rocoto/workflow_tasks.py Refs: NOAA-EMC#1021

XianwuXue-NOAA linked an issue Oct 20, 2022 that may be closed by this pull request

Add Early Cycle EnKF #1021

Closed

TravisElless-NOAA reviewed Oct 20, 2022

View reviewed changes

XianwuXue-NOAA added 2 commits October 20, 2022 19:03

Update to GDUMP to point the early-cycle to the correct directory

ce13ea9

Changes to be committed: modified: jobs/JGDAS_ENKF_SFC Refs: NOAA-EMC#1021

Modify to run gfsefcsN for early cycle enkf

33a1f0d

Changes to be committed: modified: jobs/JGDAS_ENKF_FCST modified: ush/forecast_predet.sh Refs: NOAA-EMC#1021

github-advanced-security bot found potential problems Oct 20, 2022

View reviewed changes

ush/forecast_predet.sh Fixed Show fixed Hide fixed

Merge branch 'develop' into feature/dev_early_enkf

554d1a5

XianwuXue-NOAA added 2 commits October 21, 2022 21:09

To fix lint error

ed556b8

On branch feature/dev_early_enkf Changes to be committed: modified: ush/forecast_predet.sh Refs: NOAA-EMC#1021

To fix the lint error Error: SHELLCHECK_WARNING

8eef956

Prefer [ p ] && [ q ] as [ p -a q ] is not well defined. On branch feature/dev_early_enkf Changes to be committed: modified: ush/forecast_predet.sh Refs: NOAA-EMC#1021

Merge branch 'develop' into feature/dev_early_enkf

1e31cf7

XianwuXue-NOAA added 2 commits October 26, 2022 12:15

Remove the dead code

2e58e0a

On branch feature/dev_early_enkf Changes to be committed: modified: workflow/rocoto/workflow_tasks.py Refs: NOAA-EMC#1021

Delete the unused codes

3db80e3

On branch feature/dev_early_enkf Changes to be committed: modified: ush/calcanl_gfs.py Refs: NOAA-EMC#1021

WalterKolczynski-NOAA approved these changes Oct 26, 2022

View reviewed changes

Merge branch 'develop' into feature/dev_early_enkf

2f089d7

TravisElless-NOAA approved these changes Oct 31, 2022

View reviewed changes

CatherineThomas-NOAA approved these changes Oct 31, 2022

View reviewed changes

WalterKolczynski-NOAA merged commit c5fc2b7 into NOAA-EMC:develop Oct 31, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add Early Cycle EnKF workflow #1022

Add Early Cycle EnKF workflow #1022

XianwuXue-NOAA commented Sep 19, 2022 •

edited by WalterKolczynski-NOAA

Loading

WalterKolczynski-NOAA Sep 27, 2022

XianwuXue-NOAA Sep 27, 2022

WalterKolczynski-NOAA commented Oct 4, 2022

XianwuXue-NOAA commented Oct 4, 2022

XianwuXue-NOAA commented Oct 11, 2022

WalterKolczynski-NOAA left a comment

XianwuXue-NOAA commented Oct 12, 2022

TravisElless-NOAA Oct 20, 2022

XianwuXue-NOAA commented Oct 21, 2022

XianwuXue-NOAA commented Oct 22, 2022

XianwuXue-NOAA commented Oct 24, 2022

WalterKolczynski-NOAA commented Oct 26, 2022

XianwuXue-NOAA commented Oct 26, 2022

XianwuXue-NOAA commented Oct 26, 2022

WalterKolczynski-NOAA commented Oct 31, 2022

TravisElless-NOAA left a comment

CatherineThomas-NOAA left a comment

XianwuXue-NOAA commented Oct 31, 2022

		@@ -97,7 +97,7 @@ export COMIN="$ROTDIR/$CDUMP.$PDY/$cyc/$COMPONENT"
		export COMIN_GES="$ROTDIR/$CDUMP.$gPDY/$gcyc/$COMPONENT"

Add Early Cycle EnKF workflow #1022

Add Early Cycle EnKF workflow #1022

Conversation

XianwuXue-NOAA commented Sep 19, 2022 • edited by WalterKolczynski-NOAA Loading

WalterKolczynski-NOAA Sep 27, 2022

Choose a reason for hiding this comment

XianwuXue-NOAA Sep 27, 2022

Choose a reason for hiding this comment

WalterKolczynski-NOAA commented Oct 4, 2022

XianwuXue-NOAA commented Oct 4, 2022

XianwuXue-NOAA commented Oct 11, 2022

WalterKolczynski-NOAA left a comment

Choose a reason for hiding this comment

XianwuXue-NOAA commented Oct 12, 2022

TravisElless-NOAA Oct 20, 2022

Choose a reason for hiding this comment

XianwuXue-NOAA commented Oct 21, 2022

XianwuXue-NOAA commented Oct 22, 2022

XianwuXue-NOAA commented Oct 24, 2022

WalterKolczynski-NOAA commented Oct 26, 2022

XianwuXue-NOAA commented Oct 26, 2022

XianwuXue-NOAA commented Oct 26, 2022

WalterKolczynski-NOAA commented Oct 31, 2022

TravisElless-NOAA left a comment

Choose a reason for hiding this comment

CatherineThomas-NOAA left a comment

Choose a reason for hiding this comment

XianwuXue-NOAA commented Oct 31, 2022

XianwuXue-NOAA commented Sep 19, 2022 •

edited by WalterKolczynski-NOAA

Loading