Update submodule tags to pass runoff from CISM to ROF as per issue #2590 #2605

slevis-lmwg · 2024-06-20T23:46:37Z

Description of changes

Details in #2590

Specific notes

Contributors other than yourself, if any:
@ekluzek @mvertens @jedwards4b @billsacks @Katetc

CTSM Issues Fixed (include github issue #):
Fixes #2590

Are answers expected to change (and if so in what way)?
Yes, due to change in default behavior

Any User Interface Changes (namelist or namelist defaults changes)?
Default behavior has changed as explained in #2590

Does this create a need to change or add documentation? Did you do so?
Probably, but I did not.

.gitmodules

slevis-lmwg · 2024-07-01T22:33:31Z

From the TODO list in the issue
Most of aux_clm and ctsm_sci is failing for now.

UPDATE: Trying aux_clm on izumi
FAIL, so try bisecting between the latest tags that I used versus the branch hashes that I pointed to when the tests passed

slevis-lmwg · 2024-07-02T23:11:34Z

Bisecting between my passing and failing aux_clm test-suites:

no code diffs between rtm1_0_80 and 420c5979d8abf05184992fdfba5ac41c41c91c2e
no code diffs between mosart1.1.02 and ca0acc0336365456bec653c4c233abb57d9bc62d
large code diffs between cmeps0.14.67 and 4dca7c602e54189b74b1d0195c433d1b83475553
Here ESCOMP/CMEPS@f960f3e I see that I may have had to have used cmeps0.14.77
@jedwards4b I should have checked with you which tag to use, for this one and the next one. I have resubmitted aux_clm on izumi already, so I hope to know by tomorrow, but pls let me know if you think that I'm still off.
ccs_config now I see here ESMCI/ccs_config_cesm@4e337b5 that I may have had to have used ccs_config_cesm1.0.0
cism should be fine because I asked @Katetc specifically which tag to use

slevis-lmwg · 2024-07-03T17:15:27Z

aux_clm on izumi now OK
./cs.status.fails | grep -v 'otherwise bit-for-b' | grep -v NLCOMP | grep -v 'EXPECTED FAILURE' | grep -v PASS

on derecho
aux_clm tests_0703-123700de
./cs.status.fails | grep -v 'wise bit-for-bit' | grep -v NLCOMP | grep -v 'EXPECTED FAILURE' | grep -v PASS

    FAIL ERS_Ly3_P64x2.f10_f10_mg37.IHistClm50BgcCropG.derecho_intel.clm-cropMonthOutput BASELINE cismtorof.n01_ctsm5.2.007: DIFF
    FAIL SMS_Lm37.f10_f10_mg37.I1850Clm50SpG.derecho_intel.clm-glcMEC_long BASELINE cismtorof.n01_ctsm5.2.007: DIFF

In the first of these two, the largest NORMALIZED DIFF is
ERS_Ly3_P64x2.f10_f10_mg37.IHistClm50BgcCropG.derecho_intel.clm-cropMonthOutput.C.0703-123700de_int.clm2.h0.1852-12.nc.cprnc.out: RMS ERRH2OSNO 2.5269E-17 NORMALIZED 7.0110E-02
And in the latter, the largest NORMALIZED DIFF is
SMS_Lm37.f10_f10_mg37.I1850Clm50SpG.derecho_intel.clm-glcMEC_long.C.0703-123700de_int.cism.gris.h.0004-01-01-00000.nc.cprnc.out: RMS calving_rate 2.2858E-14 NORMALIZED 7.6445E-12
I interpret both as roundoff diffs that have grown over time.

ctsm_sci tests_0703-123641de
./cs.status.fails |grep -v NLCOMP|grep -v 'EXPECTED FAILURE'|grep -v 'wise bit-for-bit'

    FAIL ERP_D_Ld5.f09_g17.I2000Clm50Vic.derecho_intel.clm-vrtlay BASELINE ctsm_sci-ctsm5.2.007: DIFF
    FAIL ERP_Ld5.f09_g17.I2000Clm50Vic.derecho_intel.clm-vrtlay BASELINE ctsm_sci-ctsm5.2.007: DIFF
    FAIL SMS_D_Ld3_PS.f09_g17.I1850Clm60SpNoAnthro.derecho_intel.clm-decStart1851_noinitial BASELINE ctsm_sci-ctsm5.2.007: DIFF
    FAIL SMS_Ld1.f19_g17.I2000Clm50Vic.derecho_intel.clm-default BASELINE ctsm_sci-ctsm5.2.007: DIFF
    FAIL SMS_Ld3_PS.f09_g17.I1850Clm60BgcNoAnthro.derecho_intel.clm-decStart1851_noinitial BASELINE ctsm_sci-ctsm5.2.007: DIFF

In the last of these, I see larger diffs including in variables that I would not have expected initially. This is the only Bgc case here. Should I expect changes in CH4 variables as a result of the changes in river flow @swensosc?

SMS_Ld3_PS.f09_g17.I1850Clm60BgcNoAnthro.derecho_intel.clm-decStart1851_noinitial.C.0703-123641de_int.clm2.h0.1852-01-02-00000.nc.cprnc.out: RMS CH4PROD                          1.4435E-12            NORMALIZED  7.6136E-03
SMS_Ld3_PS.f09_g17.I1850Clm60BgcNoAnthro.derecho_intel.clm-decStart1851_noinitial.C.0703-123641de_int.clm2.h0.1852-01-02-00000.nc.cprnc.out: RMS CH4_SURF_AERE_SAT                6.8893E-14            NORMALIZED  9.1275E-02
SMS_Ld3_PS.f09_g17.I1850Clm60BgcNoAnthro.derecho_intel.clm-decStart1851_noinitial.C.0703-123641de_int.clm2.h0.1852-01-02-00000.nc.cprnc.out: RMS CH4_SURF_DIFF_SAT                5.9901E-12            NORMALIZED  4.6836E-02
SMS_Ld3_PS.f09_g17.I1850Clm60BgcNoAnthro.derecho_intel.clm-decStart1851_noinitial.C.0703-123641de_int.clm2.h0.1852-01-02-00000.nc.cprnc.out: RMS CH4_SURF_EBUL_SAT                1.5077E-11            NORMALIZED  5.6659E-02
SMS_Ld3_PS.f09_g17.I1850Clm60BgcNoAnthro.derecho_intel.clm-decStart1851_noinitial.C.0703-123641de_int.clm2.h0.1852-01-02-00000.nc.cprnc.out: RMS FCH4                             3.5318E-16            NORMALIZED  2.9839E-04
SMS_Ld3_PS.f09_g17.I1850Clm60BgcNoAnthro.derecho_intel.clm-decStart1851_noinitial.C.0703-123641de_int.clm2.h0.1852-01-02-00000.nc.cprnc.out: RMS FCH4TOCO2                        2.1029E-13            NORMALIZED  3.5289E-04
SMS_Ld3_PS.f09_g17.I1850Clm60BgcNoAnthro.derecho_intel.clm-decStart1851_noinitial.C.0703-123641de_int.clm2.h0.1852-01-02-00000.nc.cprnc.out: RMS FCH4_DFSAT                       3.5660E-16            NORMALIZED  5.2060E+00
SMS_Ld3_PS.f09_g17.I1850Clm60BgcNoAnthro.derecho_intel.clm-decStart1851_noinitial.C.0703-123641de_int.clm2.h0.1852-01-02-00000.nc.cprnc.out: RMS FINUNDATED                       6.8805E-05            NORMALIZED  8.7284E-03
SMS_Ld3_PS.f09_g17.I1850Clm60BgcNoAnthro.derecho_intel.clm-decStart1851_noinitial.C.0703-123641de_int.clm2.h0.1852-01-02-00000.nc.cprnc.out: RMS NEM                              1.3188E-12            NORMALIZED  2.6262E-03
SMS_Ld3_PS.f09_g17.I1850Clm60BgcNoAnthro.derecho_intel.clm-decStart1851_noinitial.C.0703-123641de_int.clm2.h0.1852-01-02-00000.nc.cprnc.out: RMS TOTCOLCH4                        8.2472E-08            NORMALIZED  2.3369E-04
SMS_Ld3_PS.f09_g17.I1850Clm60BgcNoAnthro.derecho_intel.clm-decStart1851_noinitial.C.0703-123641de_int.clm2.h0.1852-01-02-00000.nc.cprnc.out: RMS TWS                              1.5401E+00            NORMALIZED  1.0340E-04
SMS_Ld3_PS.f09_g17.I1850Clm60BgcNoAnthro.derecho_intel.clm-decStart1851_noinitial.C.0703-123641de_int.clm2.h0.1852-01-02-00000.nc.cprnc.out: RMS VOLR                             2.0453E+07            NORMALIZED  2.1061E-01
SMS_Ld3_PS.f09_g17.I1850Clm60BgcNoAnthro.derecho_intel.clm-decStart1851_noinitial.C.0703-123641de_int.clm2.h0.1852-01-02-00000.nc.cprnc.out: RMS VOLRMCH                          2.0453E+07            NORMALIZED  2.1061E-01
SMS_Ld3_PS.f09_g17.I1850Clm60BgcNoAnthro.derecho_intel.clm-decStart1851_noinitial.C.0703-123641de_int.clm2.h0.1852-01-02-00000.nc.cprnc.out: RMS CONC_O2_SAT                      4.8355E-06            NORMALIZED  3.8462E-03
SMS_Ld3_PS.f09_g17.I1850Clm60BgcNoAnthro.derecho_intel.clm-decStart1851_noinitial.C.0703-123641de_int.cpl.hi.1852-01-02-00000.nc.cprnc.out: RMS lndExp_Flrr_volr                 1.0665E-03            NORMALIZED  4.0627E-01
SMS_Ld3_PS.f09_g17.I1850Clm60BgcNoAnthro.derecho_intel.clm-decStart1851_noinitial.C.0703-123641de_int.cpl.hi.1852-01-02-00000.nc.cprnc.out: RMS lndExp_Flrr_volrmch              1.0665E-03            NORMALIZED  4.0627E-01
SMS_Ld3_PS.f09_g17.I1850Clm60BgcNoAnthro.derecho_intel.clm-decStart1851_noinitial.C.0703-123641de_int.cpl.hi.1852-01-02-00000.nc.cprnc.out: RMS rofImp_Flrr_volr                 1.5704E-03            NORMALIZED  5.9480E-01
SMS_Ld3_PS.f09_g17.I1850Clm60BgcNoAnthro.derecho_intel.clm-decStart1851_noinitial.C.0703-123641de_int.cpl.hi.1852-01-02-00000.nc.cprnc.out: RMS rofImp_Flrr_volrmch              1.5704E-03            NORMALIZED  5.9480E-01
SMS_Ld3_PS.f09_g17.I1850Clm60BgcNoAnthro.derecho_intel.clm-decStart1851_noinitial.C.0703-123641de_int.cpl.hi.1852-01-02-00000.nc.cprnc.out: RMS rofImp_Forr_rofl                 7.2396E-06            NORMALIZED  1.0973E+01
SMS_Ld3_PS.f09_g17.I1850Clm60BgcNoAnthro.derecho_intel.clm-decStart1851_noinitial.C.0703-123641de_int.rtm.h0.1851-12.nc.cprnc.out: RMS QCHANR                           3.5056E+01            NORMALIZED  3.3240E-01
SMS_Ld3_PS.f09_g17.I1850Clm60BgcNoAnthro.derecho_intel.clm-decStart1851_noinitial.C.0703-123641de_int.rtm.h0.1851-12.nc.cprnc.out: RMS QCHOCNR                          7.3298E+00            NORMALIZED  2.3645E+00

slevis-lmwg · 2024-07-06T16:41:11Z

@swensosc verified by email that these diffs seem reasonable. Sean wrote:
I think you would see changes in methane due to the fact that the riley et al. method for calculating inundation uses a regression against TWS to determine the inundated fraction, which is an input to the methane model. So if VOLR is changed, TWS would be changed, and thus methane would see a perturbed inundated area.

Merging ctsm5.2.014

slevis-lmwg · 2024-07-19T19:24:57Z

testing on derecho
PASS ./build-namelist_test.pl (still passes after the fix mentioned below)
PASS make black and make lint (still passes after the fix mentioned below)
PASS ./run_ctsm_py_tests -u and ./run_ctsm_py_tests -s (still passes after the fix mentioned below)

./run_sys_tests -s aux_clm -c ctsm5.2.014 -g ctsm5.2.015
OK on derecho
FAIL 1 test on izumi that I have fixed.

I will rerun aux_clm on derecho and izumi to be safe.

slevis-lmwg · 2024-07-19T19:40:18Z

@jedwards4b I updated to ctsm5.2.014 and submitted tests as posted above. If all goes smoothly, I should merge this PR next.

First I wanted your input on something that seems concerning to me about ccs_config:
./bin/git-fleximod update (with or without --force) returns...

s  ccs_config ccs_config_cesm0.0.113 69a958581ecd2d32ee9cb1c38bcd3847b8b920bf is out of sync with .gitmodules ccs_config_cesm1.0.0

ccs_config updated to ccs_config_cesm1.0.0

and then status returns...

s  ccs_config ccs_config_cesm0.0.113 69a958581ecd2d32ee9cb1c38bcd3847b8b920bf is out of sync with .gitmodules ccs_config_cesm1.0.0

and git describe returns...
ccs_config_cesm0.0.113
while .gitmodules says...
fxtag = ccs_config_cesm1.0.0
What do you think?

slevis-lmwg · 2024-07-19T22:32:12Z

@jedwards4b
I tried git diff ccs_config_cesm1.0.0 ccs_config_cesm0.0.113
and got no difference, so I will assume that ccs_config is acceptable for the purposes of this PR's testing.

ERS_D.f19_g17.I1850Clm50BgcCrop.izumi_nag.clm-ciso_monthly_matrixcn_spinup failed during the build because a namelist variable needed renaming

slevis-lmwg · 2024-07-22T16:01:05Z

Thank you @jedwards4b

aux_clm results
./cs.status.fails | grep -v 'ise bit-for-b' | grep -v NLCOMP|grep -v PASS | grep -v '14: DIF'
OK izumi
OK derecho
Last TODO before merge:
I want to double check the nature of the DIFFs from baseline.

slevis-lmwg · 2024-07-22T18:30:27Z

These differences seem out of the ordinary:

SMS_D.1x1_brazil.I2000Clm60FatesSpCruRsGs.derecho_gnu.clm-FatesColdDryDepSatPhen.GC.0719-175606de_gnu/SMS_D.1x1_brazil.I2000Clm60FatesSpCruRsGs.derecho_gnu.clm-FatesColdDryDepSatPhen.GC.0719-175606de_gnu.clm2.h0.2000-01-01-00000.nc.cprnc.out: RMS FATES_TRANSITION_MATRIX_LULU       Infinity            NORMALIZED    Infinity
SMS_D.1x1_brazil.I2000Clm60FatesSpCruRsGs.derecho_gnu.clm-FatesColdMeganSatPhen.GC.0719-175606de_gnu/SMS_D.1x1_brazil.I2000Clm60FatesSpCruRsGs.derecho_gnu.clm-FatesColdMeganSatPhen.GC.0719-175606de_gnu.clm2.h0.2000-01-01-00000.nc.cprnc.out: RMS FATES_TRANSITION_MATRIX_LULU       Infinity            NORMALIZED    Infinity

@glemieux are these new tests that came with your recent PR? Does this concern you?

glemieux · 2024-07-22T20:52:42Z

These differences seem out of the ordinary:

SMS_D.1x1_brazil.I2000Clm60FatesSpCruRsGs.derecho_gnu.clm-FatesColdDryDepSatPhen.GC.0719-175606de_gnu/SMS_D.1x1_brazil.I2000Clm60FatesSpCruRsGs.derecho_gnu.clm-FatesColdDryDepSatPhen.GC.0719-175606de_gnu.clm2.h0.2000-01-01-00000.nc.cprnc.out: RMS FATES_TRANSITION_MATRIX_LULU       Infinity            NORMALIZED    Infinity
SMS_D.1x1_brazil.I2000Clm60FatesSpCruRsGs.derecho_gnu.clm-FatesColdMeganSatPhen.GC.0719-175606de_gnu/SMS_D.1x1_brazil.I2000Clm60FatesSpCruRsGs.derecho_gnu.clm-FatesColdMeganSatPhen.GC.0719-175606de_gnu.clm2.h0.2000-01-01-00000.nc.cprnc.out: RMS FATES_TRANSITION_MATRIX_LULU       Infinity            NORMALIZED    Infinity

@glemieux are these new tests that came with your recent PR? Does this concern you?

@slevis-lmwg can you point me to the test folder? I'd like to check out the cprnc output.

slevis-lmwg · 2024-07-22T20:57:05Z

@glemieux please look here
/glade/work/slevis/git_externals/cism2mosart/tests_0719-175606de

jedwards4b · 2024-07-22T21:03:01Z

There is a permissions problem for your baseline directory.
ls: cannot open directory '/glade/campaign/cgd/tss/ctsm_baselines/ctsm5.2.014/': Permission denied

slevis-lmwg · 2024-07-22T21:07:33Z

@jedwards4b I updated permissions for that one and will go through and update for my other ones, too. LMK if you still have trouble accessing.

jedwards4b · 2024-07-22T21:15:04Z

Thank you for fixing that - the values for the field FATES_TRANSITION_MATRIX_LULU in both the baseline and the case look like garbage to me. Can someone confirm these values are expected?

slevis-lmwg · 2024-07-22T21:17:36Z

@glemieux is taking a look.

glemieux · 2024-07-22T21:26:15Z

Yep, those cases are definitely garbage, but they've been that way for these two test since that variable was introduced in ctsm5.2.013. I missed this during testing. Those values should be zero'd out. I'm not sure why the FatesColdSatPhen correctly zeros FATES_TRANSITION_MATRIX_LULU, but FatesColdDryDepSatPhen and FatesColdMeganSatPhen does not.

I'd say this is ok as is to go forward, but an issue should be created to fix this as it will likely flag as a DIFF again in the future given the randomness of the garbage.

@slevis-lmwg and @jedwards4b how does that sound to y'all?

slevis-lmwg · 2024-07-22T21:29:05Z

I'm fine with that. I can open the issue and you can add to it as you see fit @glemieux
Thanks for looking into this.

Temporary .gitmodules points to the branches of the modules that changed

d3e6814

slevis-lmwg self-assigned this Jun 20, 2024

slevis-lmwg added priority: high High priority to fix/merge soon, e.g., because it is a problem in important configurations tag: enh - new science labels Jun 20, 2024

slevis-lmwg added 2 commits June 21, 2024 16:34

Update .gitmodules with latest mosart commit

540d87f

Update .gitmodules with new mosart and rtm tags

596dc13

slevis-lmwg changed the base branch from master to tmp-240620 June 24, 2024 17:32

slevis-lmwg changed the base branch from tmp-240620 to master July 1, 2024 17:08

Update .gitmodules with new cismwrapper, cmeps, and ccs_config tags

6dd55ee

slevis-lmwg commented Jul 1, 2024

View reviewed changes

.gitmodules Outdated Show resolved Hide resolved

slevis-lmwg added 2 commits July 1, 2024 12:59

Update cismwrapper tag to the correct new one

fb95295

Ran ./bin/git-fleximod update, so now update the cism module itself

de1b44c

Correct the tags that I used for ccs_config and cmeps

e1b2bda

slevis-lmwg mentioned this pull request Jul 5, 2024

Update CMEPS/MOSART/CISM/RTM tags #2590

Closed

ekluzek changed the title ~~Update submodule tags as per issue #2590~~ Update submodule tags to pass runoff from CISM to ROF as per issue #2590 Jul 11, 2024

slevis-lmwg added the PR status: ready PR: this is ready to merge in, with all tests satisfactory and reviews complete label Jul 11, 2024

Merge remote-tracking branch 'escomp/master' into update_submodules

0524efa

Merging ctsm5.2.014

slevis-lmwg and others added 2 commits July 19, 2024 17:19

Namelist corrections after an izumi test failed

63b3c3f

ERS_D.f19_g17.I1850Clm50BgcCrop.izumi_nag.clm-ciso_monthly_matrixcn_spinup failed during the build because a namelist variable needed renaming

Draft ChangeLog/ChangeSum (comments welcome)

a32a9eb

slevis-lmwg requested a review from jedwards4b July 20, 2024 00:58

jedwards4b approved these changes Jul 22, 2024

View reviewed changes

slevis-lmwg mentioned this pull request Jul 22, 2024

History variable FATES_TRANSITION_MATRIX_LULU contains garbage for FatesColdDryDepSatPhen and FatesColdMeganSatPhen #2656

Closed

Update ChangeLog/ChangeSum

a4a9ede

slevis-lmwg merged commit a5ef062 into ESCOMP:master Jul 22, 2024
2 checks passed

slevis-lmwg deleted the update_submodules branch July 22, 2024 22:06

samsrabin added enhancement new capability or improved behavior of existing capability science Enhancement to or bug impacting science labels Aug 8, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update submodule tags to pass runoff from CISM to ROF as per issue #2590 #2605

Update submodule tags to pass runoff from CISM to ROF as per issue #2590 #2605

slevis-lmwg commented Jun 20, 2024 •

edited

Loading

slevis-lmwg commented Jul 1, 2024 •

edited

Loading

slevis-lmwg commented Jul 2, 2024 •

edited

Loading

slevis-lmwg commented Jul 3, 2024 •

edited

Loading

slevis-lmwg commented Jul 6, 2024

slevis-lmwg commented Jul 19, 2024 •

edited

Loading

slevis-lmwg commented Jul 19, 2024 •

edited

Loading

slevis-lmwg commented Jul 19, 2024

slevis-lmwg commented Jul 22, 2024

slevis-lmwg commented Jul 22, 2024

glemieux commented Jul 22, 2024

slevis-lmwg commented Jul 22, 2024

jedwards4b commented Jul 22, 2024

slevis-lmwg commented Jul 22, 2024

jedwards4b commented Jul 22, 2024

slevis-lmwg commented Jul 22, 2024

glemieux commented Jul 22, 2024 •

edited

Loading

slevis-lmwg commented Jul 22, 2024

Update submodule tags to pass runoff from CISM to ROF as per issue #2590 #2605

Update submodule tags to pass runoff from CISM to ROF as per issue #2590 #2605

Conversation

slevis-lmwg commented Jun 20, 2024 • edited Loading

Description of changes

Specific notes

slevis-lmwg commented Jul 1, 2024 • edited Loading

slevis-lmwg commented Jul 2, 2024 • edited Loading

slevis-lmwg commented Jul 3, 2024 • edited Loading

slevis-lmwg commented Jul 6, 2024

slevis-lmwg commented Jul 19, 2024 • edited Loading

slevis-lmwg commented Jul 19, 2024 • edited Loading

slevis-lmwg commented Jul 19, 2024

slevis-lmwg commented Jul 22, 2024

slevis-lmwg commented Jul 22, 2024

glemieux commented Jul 22, 2024

slevis-lmwg commented Jul 22, 2024

jedwards4b commented Jul 22, 2024

slevis-lmwg commented Jul 22, 2024

jedwards4b commented Jul 22, 2024

slevis-lmwg commented Jul 22, 2024

glemieux commented Jul 22, 2024 • edited Loading

slevis-lmwg commented Jul 22, 2024

slevis-lmwg commented Jun 20, 2024 •

edited

Loading

slevis-lmwg commented Jul 1, 2024 •

edited

Loading

slevis-lmwg commented Jul 2, 2024 •

edited

Loading

slevis-lmwg commented Jul 3, 2024 •

edited

Loading

slevis-lmwg commented Jul 19, 2024 •

edited

Loading

slevis-lmwg commented Jul 19, 2024 •

edited

Loading

glemieux commented Jul 22, 2024 •

edited

Loading