MatrixCN in ctsm5.2.001 (ctsm52001_[f09,f19,ne30]_GSWP3_bgccrop_[1850AD, 1850SASU, 1850pSASU, hist, 2000pSASU])

[slevis-lmwg]

Description:
New spin-ups of matrixCN (for previous, see #45), initially to assess whether test SMS_Lm13_PS.f19_g17.I2000Clm60BgcCrop.derecho_intel.clm-cropMonthOutput fails with negative livestemn_patch due to incompatible initial conditions in the finidat (discussed in PR 640).

---

Case directory:
Locally:
DONE ~slevis/cases_LMWG_dev/ctsm52001_[f19, f09, ne30]_GSWP3_bgccrop_[1850AD,1850SASU,1850pSASU]
DONE ~slevis/cases_LMWG_dev/ctsm52001_f19_GSWP3_bgccrop_hist
The hist simulation crashed with matrixcn (details in a post below), so I completed it with matrixcn off.
DONE ~slevis/cases_LMWG_dev/ctsm52001_f19_GSWP3_bgccrop_2000pSASU

---

Sandbox:
Locally:
/glade/work/slevis/git/cn-matrix_v3
git branch: cn-matrix_v3
git describe: ctsm5.2.001-60-gbbfe2b454

---

user_nl_ changes:
The first 3 lines fix the incorrect defaults that crept into main with ctsm5.2 (see ESCOMP/CTSM#2492).
The next 2 lines turn off matrix for the AD phase only (cold-start simulations) and are replaced with finidat in the subsequent phases (hybrid simulations).
The last 4 lines turn on isotopes in case we use this spin-up to initialize isotope simulations.

 irrigate = .true.
 snicar_snobc_intmix = .true.
 snow_thermal_cond_method = 'Sturm1997'

 use_matrixcn = .false.
 use_soil_matrixcn = .false.

 use_c13 = .true.
 use_c14 = .true.
 use_c13_timeseries = .true.
 use_c14_bombspike = .true.

Result from diff ctsm52001_f19_GSWP3_bgccrop_1850SASU/run/lnd_in ctsm52001_f19_GSWP3_bgccrop_1850AD/run/. and similar from the f09 and ne30:

<  finidat = '/glade/derecho/scratch/slevis/archive/ctsm52001_f19_GSWP3_bgccrop_1850AD/rest/0281-01-01-00000/ctsm52                                           001_f19_GSWP3_bgccrop_1850AD.clm2.r.0281-01-01-00000.nc'
---
>  finidat = ' '
32d31
<  iloop_avg = -999
39,40d37
<  nyr_forcing = 20
<  nyr_sasu = 20
54,55c51
<  spinup_matrixcn = .true.
<  spinup_state = 0
---
>  spinup_state = 2
83c79
<  use_matrixcn = .true.
---
>  use_matrixcn = .false.
87c83
<  use_soil_matrixcn = .true.
---
>  use_soil_matrixcn = .false.
316c312
<  reseed_dead_plants = .false.
---
>  reseed_dead_plants = .true.

Result from diff ctsm52001_f19_GSWP3_bgccrop_1850pSASU/run/lnd_in ctsm52001_f19_GSWP3_bgccrop_1850SASU/run/. and similar from the f09 and ne30:

15c15
<  finidat = '/glade/derecho/scratch/slevis/archive/ctsm52001_f19_GSWP3_bgccrop_1850SASU/rest/0241-01-01-00000/ctsm52001_f19_GSWP3_bgccrop_1850SASU.clm2.r.0241-01-01-00000.nc'
---
>  finidat = '/glade/derecho/scratch/slevis/archive/ctsm52001_f19_GSWP3_bgccrop_1850AD/rest/0281-01-01-00000/ctsm52001_f19_GSWP3_bgccrop_1850AD.clm2.r.0281-01-01-00000.nc'
25a26
>  hist_empty_htapes = .true.
26a28,30
>  hist_fincl1 = 'TOTECOSYSC', 'TOTECOSYSN', 'TOTSOMC', 'TOTSOMN', 'TOTVEGC', 'TOTVEGN', 'TLAI', 'GPP', 'CPOOL', 'NPP', 'TWS'
>  hist_mfilt = 20
>  hist_nhtfrq = -8760
27a32
>  iloop_avg = -999
33a39,40
>  nyr_forcing = 20
>  nyr_sasu = 20
47c54
<  spinup_matrixcn = .false.
---
>  spinup_matrixcn = .true.
308a316
>  reseed_dead_plants = .false.

Result from diff ctsm52001_f19_GSWP3_bgccrop_hist/run/lnd_in ctsm52001_f19_GSWP3_bgccrop_1850pSASU/run/. and later added use_matrixcn = .false. and use_soil_matrixcn = .false. to avoid the balance error posted below:

5c5,6
<  co2_type = 'diagnostic'
---
>  co2_ppmv = 284.7
>  co2_type = 'constant'
14c15
<  finidat = '/glade/derecho/scratch/slevis/archive/ctsm52001_f19_GSWP3_bgccrop_1850pSASU/rest/0061-01-01-00000/ctsm52001_f19_GSWP3_bgccrop_1850pSASU.clm2.r.0061-01-01-00000.nc'
---
>  finidat = '/glade/derecho/scratch/slevis/archive/ctsm52001_f19_GSWP3_bgccrop_1850SASU/rest/0241-01-01-00000/ctsm52001_f19_GSWP3_bgccrop_1850SASU.clm2.r.0241-01-01-00000.nc'
72d72
<  use_init_interp = .true.
154,160d153
<  do_grossunrep = .false.
<  do_harvest = .true.
<  do_transient_crops = .true.
<  do_transient_lakes = .true.
<  do_transient_pfts = .true.
<  do_transient_urban = .true.
<  flanduse_timeseries = '/glade/campaign/cesm/cesmdata/inputdata/lnd/clm2/surfdata_esmf/ctsm5.2.0/landuse.timeseries_1.9x2.5_SSP2-4.5_1850-2100_78pfts_c240216.nc'
171d163
<  init_interp_method = 'general'

Result from diff ctsm52001_f19_GSWP3_bgccrop_2000pSASU/user_nl_clm ctsm52001_f19_GSWP3_bgccrop_1850pSASU

20,21c20
<  use_init_interp = .true.
<  finidat = '/glade/derecho/scratch/slevis/archive/ctsm52001_f19_GSWP3_bgccrop_hist/rest/2000-01-01-00000/ctsm52001_f19_GSWP3_bgccrop_hist.clm2.r.2000-01-01-00000.nc'
---
>  finidat = '/glade/derecho/scratch/slevis/archive/ctsm52001_f19_GSWP3_bgccrop_1850SASU/rest/0241-01-01-00000/ctsm52001_f19_GSWP3_bgccrop_1850SASU.clm2.r.0241-01-01-00000.nc'

---

SourceMods:
NONE

---

Diagnostics:
Diags (if available)

https://webext.cgd.ucar.edu/I1850/$CASE/lnd/

---

Output:
Output (if still available):
/glade/derecho/scratch/slevis/archive/ctsm52001_[f19, f09, ne30]GSWP3_bgccrop_1850AD
/glade/derecho/scratch/slevis/archive/ctsm52001[f19, f09, ne30]GSWP3_bgccrop_1850SASU
/glade/derecho/scratch/slevis/archive/ctsm52001[f19, f09, ne30]GSWP3_bgccrop_1850pSASU
/glade/derecho/scratch/slevis/archive/ctsm52001_f19_GSWP3_bgccrop_hist
/glade/derecho/scratch/slevis/archive/ctsm52001[f19, f09, ne30]_GSWP3_bgccrop_2000pSASU

I copied to
/glade/campaign/cgd/tss/people/slevis/i_cases/cases_LMWG_dev with the same case-names.

---

Contacts:
@slevis-lmwg

---

Extra details:

./create_newcase --case ~/cases_LMWG_dev/ctsm52001_f19_GSWP3_bgccrop_1850AD --compset 1850_DATM%GSWP3v1_CLM60%BGC-CROP_SICE_SOCN_SGLC_SWAV_SESP --res f19_g17 --run-unsupported
./create_newcase --case ~/cases_LMWG_dev/ctsm52001_f09_GSWP3_bgccrop_1850AD --compset 1850_DATM%GSWP3v1_CLM60%BGC-CROP_SICE_SOCN_SGLC_SWAV_SESP --res f09_g17 --run-unsupported
./create_newcase --case ~/cases_LMWG_dev/ctsm52001_ne30_GSWP3_bgccrop_1850AD --compset 1850_DATM%GSWP3v1_CLM60%BGC-CROP_SICE_SOCN_SGLC_SWAV_SESP --res ne30pg3_ne30pg3_mg17 --run-unsupported

./create_newcase --case /glade/u/home/slevis/cases_LMWG_dev/ctsm52001_f19_GSWP3_bgccrop_hist --compset HIST_DATM%GSWP3v1_CLM60%BGC-CROP_SICE_SOCN_SGLC_SWAV_SESP --res f19_g17 --run-unsupported

./create_newcase --case /glade/u/home/slevis/cases_LMWG_dev/ctsm52001_f19_GSWP3_bgccrop_2000pSASU --compset 2000_DATM%GSWP3v1_CLM60%BGC-CROP_SICE_SOCN_SGLC_SWAV_SESP --res f19_g17 --run-unsupported

I cloned the AD cases to SASU and the SASU cases to pSASU.
1850AD cold-started.
1850SASU and 1850pSASU hybrid-started with finidat files from the preceding stage.
hist used start-up with finidat from 1850pSASU.
2000pSASU used start-up with 2000 finidat from hist.

Sequence of simulations:
1850AD 280 yrs --> 1850SASU 240 yrs --> 1850pSASU 60 yrs --> hist --> 2000pSASU 60 yrs

Spinup disequilibriums:
f09 1850AD (2.09%), 1850SASU (11.42%), 1850pSASU (1.85%)
f19 1850AD (1.86%), 1850SASU (11.98%), 1850pSASU (1.86%)
ne30 1850AD (1.35%), 1850SASU (11.44%), 1850pSASU (1.47%)

[slevis-lmwg]

@wwieder @ekluzek @olyson
No need to review unless interested. I just wanted to make you aware of the new runs.

[slevis-lmwg]

We have a cbalance error in the hist simulation:
The simulation started in 1850 (datm using 1901-1920) and crashed 80 timesteps (or so?) into 1901. I restarted from 1900-01-01 (most recent restart file) and got the same result b4b. I started hybrid from 1900-01-01 (same restart file) changing datm to use 1901-2014 and got the error but not b4b, presumably because datm was in 2014.

The error (startup and b4b restart):

dec0359.hsn.de.hpc.ucar.edu 259:  column cbalance error    =   3.217936017520587E-005          34
dec0359.hsn.de.hpc.ucar.edu 259:  is fates column?         =  F
dec0359.hsn.de.hpc.ucar.edu 259:  Latdeg,Londeg=  -36.9473684210527        145.000000000000
dec0359.hsn.de.hpc.ucar.edu 259:  begcb                    =    6751.39552603758
dec0359.hsn.de.hpc.ucar.edu 259:  endcb                    =    6751.37391112644
dec0359.hsn.de.hpc.ucar.edu 259:  delta store              =  -2.161491113565717E-002
dec0359.hsn.de.hpc.ucar.edu 259:  --- Inputs ---
dec0359.hsn.de.hpc.ucar.edu 259:  gpp                      =   0.000000000000000E+000
dec0359.hsn.de.hpc.ucar.edu 259:  --- Outputs ---
dec0359.hsn.de.hpc.ucar.edu 259:  er                       =   2.158273177092730E-002
dec0359.hsn.de.hpc.ucar.edu 259:  col_fire_closs           =   0.000000000000000E+000
dec0359.hsn.de.hpc.ucar.edu 259:  col_hrv_xsmrpool_to_atm  =   0.000000000000000E+000
dec0359.hsn.de.hpc.ucar.edu 259:  col_xsmrpool_to_atm      =   4.636403487949891E-012
dec0359.hsn.de.hpc.ucar.edu 259:  wood_harvestc            =   0.000000000000000E+000
dec0359.hsn.de.hpc.ucar.edu 259:  crop_harvestc_to_cropprodc =   0.000000000000000E+000
dec0359.hsn.de.hpc.ucar.edu 259:  -1*som_c_leached         =   8.174576120237008E-014
dec0359.hsn.de.hpc.ucar.edu 259: iam = 131: local  column   index = 34
dec0359.hsn.de.hpc.ucar.edu 259: iam = 131: global column   index = 46273
dec0359.hsn.de.hpc.ucar.edu 259: iam = 131: global landunit index = 4985
dec0359.hsn.de.hpc.ucar.edu 259: iam = 131: global gridcell index = 1668
dec0359.hsn.de.hpc.ucar.edu 259: iam = 131: gridcell longitude    =  145.0000000
dec0359.hsn.de.hpc.ucar.edu 259: iam = 131: gridcell latitude     =  -36.9473684
dec0359.hsn.de.hpc.ucar.edu 259: iam = 131: column   type         = 220
dec0359.hsn.de.hpc.ucar.edu 259: iam = 131: landunit type         = 2
dec0359.hsn.de.hpc.ucar.edu 259:  ENDRUN:
dec0359.hsn.de.hpc.ucar.edu 259:  ERROR: ERROR in CNBalanceCheckMod.F90 at line 384

The same historical simulation completes with matrixcn turned off. We need to discuss whether to debug.

[olyson]

I've added the spinup disequilibriums for these in the main text. They look spunup.

[slevis-lmwg]

Thank you @olyson!