b.e23_alpha17f.BLT1850.ne30_t232.095

[cecilehannay]

Purpose:
Tuning coupled run with rrtmgp

---

Description:
Same as 93 (#538) except:

clubb_c8 = 4.25 -> 4.35

---

Case directory:

• Locally (if still available):
  /glade/campaign/cesm/cesmdata/cseg/runs/cesm2_0/b.e23_alpha17f.BLT1850.ne30_t232.095

• On github:
  https://github.com/NCAR/amwg_dev/tree/b.e23_alpha17f.BLT1850.ne30_t232.095

---

Sandbox:

• Locally (if still available): /glade/work/hannay/cesm_tags/cesm2_3_alpha17f

• On github: https://github.com/ESCOMP/cesm/tree/cesm2_3_alpha17f

---

Diagnostics:

• Diags (if available):
  https://webext.cgd.ucar.edu/BLT1850/b.e23_alpha17f.BLT1850.ne30_t232.095/

---

Output:

• Raw output on derecho (if still available): /glade/derecho/scratch/hannay/archive/b.e23_alpha17f.BLT1850.ne30_t232.095/
• CAM Climos: /glade/campaign/cgd/amp/amwg/climo/b.e23_alpha17f.BLT1850.ne30_t232.095

---

Contacts:
@cecilehannay, @PeterHjortLauritzen, @adamrher, @brianpm, @JulioTBacmeister, @swrneale, @bstephens82,
@gustavo-marques, @fobryan3, @klindsay28,
@dabail10,
@wwieder, @slevis-lmwg, @olyson

---

Extra details:

[cecilehannay]

@gustavo-marques:
This run crashed in year 13 with the error:

FATAL from PE   255: write_energy : Ocean velocity has been truncated too many times.

It is the first time I got that error. Do you have any advice? Is it related to switching to HYCOM1?

[gustavo-marques]

@cecilehannay,
Please add the following into user_nl_mom and resubmit the case:
MAXTRUNC = 100

[cecilehannay]

I am getting an error in the radiation:

dec1337.hsn.de.hpc.ucar.edu 1092:  ERROR:
dec1337.hsn.de.hpc.ucar.edu 1092:  radiation_tend: ERROR: kdist_lw%gas_optics: gas_optics(): array tsfc has values
dec1337.hsn.de.hpc.ucar.edu 1092:   outside range

[cecilehannay]

Jiang made the following change to ensure the LGM runs:
cam/src/physics/rrtmgp/ext/rte-frontend/mo_rte_config.F90

<   logical(wl), protected, public :: check_values  = .false.
---
>   logical(wl), protected, public :: check_values  = .true.

I am using this for now.

[cecilehannay]

From Brian:

The RRTMGP framework should be able to deal with a fairly large range of atmospheric conditions. I’m surprised LGM produces values that are outside its range. 

I think for right now it is probably okay to turn off the checks, but it might not be a long-term solution. From what I remember, there are several places where values are compared to valid ranges, and I bet that this option turns them all off. I do not know what happens to the calculation if values are out of the ranges; it could be fine, but I kind of doubt it. 

I’d suggest that we try to figure out what field is failing the checks. If we think the values going into the radiation are correct, then we should reach out to Robert Pincus for advice about how to proceed.

[brianpm]

I think that the temperature range being checked is specified by the rrtmgp_coefs_lw_file file, and I think it comes from the first and last values of this array:

temp_ref = 160, 175, 190, 205, 220, 235, 250, 265, 280, 295, 310, 325, 340, 355 ;

Is there any reason to think that the surface temperature during LGM should ever get outside this range?

[cecilehannay]

Brian: To give you some context.

The current crash was in the BLT1850. It was our first attempt to equilibrate the latest cesm tag + RRTMGP by changing c8 = 4.25 -> 4.35. But the run crashed in year 39 with that error.

Jiang had the same error in his LGM run and he modified:

<   logical(wl), protected, public :: check_values  = .false.
---
>   logical(wl), protected, public :: check_values  = .true.

[jiang-zhu]

@brianpm @cecilehannay I am adding my plots here in case it is useful for other people to see.

The red line with shading shows the mean and full range of air temperature at every time step in various CESM2 simulations.
A few notes:

1. Even in a CAM6 PI run (low-top), the model temperature range could be wide, 180–320K.
2. Abrupt 4×CO2 is more prone to break the temperature bounds, especially in the stratosphere.
3. Eocene may break the upper temperature bound at the surface.
4. RRTMGP+RTE with a wider bound (160–355K) is much better than RRTMG in this regard.

According to wiki, we may see surface temperatures of ~330K at Death Valley.