Tuning taus code in CAM

[bstephens82]

I previously started another amwg_dev taus-related discussion but that one was not so much about tuning as about understanding better how the taus code behaves in CAM. This thread will be primarily focused on tuning the CLUBB taus code for use in CAM. (I am focusing on using CAM for tuning, while @domkesteffen at UWM is doing related tuning using single-column CLUBB.) I will try to put a summary of my work so far here later today. Two areas of concern, possibly related, are

1. a bright spot in SWCF off the western coast of the Ecuador-Peru region that consistently appears when the taus code is used (@zarzycki and his students have also observed this), and
2. the fact that the bright Sc area off that same coast extends too far toward the equator---however, @vlarson and I were looking at old test cases without the taus code and this seems to have been an issue for a while to varying degrees (for example there is evidence of it in my 078 cam_dev L58 run without taus code or any other modifications), although the taus code does seem to make the issue worse.

[bstephens82]

Here are diags from a 5-year run I did with a new taus tuning...this is with the latest additional elements, including prognostic momentum fluxes and the adjusted non-CLUBB parameters I found for dimming the overly bright SWCF (i.e. this setup is what the latest CESM tests are using, I believe, apart from having the taus code on and Zhun's parameters and flags. Some of these flags we may ultimately not want to use, I'm not sure...but it seems a proof of concept that we can get a fairly decent SWCF in CAM with the taus code on.
https://webext.cgd.ucar.edu/F2000climo/113.f2c.taus_zhun_c11b0.8_c80.75_pmf/atm/113.f2c.taus_zhun_c11b0.8_c80.75_pmf-obs.0001_5/

I also did a 1-year coupled test with these settings, because the RESTOM in the CAM test was high, but coupling to the ocean brings it down. Coupling to the ocean brought RESTOM down from 3-4 to around 0.5, but also brightened the Sc off South America and brightened the southern ocean. I think the South American stratus decks are still too bright relative to the CA decks, but I'm not sure how to approach that with tuning.

Here are the namelist settings I used for this run:

clubb_C1 = 1.00000000000000
clubb_C1b = 1.00000000000000
clubb_C1c = 0.750000000000000
clubb_C2rt = 1.00000000000000
clubb_C2thl = 1.00000000000000
clubb_C2rtthl = 1.00000000000000
clubb_C4 = 1.00000000000000
clubb_C_uu_shr = 0.1 !0.000000000000000E+000
clubb_C_uu_buoy = 0.000000000000000E+000
clubb_C6rt = 1.00000000000000
clubb_C6rtb = 1.00000000000000
clubb_C6rtc = 0.500000000000000
clubb_C6thl = 1.00000000000000
clubb_C6thlb = 1.00000000000000
clubb_C6thlc = 0.500000000000000
clubb_C7 = 0.1 !0.700000000000000
clubb_C7b = 0.700000000000000
clubb_C7c = 0.500000000000000
clubb_C8 = 0.7500000000000
clubb_C8b = 0.000000000000000E+000
clubb_C11 = 0.500000000000000
clubb_C11b = 0.800000000000000
clubb_C11c = 0.850000000000000
clubb_C12 = 1.00000000000000
clubb_C14 = 1.00000000000000
clubb_C_wp3_pr_turb = 1.00000000000000
clubb_C_wp2_splat = 0.000000000000000E+000
clubb_C6rt_Lscale0 = 14.0000000000000
clubb_C6thl_Lscale0 = 14.0000000000000
clubb_C7_Lscale0 = 0.850000000000000
clubb_wpxp_L_thresh = 100.000000000000
clubb_c_K = 0.200000000000000
clubb_c_K1 = 1.00000000000000
clubb_nu1 = 10.0000000000000
clubb_c_K2 = 0.100000000000000
clubb_nu2 = 1.00000000000000
clubb_c_K6 = 0.375000000000000
clubb_nu6 = 5.00000000000000
clubb_c_K8 = 10.0000000000000
clubb_nu8 = 20.0000000000000
clubb_c_K9 = 0.250000000000000
clubb_nu9 = 20.0000000000000
clubb_nu10 = 0.000000000000000E+000
clubb_c_K_hm = 0.750000000000000
clubb_c_K_hmb = 0.750000000000000
clubb_K_hm_min_coef = 0.100000000000000
clubb_nu_hm = 1.50000000000000
clubb_slope_coef_spread_DG_means_w = 21.0000000000000
clubb_pdf_component_stdev_factor_w = 1.00000000000000
clubb_coef_spread_DG_means_rt = 0.800000000000000
clubb_coef_spread_DG_means_thl = 0.800000000000000
clubb_gamma_coef = 0.300000000000000
clubb_gamma_coefb = 0.300000000000000
clubb_gamma_coefc = 1.20000000000000
clubb_mu = 5.000000000000000E-004
clubb_beta = 2.00000000000000
clubb_lmin_coef = 0.100000000000000
clubb_omicron = 0.500000000000000
clubb_zeta_vrnce_rat = 0.000000000000000E+000
clubb_upsilon_precip_frac_rat = 0.550000000000000
clubb_lambda0_stability_coef = 3.000000000000000E-002
clubb_mult_coef = 1.00000000000000
clubb_taumin = 90.0000000000000
clubb_taumax = 3600.00000000000
clubb_Lscale_mu_coef = 2.00000000000000
clubb_Lscale_pert_coef = 0.100000000000000
clubb_alpha_corr = 0.150000000000000
clubb_Skw_denom_coef = 0.000000000000000E+000
clubb_c_K10 = 0.300000000000000
clubb_c_K10h = 0.350000000000000
clubb_thlp2_rad_coef = 1.00000000000000
clubb_thlp2_rad_cloud_frac_thresh = 0.100000000000000
clubb_up2_sfc_coef = 2.00000000000000
clubb_Skw_max_mag = 4.50000000000000
clubb_C_invrs_tau_bkgnd = 1.10000000000000
clubb_C_invrs_tau_sfc = 5.000000000000000E-002
clubb_C_invrs_tau_shear = 0.220000000000000
clubb_C_invrs_tau_N2 = 0.300000000000000
clubb_C_invrs_tau_N2_wp2 = 0.100000000000000
clubb_C_invrs_tau_N2_xp2 = 0.000000000000000E+000
clubb_C_invrs_tau_N2_wpxp = 2.00000000000000
clubb_C_invrs_tau_N2_clear_wp3 = 2.00000000000000
clubb_C_invrs_tau_wpxp_Ri = 3.00000000000000
clubb_C_invrs_tau_wpxp_N2_thresh = 3.500000000000000E-004
clubb_xp3_coef_base = 0.250000000000000
clubb_xp3_coef_slope = 1.000000000000000E-002
clubb_altitude_threshold = 150.000000000000
clubb_rtp2_clip_coef = 0.500000000000000
clubb_Cx_min = 0.330000000000000
clubb_Cx_max = 0.950000000000000
clubb_Richardson_num_min = 0.250000000000000
clubb_Richardson_num_max = 400.000000000000
clubb_a3_coef_min = 1.60000000000000
clubb_a_const = 1.6000000000000
clubb_c_wp2_pr_dfsn = 0.
clubb_c_wp3_pr_dfsn = 0.
clubb_c_wp3_pr_tp = 0.

clubb_l_use_precip_frac = .true.
clubb_l_predict_upwp_vpwp = .true.
clubb_l_min_wp2_from_corr_wx = .false.
clubb_l_min_xp2_from_corr_wx = .false.
clubb_l_C2_cloud_frac = .false.
clubb_l_diffuse_rtm_and_thlm = .false.
clubb_l_stability_correct_Kh_N2_zm = .false.
clubb_l_calc_thlp2_rad = .true.
clubb_l_upwind_xpyp_ta = .true.
clubb_l_upwind_xm_ma = .true.
clubb_l_uv_nudge = .false.
clubb_l_rtm_nudge = .false.
clubb_l_tke_aniso = .true.
clubb_l_vert_avg_closure = .false.
clubb_l_trapezoidal_rule_zt = .false.
clubb_l_trapezoidal_rule_zm = .false.
clubb_l_call_pdf_closure_twice = .false.
clubb_l_standard_term_ta = .false.
clubb_l_partial_upwind_wp3 = .false.
clubb_l_godunov_upwind_wpxp_ta = .false.
clubb_l_godunov_upwind_xpyp_ta = .false.
clubb_l_use_cloud_cover = .false.
clubb_l_diagnose_correlations = .false.
clubb_l_calc_w_corr = .false.
clubb_l_const_Nc_in_cloud = .false.
clubb_l_fix_w_chi_eta_correlations = .true.
clubb_l_stability_correct_tau_zm = .false.
clubb_l_damp_wp2_using_em = .true.
clubb_l_do_expldiff_rtm_thlm = .true.
clubb_l_Lscale_plume_centered = .false.
clubb_l_diag_Lscale_from_tau = .true.
clubb_l_use_C7_Richardson = .true.
clubb_l_use_C11_Richardson = .false.
clubb_l_use_shear_Richardson = .false.
clubb_l_brunt_vaisala_freq_moist = .false.
clubb_l_use_thvm_in_bv_freq = .false.
clubb_l_rcm_supersat_adj = .true.
clubb_l_damp_wp3_Skw_squared = .true.
clubb_l_prescribed_avg_deltaz = .false.
clubb_l_lmm_stepping = .false.
clubb_l_e3sm_config = .true.
clubb_l_vary_convect_depth = .false.
clubb_l_use_tke_in_wp3_pr_turb_term = .false.
clubb_l_use_tke_in_wp2_wp3_K_dfsn = .false.
clubb_l_smooth_Heaviside_tau_wpxp = .false.
clubb_l_enable_relaxed_clipping = .true.
clubb_l_linearize_pbl_winds = .false.
clubb_l_mono_flux_lim_thlm = .true.
clubb_l_mono_flux_lim_rtm = .true.
clubb_l_mono_flux_lim_um = .true.
clubb_l_mono_flux_lim_vm = .true.
clubb_l_mono_flux_lim_spikefix = .true.

dust_emis_fact = 0.80D0
micro_mg_dcs=0.00025
micro_mg_vtrmi_factor=0.5
microp_aero_wsubi_scale=2.5
seasalt_emis_scale=1.5

[bstephens82]

@vlarson @JulioTBacmeister Here are the diags from the previous 5-year run that looked pretty good, extended to 10 years. The results are not markedly different from the 5-year run. When the "control" run completes 10 years, I will post the diags from model v model comparison.
https://webext.cgd.ucar.edu/F2000climo/113.f2c.taus_zhun_c11b0.8_c80.75_pmf/atm/113.f2c.taus_zhun_c11b0.8_c80.75_pmf-obs.0001_10/

[zarzycki]

@bstephens82 out of curiosity, is clubb_l_intr_sfc_flux_smooth=.true. in your runs? We generally found we needed that for multi-year stability in our tuning runs from last year. It would be interesting if the model is stable with that turned off now (although you might go back to getting oscillatory behavior in the winds).

[bstephens82]

Yes that is automatically set to T if cam_dev is used (which I always do use).

[bstephens82]

Ok, this took a while to do for various reasons, but I finally have the requested comparison between the 10-year run I did above with (mostly) Zhun's E3SM tuning, and a 10-year Lscale run with the latest parameter settings (which are now being reviewed and will likely be updated, bear in mind):
https://webext.cgd.ucar.edu/F2000climo/113.f2c.taus_zhun_c11b0.8_c80.75_pmf/atm/113.f2c.taus_zhun_c11b0.8_c80.75_pmf.0001_10-114.f2c.Lscale.latest_mods.0001_10/

[zarzycki]

These look "in the ballpark" to me.

First blush things for taus (vs Lscale, which I'm considering the reference) that stood out were a degradation in the overland precip in the Taylor diags, although noting systematic in the panel plots. Looks like it worsens the stress bias a bit. PBLH goes up by 30m globally. Interestingly weakens the 850mb EKE by 7% globally, perhaps that's tied to the PBLH going up, near surface winds are more easily diffused, leading to less peaked wind max at 850. Not sure which one is "better" though. I know the CAM-chem group thought the PBLH was too high in CAM6 but I haven't found a lot of actual evidence that supported that (i.e., soundings/comparison with reanalysis, etc.)

[zarzycki]

FWIW, @kylemnardi and I have quite a few differences in our baseline taus runs. Here are all the bools we've been using for our taus runs that differ from the above (our settings below, so @bstephens82's are the inverse). l_e3sm_config and l_stability_correct_tau_zm stand out to me in particular.

+ l_trapezoidal_rule_zt =  T
+ l_trapezoidal_rule_zm =  T
+ l_call_pdf_closure_twice =  T
+ l_standard_term_ta =  T
+ l_partial_upwind_wp3 =  T
+ l_godunov_upwind_wpxp_ta =  T
+ l_godunov_upwind_xpyp_ta =  T
+ l_use_cloud_cover =  T
+ l_stability_correct_tau_zm =  T
+ l_damp_wp2_using_em =  F
+ l_do_expldiff_rtm_thlm =  F
+ l_use_C7_Richardson =  F
+ l_rcm_supersat_adj =  F
+ l_damp_wp3_Skw_squared =  F
+ l_e3sm_config =  F
+ l_vary_convect_depth T
+ l_smooth_Heaviside_tau_wpxp =  T
+ l_mono_flux_lim_um =  F
+ l_mono_flux_lim_vm =  F

Some of these come from this comment thread: #106 (comment)

Do we expect these flags to be important? Question for smarter CLUBB people (i.e., @bstephens82 and @vlarson) should we try and decide on a fixed flag deck for all tuning exercises or should we also consider these free parameters?

FWIW, our tunable parameter "defaults" (differences from Ben's) are below. These aren't optimized, just what we use as our "baseline" setup for the MOAT and emulator runs (so some like c_uu_shr, C11, etc. get perturbed anyways, but not all).

I recall C1 and C1b were omitted (set to 1) in Zhun's paper. Does C1c fall under that category? Everything else seems pretty fair game although I'll admit to having little interpretation of any of their behaviors!

+ C1c                          =    1.00000000000000     
+ C4                           =    5.20000000000000     
+ C_uu_shr                     =   0.10000000000000     
+ C_uu_buoy                    =   0.300000000000000     
+ C6rtc                        =    1.00000000000000     
+ C6thlc                       =    1.00000000000000     
+ C7                           =   0.500000000000000     
+ C7b                          =   0.500000000000000     
+ C8                           =    3.44037777600000     
+ C10                          =    3.30000000000000     
+ C11                          =   0.310574118000000     
+ C11b                         =   0.310574118000000     
+ C11c                         =   0.500000000000000     
+ C_wp2_pr_dfsn                =   0.000000000000000E+000
+ C_wp3_pr_tp                  =   0.000000000000000E+000
+ C_wp3_pr_turb                =   0.400000000000000     
+ C_wp3_pr_dfsn                =   0.000000000000000E+000
+ wpxp_L_thresh                =    60.0000000000000     
+ c_K1                         =   0.750000000000000     
+ nu1                          =    20.0000000000000     
+ c_K2                         =   0.125000000000000     
+ nu2                          =    5.00000000000000     
+ c_K8                         =    1.25000000000000     
+ mu                           =   1.000000000000000E-003
+ beta                         =    2.27756371200000     
+ lambda0_stability_coef       =   4.000000000000000E-002
+ up2_sfc_coef                 =    5.00000000000000   

[bstephens82]

Yeah, what happened with this latest test was that it was clear @zhunguo was getting some decent results with the taus code in E3SM, and I was curious to see if those parameters & flags would also look good in CAM. It turned out they needed some additional adjustment in CAM, but I was able to get a pretty decent looking SWCF, maybe the best I've personally seen so far with the taus code. I am not sure of Zhun's rationales for choosing these particular flag settings, or whether all of these flags would be important for getting the same general kind of result in CAM, but some are surely minor. But thanks for listing the ones that differ, I think I should probably experiment with flipping some back the way we had before.

Regarding the two particular flags you pointed out, I don't know if there was a miscommunication somewhere along the way but I believe that if l_diag_Lscale_from_tau=T, we want l_stability_correct_tau_zm=F. This basically causes CLUBB to use some of the taus we've defined in the appropriate places. The l_e3sm_config flag ultimately just changes the formulation of the taus code in some minor ways (minor as far as code changes I mean), although it can have a big impact on results. But I don't believe there's any inherent reason it shouldn't be set to true in CAM, although we did previously agree to set it to false. Maybe @vlarson would have more to say about that.

Personally I think it would be best to have an agreed-upon flag deck that we both use, but if we want to get the taus code into CAM sooner rather than later, we may want to reopen the question of some flag settings, if Zhun's seem to work well...

[zarzycki]

Based on a call with @vlarson, @JulioTBacmeister, and @bstephens82...

l_vary_convect_depth is considered experimental and we should set to F for the time being. I vaguely recall this being beneficial for CLASP runs (cc: @megandevlan).

l_e3sm_config modifies the invrs_tau_zm profiles to reproduce @zhunguo's JAMES paper, which was tuned with a no-ZM version of E3SM -- ideally, we'd like to have tunings behave relatively similarly if this flag is T or F, that way any E3SM tunings would be semi-transferable. Ben trying a run removing a hardcoded coefficient when l_e3sm_config = F to see if that's the source of climo differences between the flag on/off ... If yes, I would prefer l_e3sm_config=F for interpretability since the code is cleaner with fewer branches.

l_trapezoidal_rule_zt and l_trapezoidal_rule_zm are F in Zhun's tunings. We had set them to T at PSU based on some conversations with @adamrher a few years back. Ben reports there may be little difference in having it T or F, Julio speculates at higher vertical res they are less important. Vince notes setting some of these bools to F could reduce CLUBB cost so if unimportant that should be our default strategy? Ex: we at PSU can also set l_call_pdf_closure_twice = F, etc.

l_use_C7_Richardson is relatively important (i.e., climate-changing) but also "experimental." PSU will set to T to match Ben's config unless he tells us otherwise.

As Ben said above, if l_diag_Lscale_from_tau=T, we want l_stability_correct_tau_zm=F.

[megandevlan]

Just to add briefly, we did also see that l_vary_convect_depth had a pretty strong impact on the results in the CLASP runs. We've been running with this set to False, since it's still experimental and we could make more physical sense of the results this way (that said, we also used z_const = zbot for consistency with CLM).

[bstephens82]

With regard to Colin's point about some CLUBB booleans having changed, I ran a few (not very systematic) tests to see what flags if any we could switch back to our prior settings without much affecting the nice results I got with the tuning I've been using lately based on Zhun's work with E3SM (I'll just call that run "Z+").

The first test I did was to flip these flags (i.e. I switched to the following values):

clubb_l_trapezoidal_rule_zt =  .true.
clubb_l_trapezoidal_rule_zm =  .true.
clubb_l_call_pdf_closure_twice =  .true.
clubb_l_do_expldiff_rtm_thlm =  .false.

With these flags flipped back to their old values, I got results that are pretty similar to what we had before and for SWCF possibly even slightly better. Here are 5-year diagnostics for this setup compared to Z+:
https://webext.cgd.ucar.edu/F2000climo/113.f2c.taus_zhun_c11b0.8_c80.75_pmf/atm/117.f2c.taus_zhmod_c11b0.8_c80.75_pmf.0001_5-113.f2c.taus_zhun_c11b0.8_c80.75_pmf.0001_5/

The second test I did was to keep the prior changes, and then switch to the following settings as well:

clubb_l_upwind_xpyp_ta =  .false.
clubb_l_standard_term_ta =  .true.
clubb_l_partial_upwind_wp3 =  .true.
clubb_l_godunov_upwind_wpxp_ta =  .true.
clubb_l_godunov_upwind_xpyp_ta =  .true.
clubb_l_smooth_Heaviside_tau_wpxp =  .true.

This set of flags had a much larger impact, increasing the magnitude of SWCF to -50+ W/m2, although I have not yet tried to determine which flags are the most important. I thought switching the upwind flags would be minor but maybe it has more impact than I thought, or maybe the Heaviside flag has a big impact although I would've thought it was just applying some smoothing. Here are those diags compared to Z+:
https://webext.cgd.ucar.edu/F2000climo/113.f2c.taus_zhun_c11b0.8_c80.75_pmf/atm/117.f2c.taus_zhmod2_c11b0.8_c80.75_pmf.0001_1-113.f2c.taus_zhun_c11b0.8_c80.75_pmf.0001_1/

A few other points:

• I've turned on the two MFL flags, i.e. l_mono_flux_lim_um=T and l_mono_flux_lim_vm=T. I think these are minor but they should be used anyway if we are using prognostic momentum fluxes.
• l_vary_convect_depth=T has a big impact, but Zhun has set it to F, and it is arguably experimental, so I think we can leave it F for now.
• l_use_C7_Richardson also has a big impact (turning it off and leaving everything else the same increases SWCF to about -50 W/m2). I am going with Zhun's setting (l_use_C7_Richardson=T) for now. As I understand it there is not a strong argument to be made either way. Although note that if this flag is true, the C7 parameter is no longer used and therefore no need to include it in tuning (we might want to think about what other leverage we have over momentum fluxes in this case, including possibly the Richardson_num_min and _max parameters.
• Zhun has set l_e3sm_config=T. We know this flag has a big impact, but I am testing whether we can set this flag to false by making a minor change to that branch of the if statement.

[adamrher]

Just a quick follow-up about these two flags in your first experiment:

clubb_l_call_pdf_closure_twice =  .true.
clubb_l_do_expldiff_rtm_thlm =  .false.

Were these not always set that way in all your experiments so far? We did look at the 2X pdf closure a while back (see here) and it looked to help with the pdf derived terms on the zm levels such as the buoyancy flux, near the inversion height. The other namelist clubb_l_do_expldiff_rtm_thlm I really can't imagine that you ever had it set to =T, as that smooths out the inversion layer and reduces low clouds drastically.

Just to follow up on the trapezoidal rules convo, they had a much smaller effect at L58 compared to L32, and so we actually reverted them yet again to be =T.

[bstephens82]

Were these not always set that way in all your experiments so far?

Yes, those are the settings I usually use (which is why Colin pointed them out since they were different in this latest run using Zhun's E3SM settings)

Just to follow up on the trapezoidal rules convo, they had a much smaller effect at L58 compared to L32, and so we actually reverted them yet again to be =T.

Yeah setting these to T didn't mess up Zhun's results in any obvious way so I'm going to continue to stick with that going forward.

[bstephens82]

I tried a run with l_e3sm_config=F, but with the factor of 0.1 removed from the invrs_tau_bkgnd term in the definition of invrs_tau_xp2_zm. I think this run is competitive or better with what we had before. So I think perhaps a new baseline to build on from this point forward could be this case, but also with (at least) the first four flags flipped that I mentioned above (trapezoidals, etc.). Here is the l_e3sm_config=F test compared with the original for 1 year:
https://webext.cgd.ucar.edu/F2000climo/113.f2c.taus_zhun_c11b0.8_c80.75_pmf/atm/117.f2c.taus_zhun_c11b0.8_c80.75_pmf_e3smmod.0001_1-113.f2c.taus_zhun_c11b0.8_c80.75_pmf.0001_1/

[bstephens82]

Here's a comparison between 5-year tests of the "original" tuning we liked vs. the one with l_e3sm_config=F but the factor of 0.1 removed from invrs_tau_bkgnd that we talked about. They are quite similar:
https://webext.cgd.ucar.edu/F2000climo/113.f2c.taus_zhun_c11b0.8_c80.75_pmf/atm/117.f2c.taus_zhun_c11b0.8_c80.75_pmf_e3smmod.0001_5-113.f2c.taus_zhun_c11b0.8_c80.75_pmf.0001_5/

[bstephens82]

So I have a new baseline case that I've come up with (took a little longer due some silly mistakes I made previously). It's called 117.f2c.taus_new_base3 due to having to tweak some things. Anyway, the diagnostics are available online here. The RESTOM is high, about 5 W/m2 or so, but the SWCF and LWCF don't look too bad:

I tried to make my own tape recorder plot. Assuming it's done correctly it's too moist relative to obs:

(Sorry for the lack of labels, it's pressure in mb on the vertical axis and time on the horizontal axis, and the quantity plotted should be specific humidity in kg/kg.)

The setup for this case is as follows (note that there are some non-CLUBB parameters modified at the bottom):

clubb_C1 = 1.00000000000000
clubb_C1b = 1.00000000000000
clubb_C1c = 0.750000000000000
clubb_C2rt = 1.00000000000000
clubb_C2thl = 1.00000000000000
clubb_C2rtthl = 1.00000000000000
clubb_C4 = 1.00000000000000
clubb_C_uu_buoy = 0.000000000000000E+000
clubb_C6rt = 1.00000000000000
clubb_C6rtb = 1.00000000000000
clubb_C6rtc = 0.500000000000000
clubb_C6thl = 1.00000000000000
clubb_C6thlb = 1.00000000000000
clubb_C6thlc = 0.500000000000000
clubb_C7b = 0.700000000000000
clubb_C7c = 0.500000000000000
clubb_C8 = 0.600000000000
clubb_C8b = 0.000000000000000E+000
clubb_C11 = 0.500000000000000
clubb_C11b = 0.800000000000000
clubb_C11c = 0.850000000000000
clubb_C12 = 1.00000000000000
clubb_C14 = 1.00000000000000
clubb_C_wp3_pr_turb = 1.00000000000000
clubb_C_wp2_splat = 0.000000000000000E+000
clubb_C6rt_Lscale0 = 14.0000000000000
clubb_C6thl_Lscale0 = 14.0000000000000
clubb_C7_Lscale0 = 0.850000000000000
clubb_wpxp_L_thresh = 100.000000000000
clubb_c_K = 0.200000000000000
clubb_c_K1 = 1.00000000000000
clubb_nu1 = 10.0000000000000
clubb_c_K2 = 0.100000000000000
clubb_nu2 = 1.00000000000000
clubb_c_K6 = 0.375000000000000
clubb_nu6 = 5.00000000000000
clubb_c_K8 = 10.0000000000000
clubb_nu8 = 20.0000000000000
clubb_c_K9 = 0.250000000000000
clubb_nu9 = 20.0000000000000
clubb_nu10 = 0.000000000000000E+000
clubb_c_K_hm = 0.750000000000000
clubb_c_K_hmb = 0.750000000000000
clubb_K_hm_min_coef = 0.100000000000000
clubb_nu_hm = 1.50000000000000
clubb_slope_coef_spread_DG_means_w = 21.0000000000000
clubb_pdf_component_stdev_factor_w = 1.00000000000000
clubb_coef_spread_DG_means_rt = 0.800000000000000
clubb_coef_spread_DG_means_thl = 0.800000000000000
clubb_gamma_coef = 0.330000000000000
clubb_gamma_coefb = 0.330000000000000
clubb_gamma_coefc = 1.20000000000000
clubb_mu = 5.000000000000000E-004
clubb_beta = 2.00000000000000
clubb_lmin_coef = 0.100000000000000
clubb_omicron = 0.500000000000000
clubb_zeta_vrnce_rat = 0.000000000000000E+000
clubb_upsilon_precip_frac_rat = 0.550000000000000
clubb_lambda0_stability_coef = 3.000000000000000E-002
clubb_mult_coef = 1.00000000000000
clubb_taumin = 90.0000000000000
clubb_taumax = 3600.00000000000
clubb_Lscale_mu_coef = 2.00000000000000
clubb_Lscale_pert_coef = 0.100000000000000
clubb_alpha_corr = 0.150000000000000
clubb_Skw_denom_coef = 0.000000000000000E+000
clubb_c_K10 = 0.300000000000000
clubb_c_K10h = 0.350000000000000
clubb_thlp2_rad_coef = 1.00000000000000
clubb_thlp2_rad_cloud_frac_thresh = 0.100000000000000
clubb_up2_sfc_coef = 2.00000000000000
clubb_Skw_max_mag = 4.50000000000000
clubb_C_invrs_tau_bkgnd = 1.10000000000000
clubb_C_invrs_tau_sfc = 5.000000000000000E-002
clubb_C_invrs_tau_shear = 0.220000000000000
clubb_C_invrs_tau_N2 = 0.300000000000000
clubb_C_invrs_tau_N2_wp2 = 0.100000000000000
clubb_C_invrs_tau_N2_xp2 = 0.000000000000000E+000
clubb_C_invrs_tau_N2_wpxp = 2.00000000000000
clubb_C_invrs_tau_N2_clear_wp3 = 2.00000000000000
clubb_C_invrs_tau_wpxp_Ri = 3.00000000000000
clubb_C_invrs_tau_wpxp_N2_thresh = 3.500000000000000E-004
clubb_xp3_coef_base = 0.250000000000000
clubb_xp3_coef_slope = 1.000000000000000E-002
clubb_altitude_threshold = 150.000000000000
clubb_rtp2_clip_coef = 0.500000000000000
clubb_Cx_min = 0.330000000000000
clubb_Cx_max = 0.950000000000000
clubb_Richardson_num_min = 0.250000000000000
clubb_Richardson_num_max = 400.000000000000
clubb_a3_coef_min = 1.60000000000000
clubb_a_const = 1.6000000000000
clubb_c_wp2_pr_dfsn = 0.
clubb_c_wp3_pr_dfsn = 0.
clubb_c_wp3_pr_tp = 0.

clubb_l_use_precip_frac = .true.
clubb_l_predict_upwp_vpwp = .true.
clubb_l_min_wp2_from_corr_wx = .false.
clubb_l_min_xp2_from_corr_wx = .false.
clubb_l_C2_cloud_frac = .false.
clubb_l_diffuse_rtm_and_thlm = .false.
clubb_l_stability_correct_Kh_N2_zm = .false.
clubb_l_calc_thlp2_rad = .true.
clubb_l_upwind_xpyp_ta = .true.
clubb_l_upwind_xm_ma = .true.
clubb_l_uv_nudge = .false.
clubb_l_rtm_nudge = .false.
clubb_l_tke_aniso = .true.
clubb_l_vert_avg_closure = .false.
clubb_l_trapezoidal_rule_zt = .true.
clubb_l_trapezoidal_rule_zm = .true.
clubb_l_call_pdf_closure_twice = .true.
clubb_l_standard_term_ta = .false.
clubb_l_partial_upwind_wp3 = .false.
clubb_l_godunov_upwind_wpxp_ta = .false.
clubb_l_godunov_upwind_xpyp_ta = .false.
clubb_l_use_cloud_cover = .false.
clubb_l_diagnose_correlations = .false.
clubb_l_calc_w_corr = .false.
clubb_l_const_Nc_in_cloud = .false.
clubb_l_fix_w_chi_eta_correlations = .true.
clubb_l_stability_correct_tau_zm = .false.
clubb_l_damp_wp2_using_em = .true.
clubb_l_do_expldiff_rtm_thlm = .false.
clubb_l_Lscale_plume_centered = .false.
clubb_l_diag_Lscale_from_tau = .true.
clubb_l_use_C7_Richardson = .true.
clubb_l_use_C11_Richardson = .false.
clubb_l_use_shear_Richardson = .false.
clubb_l_brunt_vaisala_freq_moist = .false.
clubb_l_use_thvm_in_bv_freq = .false.
clubb_l_rcm_supersat_adj = .true.
clubb_l_damp_wp3_Skw_squared = .true.
clubb_l_prescribed_avg_deltaz = .false.
clubb_l_lmm_stepping = .false.
clubb_l_e3sm_config = .false.
clubb_l_vary_convect_depth = .false.
clubb_l_use_tke_in_wp3_pr_turb_term = .false.
clubb_l_use_tke_in_wp2_wp3_K_dfsn = .false.
clubb_l_smooth_Heaviside_tau_wpxp = .false.
clubb_l_enable_relaxed_clipping = .true.
clubb_l_linearize_pbl_winds = .false.
clubb_l_mono_flux_lim_thlm = .true.
clubb_l_mono_flux_lim_rtm = .true.
clubb_l_mono_flux_lim_um = .true.
clubb_l_mono_flux_lim_vm = .true.
clubb_l_mono_flux_lim_spikefix = .true.

! non-CLUBB params
micro_mg_dcs=0.00025
micro_mg_vtrmi_factor=0.5
microp_aero_wsubi_scale=2.5
seasalt_emis_scale=1.5
dust_emis_fact = 0.80D0

[vlarson]

So I have a new baseline case that I've come up with (took a little longer due some silly mistakes I made previously). It's called 117.f2c.taus_new_base3 due to having to tweak some things. Anyway, the diagnostics are available online here. The RESTOM is high, about 5 W/m2 or so, but the SWCF and LWCF don't look too bad:

This doesn't look too bad to me, but the DJF 200-mb wind is still too zonal. I'm not sure what parameters influence this.

[zarzycki]

Makes sense (to me) that the upper atmosphere seems more of a problem area in those runs (LWCF, tape recorder, U200).

@kylemnardi, do you have any U200 (or other upper level) outputs from any of your Morris runs? In the "dynamical" deck?

[kylemnardi]

I have U200, Z500, and omega at 700 hPa from my latest Morris runs. The first plot is average response of each output to each input parameter (darker colors imply greater average response of the output). The second plot is the direction of the response (dark reds imply that increasing the input parameter frequently increases the output). These are globally-averaged output metrics at 72 hours. The U's here are wind speeds. This is output averaged over four different initializations

[zarzycki]

Thanks!

So just skimming the U200 column, micro_mg_vtrmi_factor and micro_mg_dcs are important and unidirectional and seem like logical targets, especially given the tape recorder results @PeterHjortLauritzen mentions below. Perhaps changing micro_mg_vtrmi_factor will naturally improve U200.

c_uu_shr and c_invrs_tau_shear also seem important and quasi-directional but would seem to be less ideal knobs since they'd really hit low-level winds and clouds, which @bstephens82 thinks look good.

[PeterHjortLauritzen]

Thanks Ben!

The tape recorder might be recovered with setting micro_mg_vtrmi_factor and microp_aero_wsubi_scale back to 1.

Some maybe notable/important differences in MG namelist variables between your setup and Cecile's new baseline are:

micro_mg_vtrmi_factor. =1.0
microp_aero_wsubi_scale = 1.0
micro_mg_dcs = 500.D-6

[bstephens82]

Yeah, I'm trying to run that test, but my jobs keep crashing on Cheyenne and hanging on Derecho...

[jedwards4b]

Ben - point me to your case on derecho.

[bstephens82]

well of course it's running now that I tried again...but here's the case directory:
/glade/derecho/scratch/stepheba/117.f2c.taus_new_base_latest_mods3

[jedwards4b]

I found it - update your ccs_config to the main as follows:
cd /glade/u/home/stepheba/CAM/ccs_config
git checkout main
git pull origin main

Then go to your case directory and do
./case.setup --reset
./case.build
./case.submit

This should solve the hang issue.

[bstephens82]

Ok thanks, I will try it.

[zarzycki]

FYI, @kylemnardi has been looking at the sensitivity of results to some of the CLUBB bools. This is a bit of an experimental thing we are trying with a Sobol technique, but here are some relevant plots (larger bar implies the global climate is more sensitive to the setting of the bool's switch when the taus code is on)

My quick takehomes...

clubb_l_e3sm_config is important for both LWCF and SWCF, although I think we've agreed based on Ben's results (see #262 (comment) and a few comments below) to leave that as false (with the updated coefficient in the code?). I also support false because it's cleaner/more interpretable mathematically.

clubb_l_stability_correct_tau_zm and clubb_l_do_expldiff_rtm_thlm matter, but I think we agree to leave those as false.

Perhaps the three of quasi interest to me are:
clubb_l_trapezoidal_rule_zt/m (which seems to have a non-zero impact on mid-level clouds and therefore LWCF. We are running this at L58 -- @adamrher likes true?)
clubb_l_use_cloud_cover (allows for some sort of vertical interaction in the cloud field in CLUBB? current default false)
clubb_l_call_pdf_closure_twice (seems like @adamrher and @bstephens82 like true)

Semi-interestingly, Kyle's initial results are that the upwind terms @bstephens82 listed as possibly more significant in this ticket: #262 (comment) were relatively small. However, we did not test clubb_l_smooth_Heaviside_tau_wpxp so my hypothesis is that term is actually the key player and probably modulating the inversion/cloud heights/coverage. But that's just speculation. Looks like Kyle did actually tweak that and it wasn't that sensitive. It was clubb_l_upwind_xpyp_ta he didn't test.

I have not done a deep enough dive into the timing numbers, but if there are particular bools that we think result in a penalty (i.e., loops or additional solves), we should test them and turn them off if we can't justify them physically or as a tuning improvement. Possible a question for @vlarson.

[adamrher]

clubb_l_trapezoidal_rule_zt/m (which seems to have a non-zero impact on mid-level clouds and therefore LWCF. We are running this at L58 -- @adamrher likes true?

At L32, turning the trapezoidal rules on reduced the cloud liquid, and weakened SWCF by about 3 W/m2 (see here). There was little change in LWCF. At L58 the effect is smaller (as expected); turning them on weakened SWCF by about 1.5 W/m2 (see here). Whereas in L32 the weakened SWCF happened over the MarineSc regions as well, in L58 the MarineSc regions actually brighten a bit. For the latter reason, we turned them back on in L58. Although I could see just as easily turning them off in the future, especially since folks are pinging me about concerns of super-saturation in CAM6, and so I need to see whether L58 supersaturation amounts are as sensitive to the trapezoidal rules, as in L32.

[vlarson]

l_call_pdf_closure_twice=T, l_trapezoidal_rule_zt=T, and l_trapezoidal_rule_zm=T increase the computational cost and the code complexity. If we could produce good results with those flags set to false, then that would simplify a bit.

[bstephens82]

Here's a new run with pretty decent looking SWCF (it's a bit low in mean magnitude, but will likely brighten when coupled as I understand it). LWCF is low (~21), and the tape recorder appears to still be too moist (according to my somewhat sketchy plot which I don't entirely trust).
https://webext.cgd.ucar.edu/F2000climo/117.f2c.taus_new_base_latest_mods6c/atm/117.f2c.taus_new_base_latest_mods6c-obs.0001_1/

My flag and parameter settings are as follows:
clubb_C1 = 1.00000000000000
clubb_C1b = 1.00000000000000
clubb_C1c = 0.750000000000000
clubb_C2rt = 1.00000000000000
clubb_C2thl = 1.00000000000000
clubb_C2rtthl = 1.00000000000000
clubb_C4 = 1.00000000000000
clubb_C_uu_buoy = 0.000000000000000E+000
clubb_C6rt = 1.00000000000000
clubb_C6rtb = 1.00000000000000
clubb_C6rtc = 0.500000000000000
clubb_C6thl = 1.00000000000000
clubb_C6thlb = 1.00000000000000
clubb_C6thlc = 0.500000000000000
clubb_C7b = 0.700000000000000
clubb_C7c = 0.500000000000000
clubb_C8 = 0.80000000000
clubb_C8b = 0.000000000000000E+000
clubb_C11 = 0.500000000000000
clubb_C11b = 0.800000000000000
clubb_C11c = 0.850000000000000
clubb_C12 = 1.00000000000000
clubb_C14 = 1.00000000000000
clubb_C_wp3_pr_turb = 1.00000000000000
clubb_C_wp2_splat = 0.000000000000000E+000
clubb_C6rt_Lscale0 = 14.0000000000000
clubb_C6thl_Lscale0 = 14.0000000000000
clubb_C7_Lscale0 = 0.850000000000000
clubb_wpxp_L_thresh = 100.000000000000
clubb_c_K = 0.200000000000000
clubb_c_K1 = 1.00000000000000
clubb_nu1 = 10.0000000000000
clubb_c_K2 = 0.100000000000000
clubb_nu2 = 1.00000000000000
clubb_c_K6 = 0.375000000000000
clubb_nu6 = 5.00000000000000
clubb_c_K8 = 10.0000000000000
clubb_nu8 = 20.0000000000000
clubb_c_K9 = 0.250000000000000
clubb_nu9 = 20.0000000000000
clubb_nu10 = 0.000000000000000E+000
clubb_c_K_hm = 0.750000000000000
clubb_c_K_hmb = 0.750000000000000
clubb_K_hm_min_coef = 0.100000000000000
clubb_nu_hm = 1.50000000000000
clubb_slope_coef_spread_DG_means_w = 21.0000000000000
clubb_pdf_component_stdev_factor_w = 1.00000000000000
clubb_coef_spread_DG_means_rt = 0.800000000000000
clubb_coef_spread_DG_means_thl = 0.800000000000000
clubb_gamma_coef = 0.250000000000000
clubb_gamma_coefb = 0.320000000000000
clubb_gamma_coefc = 1.20000000000000
clubb_mu = 5.000000000000000E-004
clubb_beta = 2.00000000000000
clubb_lmin_coef = 0.100000000000000
clubb_omicron = 0.500000000000000
clubb_zeta_vrnce_rat = 0.000000000000000E+000
clubb_upsilon_precip_frac_rat = 0.550000000000000
clubb_lambda0_stability_coef = 3.000000000000000E-002
clubb_mult_coef = 1.00000000000000
clubb_taumin = 90.0000000000000
clubb_taumax = 3600.00000000000
clubb_Lscale_mu_coef = 2.00000000000000
clubb_Lscale_pert_coef = 0.100000000000000
clubb_alpha_corr = 0.150000000000000
clubb_Skw_denom_coef = 0.000000000000000E+000
clubb_c_K10 = 0.300000000000000
clubb_c_K10h = 0.350000000000000
clubb_thlp2_rad_coef = 1.00000000000000
clubb_thlp2_rad_cloud_frac_thresh = 0.100000000000000
clubb_up2_sfc_coef = 2.00000000000000
clubb_Skw_max_mag = 4.50000000000000
clubb_C_invrs_tau_bkgnd = 1.10000000000000
clubb_C_invrs_tau_sfc = 5.000000000000000E-002
clubb_C_invrs_tau_shear = 0.220000000000000
clubb_C_invrs_tau_N2 = 0.300000000000000
clubb_C_invrs_tau_N2_wp2 = 0.100000000000000
clubb_C_invrs_tau_N2_xp2 = 0.000000000000000E+000
clubb_C_invrs_tau_N2_wpxp = 2.00000000000000
clubb_C_invrs_tau_N2_clear_wp3 = 2.00000000000000
clubb_C_invrs_tau_wpxp_Ri = 3.00000000000000
clubb_C_invrs_tau_wpxp_N2_thresh = 3.500000000000000E-004
clubb_xp3_coef_base = 0.250000000000000
clubb_xp3_coef_slope = 1.000000000000000E-002
clubb_altitude_threshold = 150.000000000000
clubb_rtp2_clip_coef = 0.500000000000000
clubb_Cx_min = 0.330000000000000
clubb_Cx_max = 0.950000000000000
clubb_Richardson_num_min = 0.250000000000000
clubb_Richardson_num_max = 400.000000000000
clubb_a3_coef_min = 1.60000000000000
clubb_a_const = 1.6000000000000
clubb_c_wp2_pr_dfsn = 0.
clubb_c_wp3_pr_dfsn = 0.
clubb_c_wp3_pr_tp = 0.

clubb_l_use_precip_frac = .true.
clubb_l_predict_upwp_vpwp = .true.
clubb_l_min_wp2_from_corr_wx = .false.
clubb_l_min_xp2_from_corr_wx = .false.
clubb_l_C2_cloud_frac = .false.
clubb_l_diffuse_rtm_and_thlm = .false.
clubb_l_stability_correct_Kh_N2_zm = .false.
clubb_l_calc_thlp2_rad = .true.
clubb_l_upwind_xpyp_ta = .true.
clubb_l_upwind_xm_ma = .true.
clubb_l_uv_nudge = .false.
clubb_l_rtm_nudge = .false.
clubb_l_tke_aniso = .true.
clubb_l_vert_avg_closure = .false.
clubb_l_trapezoidal_rule_zt = .true.
clubb_l_trapezoidal_rule_zm = .true.
clubb_l_call_pdf_closure_twice = .true.
clubb_l_standard_term_ta = .false.
clubb_l_partial_upwind_wp3 = .false.
clubb_l_godunov_upwind_wpxp_ta = .false.
clubb_l_godunov_upwind_xpyp_ta = .false.
clubb_l_use_cloud_cover = .false.
clubb_l_diagnose_correlations = .false.
clubb_l_calc_w_corr = .false.
clubb_l_const_Nc_in_cloud = .false.
clubb_l_fix_w_chi_eta_correlations = .true.
clubb_l_stability_correct_tau_zm = .false.
clubb_l_damp_wp2_using_em = .true.
clubb_l_do_expldiff_rtm_thlm = .false.
clubb_l_Lscale_plume_centered = .false.
clubb_l_diag_Lscale_from_tau = .true.
clubb_l_use_C7_Richardson = .true.
clubb_l_use_C11_Richardson = .false.
clubb_l_use_shear_Richardson = .false.
clubb_l_brunt_vaisala_freq_moist = .false.
clubb_l_use_thvm_in_bv_freq = .false.
clubb_l_rcm_supersat_adj = .true.
clubb_l_damp_wp3_Skw_squared = .true.
clubb_l_prescribed_avg_deltaz = .false.
clubb_l_lmm_stepping = .false.
clubb_l_e3sm_config = .false.
clubb_l_vary_convect_depth = .false.
clubb_l_use_tke_in_wp3_pr_turb_term = .false.
clubb_l_use_tke_in_wp2_wp3_K_dfsn = .false.
clubb_l_smooth_Heaviside_tau_wpxp = .false.
clubb_l_enable_relaxed_clipping = .true.
clubb_l_linearize_pbl_winds = .false.
clubb_l_mono_flux_lim_thlm = .true.
clubb_l_mono_flux_lim_rtm = .true.
clubb_l_mono_flux_lim_um = .true.
clubb_l_mono_flux_lim_vm = .true.
clubb_l_mono_flux_lim_spikefix = .true.

dust_emis_fact = 0.80D0
seasalt_emis_scale=1.5
micro_mg_dcs=0.0005
micro_mg_vtrmi_factor=1.0
microp_aero_wsubi_scale=1.0
microp_aero_wsub_scale = 0.01

[vlarson]

Thanks, Ben. This simulation is not bad for some fields. The Taylor diagram suggests that the fields that need most improvement are precip over land, sfc stress, and 300-mb wind.

For comparison purposes, it would be helpful to have output from a vanilla CAM6 run. I don't have a sense of whether the new code is better or worse than CAM6 for, e.g., PSL.