Wrong TSK output when nest starts

[mathieulandreau]

Describe the bug
I ran a 5 nested domains simulation, each domain starting later than its parent. I notice that at time when d03 starts, d02 output an unrealistically low TSK value. Same with d03 when d04 starts and with d04 when d05 starts.
It seems to apply only on land.
I have, at first sight, no problem on d01 when d02 starts.

To Reproduce
Steps to reproduce the behavior:

1. I used WRF v4.5
2. My namelist is attached

Screenshots
I join a plot of the evolution of TSK over time at a random location in land (the bug doesn't apply on sea apparently) for each domain.
I also join a video of TSK in d02 on 17 May 2020 between 15h and 22h. d03 starts at 20h, we can clearly see a drop on land.

Attachments

namelist_12_07.txt

[mathieulandreau]

Additional notes :

-Air temperature and soil temperature are not impacted by this issue.
-Since TSK seems to get back to normal values, it is probably just a matter of output

-As for issue #1927, it is probably related to the call :
start_domain(parent,.TRUE.)
in share/mediation_integrate.F, subroutine med_nest_initial)

-I can't say for sure which part of the start_domain can have an impact on TSK but the use of the variable "first_trip_for_this_domain" seems a bit dangerous in start_em.F
l.232 : IF grid%itimestep == 0 THEN first_trip_for_this_domain = .TRUE. ELSE .FALSE.
l.238 : IF ( .NOT. (restart or moved)) THEN grid%itimestep = 0
No more modification of grid%itimestep in this routine until l.316 (I can't say if anywhere else it is modified)
l.316 : IF grid%itimestep == 0 THEN first_trip_for_this_domain = .TRUE. ELSE .FALSE.

Mathieu

[mathieulandreau]

Additional notes :

GLW, Latent and sensible heat fluxes, NOAHRES and u* are also affected by the bug (figures attached)
For u* and sensible heat flux, it seems clear that the error occur also in d01 when d02 starts.

[dudhia]

We will try to repeat this problem.

[mathieulandreau]

Thank you.

I performed few more tests with a shorter simulation and there are evidences that the problem is related to the radiation scheme and more precisely the GLW variable.

Here is the namelist I used :
namelist_s12t27.txt
It is still a 5 domains simulation
It lasts 5 hours, each nest starts 1 hour after its parent begins
The radiation schemes are RRTMG for both LW and SW
The radiation timestep radt is 9 minutes
The swint_opt option is activated

For the following figures :
At each history timestep (1 m for d01 and 20 s for d02-d05), the quantity plotted is spatially average over land points only.
The time-evolution of this average is shown for each domain

TSK ( Skin temperature here in °C):
We still see the gap problem but the smaller history timestep permit to observe that this gap last longer than a simulation timestep (several minutes)

T_C (1st mesh air temperature in °C) :
It reveals that air temperature is also impacted by this bug

SH_FLX (Sensible heat flux) :
It becomes negative during gap

GLW (DOWNWARD LONG WAVE FLUX AT GROUND SURFACE) :
The value falls to 0 during gap

Other variables :
Variables not affected : SWDOWN, LWDNB, LWDNT, LWDNTC, SWDNBC, SWDNT, SWDNTC, SWUPB
Variables affected (but maybe in reaction to the lower TSK) : LWUPB, LWUPBC, LWUPT, LWUPTC,

What I understand from that :
During the call of start_domain(parent,.TRUE.) in share/mediation_integrate.F, line 838, some radiation variables are reset to 0, when the radiation scheme is called again, the right radiation flux is back.
According to the figures, it seems that GLW is the problematic variable but I can't be sure.
The use of swint_opt might also be part of the problem, I didn't try without it.

A Possible solution then :
Add another save_ variable for GLW in share/mediation_integrate.F
`
save_acsnow = parent%acsnow
save_acsnom = parent%acsnom
save_cuppt = parent%cuppt
save_rainc = parent%rainc
save_rainnc = parent%rainnc
save_sfcevp = parent%sfcevp
save_sfcrunoff = parent%sfcrunoff
save_udrunoff = parent%udrunoff
save_itimestep = parent%itimestep
parent%imask_nostag = 1
parent%imask_xstag = 1
parent%imask_ystag = 1
parent%imask_xystag = 1

 parent%press_adj = .FALSE.
 CALL start_domain ( parent , .TRUE. )

 parent%acsnow     = save_acsnow
 parent%acsnom     = save_acsnom
 parent%cuppt      = save_cuppt
 parent%rainc      = save_rainc
 parent%rainnc     = save_rainnc
 parent%sfcevp     = save_sfcevp
 parent%sfcrunoff  = save_sfcrunoff
 parent%udrunoff   = save_udrunoff
 parent%itimestep  = save_itimestep

`

Mathieu

[dudhia]

Nice work. SWDOWN may not show up because swint_opt will force a new value every time step. Without swint_opt it might have the same problem. I think your suggestion will work, so I suggest testing it to confirm. Having feedback = 0 reveals this problem.

[mathieulandreau]

Hi again,
Here is what happen when I save the GLW quantity before the start_domain(parent) call and set it back after. This time I didn't wait for the simulation to run all 5 domains so the plots concern only d01 and d02.

GLW :
Problem is solved for this variable,

TSK :
There is still an issue with TSK, but it is way less important and it last only 1 simulation timestep (in d01 the history timestep is the simulation timestep)

SH_FLX :
The gap reach exactly a value of 0. It can explain the small gap in TSK

LH_FLX (Latent Heat Flux) :
It doesn't seem to be affected, there is (probably) a response to the change of TSK at the next timestep

Conclusion :
It confirms the origin of the bug but the correction is not sufficient.
I will then try to save SH_FLX to check if it fixes the remaining errors.

[mathieulandreau]

I confirm that the problem is the same without swint_opt.
Mathieu

[dudhia]

Is there an additional problem with SWDOWN without swint_opt?

[dudhia]

TSK could also be added to the list.

[mathieulandreau]

Dudhia,
I don't see any additional problem with SWDOWN.
Did you reproduce the bug ?
And what is the point of calling start_domain(parent) ?

[dudhia]

We probably could but have not used time series output yet. Feedback also needs to be off. I am thinking start_domain is here just filling in parent values for when there is no feedback.

…

On Wed, Dec 13, 2023 at 9:51 AM mathieulandreau ***@***.***> wrote: Dudhia, I don't see any additional problem with SWDOWN. Did you reproduce the bug ? And what is the point of calling start_domain(parent) ? — Reply to this email directly, view it on GitHub <#1947 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AEIZ77ALTGGAWAKJL65RNHDYJHMKPAVCNFSM6AAAAABALHBGYOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQNJUGMZTEMJSG4> . You are receiving this because you commented.Message ID: ***@***.***>

[mathieulandreau]

When QFX and HFX are saved

HFX :

TSK :

Conclusion :
TSK still has an issue, the heat flux reacts to this modification but is not the cause.

[mathieulandreau]

Dudhia,
Thank you for this information.
Do you have an idea of which variables should be saved ?
Mathieu

[dudhia]

I think TSK, SWDOWN and GLW would be the main ones as inputs to some physics. These seem to need allocating and deallocating too like the others.

[mathieulandreau]

Ok, I will try this tomorrow.
Mathieu

[mathieulandreau]

Hi again,

I tested every 2D variable and 1st level of every 3D variable from my wrfout file. Here is the result :

Directly affected variables : albedo, canwat, emiss, hfx, glw, pblh, tke_pbl (from MYJ), u10e, ust, v10e
Indirectly affected variables : grdflx, lh, noahres, th2, thm, q2, qfx, qvapor, qflx, sfroff, thm, tsk, u10, u, v10, v

Discussion :

I might save all the directly affected variables. However, these are only the history variables. I can't say if some internal variables are also affected by the bug.

Is it possible that my first guess is right (2nd message of this thread). Is it really necessary to call phy_init subroutine for the parent start_domain ? Here is the code I mentioned in start_em.F

lines 230-239 : A first test is performed based on the timestep but then the timestep is reset to 0

   IF ( grid%itimestep .EQ. 0 ) THEN
      first_trip_for_this_domain = .TRUE.
   ELSE
      first_trip_for_this_domain = .FALSE.
   END IF

   IF ( .not. ( config_flags%restart .or. grid%moved ) ) THEN
       grid%itimestep=0
   ENDIF

lines 316-321 : The same test is performed but since the timestep had earlier been reset to 0 it might be wrong ?

   IF ( grid%itimestep .EQ. 0 ) THEN
      first_trip_for_this_domain = .TRUE.
   ELSE
      first_trip_for_this_domain = .FALSE.
   END IF

lines 898-900 : The call of phy_init depends on this second test.

   IF ( first_trip_for_this_domain ) THEN

   CALL wrf_debug ( 100 , 'start_domain_em: Before call to phy_init' )

[mathieulandreau]

I tried to save all the variables mentioned above. The problem is clearly less important than at the beginning but still remains.

[dudhia]

Did you try all those tests you listed? I am interested to know what happened, because it is possible we have some unnecessary initializations for this case of nests starting later and no feedback.

[mathieulandreau]

For the moment I tested :

1- without swint_opt
-> no change
2- saving GLW
-> big change
3- saving GLW, QFX, HFX
-> no noticeable change on TSK, corrects these variables
4- saving GLW, QFX, HFX, ALBEDO, CANWAT, EMISS, PBLH, TKE_PBL, U10E, UST, V10E
-> no noticeable change on TSK, corrects these variables

I didn't try to avoid the call of phy_init or to avoid the call of start_domain(parent)

Mathieu

[dudhia]

I would suggest adding SWDOWN when swint_opt is not used as it has a similar role to GLW and will not get recomputed until the next radiation call.

[mathieulandreau]

Oh yes I forgot SWDOWN and TSK in my 4th test description but I also saved them :

     save_glw          = parent%glw !ML Issue #1947
     save_hfx          = parent%hfx !ML Issue #1947
     save_qfx          = parent%qfx !ML Issue #1947
     save_tsk          = parent%tsk !ML Issue #1947
     save_swdown       = parent%swdown !ML Issue #1947
     save_albedo       = parent%albedo !ML Issue #1947
     save_canwat       = parent%canwat !ML Issue #1947
     save_emiss        = parent%emiss !ML Issue #1947
     save_pblh         = parent%pblh !ML Issue #1947
     save_tke_pbl      = parent%tke_pbl !ML Issue #1947
     save_u10e         = parent%u10e !ML Issue #1947
     save_ust          = parent%ust !ML Issue #1947
     save_v10e         = parent%v10e !ML Issue #1947

Also tests 2, 3 and 4 are computed using swint_opt

However, as I mentioned, I cannot know if some "not-in-history" variables are also affected. And the variables might depend on the physical schemes selected.