Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add fields for CMIP6_Emon/Eyr table (incremental) #186

Closed
lca041 opened this issue May 30, 2020 · 12 comments · Fixed by #216
Closed

Add fields for CMIP6_Emon/Eyr table (incremental) #186

lca041 opened this issue May 30, 2020 · 12 comments · Fixed by #216
Assignees
@YanchunHe @lca041

Comments

@lca041
Copy link

lca041 commented May 30, 2020

The Following fields requested by LUMIP are now supported by regridding and extracting variables into separate output files:

Fields in monthly-averaged output (clm2.h2, which stores landunit-level variables)

'gppLut      ','GPP
'raLut         ','AR
'nppLut      ','NPP
'cTotFireLut      ','COL_FIRE_CLOSS
'rhLut      ','HR
'tasLut      ','TSA
'tslsiLut      ','TSKIN
'hussLut      ','Q2M
'hflsLut      ','EFLX_LH_TOT
'hfssLut      ','FSH
'rsusLut      ','FSR
'rlusLut      ','FIRE
'laiLut      ','TLAI
'mrsosLut      ','SOILWATER_10CM
'mrroLut        ','QRUNOFF
'mrsoLut       ','TOTSOILLIQ
'irrLut            ','QIRRIG
'fahLut          ','URBAN_HEAT

Regridded file for CMORization
Under /nird/projects/NS9560K/noresm/cases/b.e21.NHISTfrc2.f19_tn14-deforest-globe.001/lnd

b.e21.NHISTfrc2.f19_tn14-deforest-globe.001.clm2.Emon.1850-01.nc

Fields in monthly-averaged output (clm2.h1, which stores pft-level variables)


'cVegTree       ','TOTVEGC
'cVegShrub       ','TOTVEGC
'cVegGrass        ','TOTVEGC
'gppTree           ','GPP
'gppShrub           ','GPP
'gppGrass           ','GPP
'nppTree           ','NPP
'nppShrub           ','NPP
'nppGrass           ','NPP
'raTree           ','AR
'raShrub           ','AR
'raGrass           ','AR
'vegHeightTree           ','HTOP
'vegHeightShrub           ','HTOP
'vegHeightGrass           ','HTOP

Regridded file for CMORization
Under /nird/projects/NS9560K/noresm/cases/b.e21.NHISTfrc2.f19_tn14-deforest-globe.001/lnd

b.e21.NHISTfrc2.f19_tn14-deforest-globe.001.clm2.Emonpft.1850-01.nc

Fields in year-end output (clm2.h4, which stores landunit-level variables)

'cSoilLut        ','TOTSOMC
'cVegLut        ','TOTVEGC
cLitterLut       ','TOTLITC

Regridded file for CMORization
Under /nird/projects/NS9560K/noresm/cases/b.e21.NHISTfrc2.f19_tn14-deforest-globe.001/lnd

b.e21.NHISTfrc2.f19_tn14-deforest-globe.001.clm2.Eyr.1850-01-01-00000.nc
@YanchunHe YanchunHe self-assigned this Jun 2, 2020
@YanchunHe YanchunHe changed the title Add fields for CMIP6_xx table (incremental) Add fields for CMIP6_Emon/Eyr table (incremental) Jun 17, 2020
@YanchunHe
Copy link
Collaborator

Hi @lca041 ,

Sorry for being slow. Finally, I managed to test all the required variables. A few issues are identified and need to be addressed in your preprocessed files:

EmonLut and Emonpft

  • Put all the preprocessed files under the same folder, e.g., /projects/NS9560K/noresm/cases/b.e21.NHISTfrc2.f19_tn14-deforest-globe.001/LUMIP/b.e21.NHISTfrc2.f19_tn14-deforest-globe.001/lnd/hist
  • You need to write the ‘time’ dimension as ‘unlimited’ dimension. You can achieve this during you write the file with NCL or by nco command afterwards, e.g., : ncks --mk_rec_dmn time infile.nc outfile.nc
  • The time_bounds variable for time is missing. Should include it as a variable in the file.
  • Use the ‘landUse’ instead of ‘landuse’ as the dimension name. e.g., with NCO, e.g., ncrename -d landuse,landUse infile.nc
  • Set units of rlusLut as W m-2. It is 1 now. ncatted -a units,rlusLut,m,c,W m-2 file.nc
  • Keep original variable name in the preprocessed data. In this way, the file meta data original model name in the processed data reflects the real original model variable name. I see some fields are derived from the same model variable. For such case, you can use for example, GPP_Tree for gppTree, GPP_Shrub for appShrub; HTOP_Tree for vgHeightTree and HTOP_Shrub for vgHeightShrub, etc.
  • Can you put all Emon variables in files with clm2.h0 tag, and put all Eyr in files with clm2.h4 tag?
  • The following variables have wrong dimension names, should use latitude/logintitude as the others
    • float vegHeightGrass(ncl7, ncl8) ;
    • float vegHeightShrub(ncl5, ncl6) ;
    • float vegHeightTree(ncl3, ncl4) ;

Update the CMIP6_data_request_v1.00.30beta1 Google Sheet file

  • in Column E, add original model name, e.g., GPP.
  • In column F, add short notes at the beginning on how it is derived, e.g., LC: derived from GPP on pft levels for Shrub; LC: derived GPP on land unit-level
  • in column I, add how this output being activated;
  • in Column K, tick the box that they are able to be cmorized.

Sample modified files are found under:
- /projects/NS9560K/noresm/cases/b.e21.NHISTfrc2.f19_tn14-deforest-globe.001/LUMIP.bak)

Sample cmorized files are found under:
- /projects/NS9034K/CMIP6/.cmorout/NorESM2-LM/deforest-globe/v20200603
(Note, as the time_bounds is missing, the cmorized output file for year 1850 is *_gn_1849-1849.nc. and its date is: 1849-12-16 12:00:00, which is correct.)

Please make just one-year sample preprocessed file, put them all under: /projects/NS9560K/noresm/cases/b.e21.NHISTfrc2.f19_tn14-deforest-globe.001/LUMIP/b.e21.NHISTfrc2.f19_tn14-deforest-globe.001/lnd/hist. I will test them again. Finally, you can preprocessed all other files and experiments, and then I will postprocess them.

Thanks!

@lca041
Copy link
Author

lca041 commented Jun 21, 2020 via email

@YanchunHe
Copy link
Collaborator

For the veg-height variables (vegHeightGrass, vegHeightTree, and vegHeightShrub), they are not key variables, and there are some other CMIP6 models that are without these variables available on ESGF.

That is OK, it is up to you.

But this issue looks quite straightforward to solve, as you can assign a time-dimension to it, and just put them latitude/latitude dimension. But you decide.

I will test this again. If everything looks OK, you can preprocess all other data.

@lca041
Copy link
Author

lca041 commented Jun 23, 2020 via email

@YanchunHe
Copy link
Collaborator

This is still problem with the time-axis of the Eye variables.

$ cdo showtimestamp b.e21.NHISTfrc2.f19_tn14-deforest-globe.001.clm2.h4.1850-01-01-00000.nc

  1850-01-01T00:00:00

$ ncdump -v time_bounds b.e21.NHISTfrc2.f19_tn14-deforest-globe.001.clm2.h4.1850-01-01-00000.nc
 time_bounds =
  -0.0208333333333333, 0 ;

$ cdo showtimestamp b.e21.NHISTfrc2.f19_tn14-deforest-globe.001.clm2.h4.1851-01-01-00000.nc
Warning (cdfCheckVars): Unsupported data type (char/string), skipped variable date_written!
Warning (cdfCheckVars): Unsupported data type (char/string), skipped variable time_written!
  1851-01-01T00:00:00

$ ncdump -v time_bounds b.e21.NHISTfrc2.f19_tn14-deforest-globe.001.clm2.h4.1851-01-01-00000.nc
 time_bounds =
  0, 365 ;


$ cdo showtimestamp b.e21.NHISTfrc2.f19_tn14-deforest-globe.001.clm2.h4.1850-01-01-00000.nc
  1850-01-01T00:00:00
$ ncdump -v time_bounds b.e21.NHISTfrc2.f19_tn14-deforest-globe.001.clm2.h4.1850-01-01-00000.nc
 time_bounds =
  -365, 0 ;
}

$ cdo showtimestamp b.e21.NHISTfrc2.f19_tn14-deforest-globe.001.clm2.h4.1851-01-01-00000.nc
  1851-01-01T00:00:00

ncdump -v time_bounds b.e21.NHISTfrc2.f19_tn14-deforest-globe.001.clm2.h4.1851-01-01-00000.nc

 time_bounds =
  0, 365 ;
}

Looks like the timestamp is not correct.

However, even if I shift the date from 1850-01-01 to 1850-06-30.

$ cdo showtimestamp b.e21.NHISTfrc2.f19_tn14-deforest-globe.001.clm2.h4.1850-01-01-00000.nc
  1850-06-30T00:00:00

The problem still exist in the cmor tool.

 fnm:
 /projects/NS9560K/noresm/cases/b.e21.NHISTfrc2.f19_tn14-deforest-globe.001/LUMI
 P/b.e21.NHISTfrc2.f19_tn14-deforest-globe.001/lnd/hist/b.e21.NHISTfrc2.f19_tn14
 -deforest-globe.001.clm2.h4.1851-01-01-00000.nc
 fnm:
 /projects/NS9560K/noresm/cases/b.e21.NHISTfrc2.f19_tn14-deforest-globe.001/LUMI
 P/b.e21.NHISTfrc2.f19_tn14-deforest-globe.001/lnd/hist/b.e21.NHISTfrc2.f19_tn14
 -deforest-globe.001.clm2.h4.1852-01-01-00000.nc
 fnm:
 WARNING: no file found for case dir|tag|year1|month1|yearn|monthn:
 /projects/NS9560K/noresm/cases/b.e21.NHISTfrc2.f19_tn14-deforest-globe.001/LUMI
 P/b.e21.NHISTfrc2.f19_tn14-deforest-globe.001|clm2.h4|        1850 |
           1 |        1851 |          12

C Traceback:
In function: cmor_close
!

**That is, the cmor tool can not correctly extract the date information, and regard the data file falling between, say, 1850-01 to 1850-12, as I set in the test.

@YanchunHe
Copy link
Collaborator

I need to look into this.

@YanchunHe
Copy link
Collaborator

YanchunHe commented Jun 25, 2020 •
edited
Loading

The "time" axis and its "time_bounds" in your preprocessed clm2.h4 files are still not correct. The 'time' is at the first day of the year, and the time_bounds actually specify the date as it one year earlier. For example, 1850-01-01 covers 1949-01-01 to 1949-12-31.

So I am afraid, for example, your 1850 year data actually falls to 1849, and hence onwards.

I tried to fix the time and time_bounds, but it still not work well with the noresm2cmor tool (in subroutine get_file_info to get correct tbnd and mbnd).

Therefore, if you need to cmorize data up to 1930, you should have data file clm2.h4.1931-01-01.

Otherwise, if you do think the covered period by the file is correct, I suggest a quick fix to shift the time bounds on year towards the right of the axis:

Therefore, I suggest a quick fix:
ncap2 -O -s 'time_bounds += 365' infidel.nc outfile.nc

examples of cmorized files under:
/projects/NS9034K/CMIP6/.cmorout/NorESM2-LM/deforest-globe/v20200603/

@lca041
Copy link
Author

lca041 commented Jun 25, 2020 via email

@YanchunHe
Copy link
Collaborator

YanchunHe commented Jun 26, 2020 •
edited
Loading

Hi, thanks!

But still, even the original output have wrong "time_bounds".

For example,

$ ncdump -v time,time_bounds b.e21.NHISTfrc2.f19_tn14-deforest-globe.001.clm2.h4.1851-01-01-00000.nc

data:

 time = 365 ;

 time_bounds =
  0, 365 ;
}

which means, the 1851 data actually covers 1850-01-01 to 1851-01-01.

Or we can interpret that, the model meant to accumulate and write all data of 1850 to 1851-01-01-00000.nc at the end of 1850, that is 1851-01-01:0000. So the timestamp of the file denotes the last point of accumulation (and then averaging) time point, instead of the intended average period, ie.g., average of 1851.

In the cmor tool, it uses the actually time defined by the 'time' and 'time_bounds'. Therefore, in the sample cmorized output, for example, cLitterLut_Eyr_NorESM2-LM_deforest-globe_r1i1p1f1_gn_1850-1859.nc, its first slice for 1850, actually use the 1851-01-01 model output. You can compare with the first slice of the first data with your 1851 model output, they are identical.

In short, I suspect the timestamp in the model output represent the average of its previous year. There, you should have 1931-01-01 data to cover the whole 1850-1930 period.

@lca041
Copy link
Author

lca041 commented Jun 26, 2020 via email

@YanchunHe
Copy link
Collaborator

Good!

@YanchunHe YanchunHe linked a pull request Jul 6, 2020 that will close this issue
Add support to cmorize pre-processed LUMIP experiments #216
Merged
@YanchunHe YanchunHe mentioned this issue Jul 6, 2020
Merged
@YanchunHe
Copy link
Collaborator

Just add a note: the cmorized data after this implement is labelled as v20200702

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@YanchunHe YanchunHe

@lca041 lca041

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Add support to cmorize pre-processed LUMIP experiments
2 participants
@YanchunHe @lca041