Update CMIP6, scam cases to use netcdf3 #3148
Conversation
Update CMIP6 1850 and 20TR cases with and without BGC to use atm input files in netcdf3 format. Also update 2 scam cases.
Remove setting of PIO_TYPENAME to netcdf. Don't need that now that input files are in netcdf3 classic format. Allows pnetcdf to be used.
|
This replaces PR #3053 |
|
@cameronsmith1 your update to SSP585 could be added to this branch. |
|
Which SSP585 update are you referring to? |
|
I don't see any changes to the ELM configuration for the ELM files. Will ELM changes be in a different PR? |
|
I guess I didn't find any netcdf4-formatted files in the ELM use-cases for these compsets. |
|
Are netcdf3 '64bit offset' files OK? Or, do they have to be netcdf3 'classic'? All of the restart files I added for SSP585, including the ELM default file I recently added to Blues, are netcdf3 '64 bit offset' files. |
|
netcdf3 '64bit offset' are OK |
|
In that case, I don't think there are any netcdf changes needed from me. |
Hang-on. I noticed that you mentioned the emissions files are NetCDF4 in the Jira task. Those files were not added recently. However, I'll check them out once Cori comes back up. |
|
Is there an easy way to: |
Yes all the emissions files in inputdata/atm/cam/chem/trop_mozart_aero/emis/CMIP6_SSP585_ne30 . You can access them on blues in /lcrc/group/acme/public_html. It looks like @hwangacme created them originally but your name was on them on blues. |
|
Hailong converted the emission files to Netcdf3 classic. I have moved them to Blues with their Netcdf4 counterparts in atm/cam/chem/trop_mozart_aero/emis/CMIP6_SSP585_ne30. I have also fixed the various permissions. |
|
Presumably we still need to change the SSP585 xml file to point to the new netcdf3 files. Will you do that, @rljacob ? |
Update CMIP6 SSP585 case to use netcdf3 input
|
@cameronsmith1 that was done. Keep an eye on the maint-1.0 test on the dashboard. It should have no diffs after this. |
|
If I am reading the CDash maint-1.0 results correctly, the output is the same and the namelists are different, which is I would expect. |
|
Caused NML diff for the following tests Request submitted to bless the DIFFs. |
|
Is there a way to see what the diffs are on CDash? (I couldn't find them) |
|
Yes, @cameronsmith1. And it is helpful these tests are on cori-knl that we all have access. The diffs are the aerosol, oxid and solar data files on c-timestamp. To locate the nml files, from CDash, click on the specific test, you can see the paths for the test and the corresponding baseline (toward the end of page). NML files for the baseline are under CaseDocs subdir. |
| @@ -1,7 +1,6 @@ | |||
| #!/bin/bash | |||
| ./xmlchange --append CAM_CONFIG_OPTS='-cosp' | |||
| ./xmlchange --id BUDGETS --val TRUE | |||
| ./xmlchange PIO_TYPENAME=netcdf | |||
There was a problem hiding this comment.
This update to maint-1.0 will be brought forward to master later.
This update didn't make it to master yet, but current master can run SMS.ne30_oECv3_ICG.A_WCYCL1850S_CMIP6.anvil_intel with PIO_TYPENAME=pnetcdf without any errors if PIO_VERSION=2 (and errors out with the current default of PIO_VERSION=1).
Jayesh (@jayeshkrishna), just checking: will the internal switch to read netcdf4 files continue?
Benchmarks show:
a. netcdf+scorpio+master is slower than pnetcdf+scorpio_classic+maint-1.0, and
b. pnetcdf+scorpio+master is faster than pnetcdf+scorpio_classic+maint-1.0.
If support for netcdf4 in scorpio will go away, then we probably can't have b. until the netcdf4-to-netcdf3 update on master.
There was a problem hiding this comment.
So master works with PIO_VERSION=2 because it does a better job of quietly switching to netcdf4 when a netcdf4 file is opened?
There was a problem hiding this comment.
Yes. There is an info note in e3sm.log:
0: PIO: WARNING: Opening file (/home/ccsm-data/inputdata/atm/cam/chem/trop_mozart_aero/emis/DECK_ne30/cmip6_mam4_num_a4_elev_1850-2014_c170525.nc) with iotype=1 failed. Switching iotype to PIO_IOTYPE_NETCDF
And file type is
$ ncdump -k /home/ccsm-data/inputdata/atm/cam/chem/trop_mozart_aero/emis/DECK_ne30/cmip6_mam4_num_a2_elev_1850-2014_c170525.nc
netCDF-4 classic model
Pio2 update Update to the latest pio2 development version 2.4.4, set pio2 as default (thought I had already done this) Test suite: scripts_regression_tests.py Test baseline: Test namelist changes: Test status: bit for bit Fixes User interface changes?: Update gh-pages html (Y/N)?: Code review:
For maint-1.0, update CMIP6, scam cases that were using a netcdf4 file to instead use the netcdf3 version.
This will allow pnetcdf3 to be used for all these cases.
Only atmosphere files were changed.
CMIP6 1850 and 20TR cases with and without BGC were updated. Also SSP585 and 2 scam use cases.
This update to maint-1.0 will be brought forward to master later.
[BFB]