CF metadata and standardization of model grids

[BrendanMD]

Hi. I am a long time user of both CMIP[5,6] ocean model output and MPIMs Climate Data Operators cdo. With CMIP6, the "correct" decision was made to release ocean model data on the native ocean model grid. But this decision leaves users of the CMIP6 ocean data with no tools to correctly interpolate and rotate vector components from the U and V grid points to geographic coordinates and directions. The weights required to do these calculations are not included as CMIP6 first tier variables.

This led me here to ask if model grid metadata has ever been considered for inclusion in CF? Or if anyone here knows of any "global" efforts to produce a metadata standard for geophysical model grids. There has been recent work on generalized vector interpolation methods e.g. the work of Grady B. Wright that could be used to "automate" the generation of vector interpolation/rotation weights. Over a decade of more ago, there was a (French?) project called Komodo, the purpose of which was to standardize ocean model grids, but I can't find a trace of it, suggesting it died on the table. A quick goggle for "ocean model grid standards" turned up the following 2006 paper on grid standards.

Is this a fool's errand? Because a grid standardization and its inclusion in CF appears to me to be a somewhat fundamental problem worth addressing.

[taylor13]

I won't comment directly on accommodating additional grid metadata in CF, but I can summarize some of the CMIP6 discussion about regridding.

When planning CMIP6, we recognized that unfamiliar native grids (namely, all but latxlon cartesian grids) would pose difficulties for all but the most sophisticated users. There was discussion of requiring such data to be regridded to a latxlon grid. This was ruled out because of the inevitable loss of information. Instead we hoped to archive weights that could be used to regrid the native-grid data to a latxlon grid of the data provider's choice. This hasn't happened (yet, at least) because I haven't had time to work out the details of how the weights should be stored. There would really be no needed changes to CF, since the weights could be stored as variables, and CF already accommodates gridded data. In any case, this would only meet your requirements if modeling groups interpolated and rotated the vector fields in a way that meets any constraints you think are important (e.g. preserving area means, conserving vorticity, etc.).

[BrendanMD]

Thanks for your reply taylor13. I think the danger is that inexperienced users (users with no experience with geophysical models and their data) are likely using the ocean vector fields willy-nilly without consideration for the proper direction. This does concern me, but not nearly as much as having to use pre-interpolated fields. Perhaps modeling groups should have to provide sample code (in the programming language of their own choice), that illustrates how they themselves perform their own vector rotation/interpolation, general scalar field interpolation, and vertical interpolation/integrations?

In any event, this is the CF github not, CMIP. However, if I may make one last barrage of comments on CMIP, because I have wanted to pass my feedback up the CoC and the CMIP guidance site does not state how one may do so. My tenure as a CMIP data user has made it obvious that there is a gigantic gap between the data creators (modeling groups) and the data users. The IT people who run the ESGF portals get a disproportionate number of questions about the data itself and waste their time answering them. I hope CMIP7 funds will be allocated for a data user help desk i.e. people hired to help guide data users in the proper use of the data and help them navigate the CMIP "experience" - not simply to help them download the data. I think there is a definite need. Has the CMIP planning process ever considered gathering feedback from the actual data users?

I wrote over one hundred emails over the past 18 months simply trying to get answers from modeling groups and/or to report that their metadata was incorrect, broken and/or non-compliant. This made it obvious that not enough resources were allocated to aid scientists in hiring staff to specialize in the conversion to CMIP compliant formats. I suspect that smaller modeling groups and non-english speaking modeling groups suffered disproportionately.

It is great to see the "user experience" that emerged with CMIP6, to witness the iterations that have made the guidance docs better organized, the data easier to access and the choice of native model coordinates. Thanks so much for your contributions in this effort. Regards, Brendan.

[taylor13]

Agreed, this is not a CF issue nor the place for a CMIP discussion, but many CF aficionados have an interest in CMIP, so I encourage those reading this thread to consider responding to the survey announcement copied below. @BrendanMD -- you might want to copy some of your above comments into the survey.
Thanks for your interest and good suggestions.
Karl

---

Dear colleagues

The WCRP Working Group on Coupled Modelling (WGCM) and the CMIP Panel are already working hard in preparation for the next phase of Coupled Model Intercomparison Project (CMIP). Specifically, we need to hear from everyone their views on the positive aspects and challenges they faced in CMIP6, and how they would like to see future phases of CMIP structured and delivered.

For that, we are inviting the community to take part in the “CMIP Community next steps survey”. It is your chance to help us shape the future of CMIP!

https://www.wcrp-climate.org/cmip-survey

We strongly encourage you to share this survey as broadly as you can. We are in particular very keen to have responses from users of the data, from the academic and policy-making world.

Deadline for completing the survey: 28 February 2022

Thanks for helping us in making CMIP better!

WCRP Secretariat

[zklaus]

You bring up good points, @BrendanMD, and you are certainly not alone in your assessment.

A number of initiatives deserve mention here.

• Es-doc tries to provide documentation for many things around CMIP. The main page is, unfortunately, a bit hard to navigate, but you might be interested in all the information that can be learned about models in general and the grids they use in particular from https://explore.es-doc.org and in the errata service that your email may well have contributed to at https://errata.es-doc.org. Other parts deal with other aspects of CMIP, like a more detailed description of the experiments.
• The CMIP6 data request provides information about some of the aspects you mention, for example in the spatial dimensions section, that is also compounded by a number of guidance documents linked from the participation guidance for modelers. I find myself particularly often returning to the section on requirements for coordinate variables in the succinctly named Specifications for output file content, structure, and metadata. There is also a github organization dedicated to the CMIP6 data request: https://github.com/cmip6dr/
• Looking to CMIP7, you may be interested in this whitepaper describing plans for the next iteration of the data request.

Furthermore, there are a few ongoing efforts approaching these issues from the user side. One of them is a Copernicus project that tries to correct faulty or incomplete metadata on-the-fly at the time of downloading from ESGF or the Copernicus Data Store. Unfortunately, I was not able to quickly find the online information.
Another is the ESMValTool that has collected a database of defects in CMIP5, CMIP6, and a few other supported data sources and integrated this together with fixes into its preprocessor (see this description of the mechanism). Full disclosure: I am one of the developers of ESMValTool.

In spite of all of that, I think it would be useful to have a simple registry of model grids in use with unique identifiers that would permit easy handling of regridding and rotation operations, particularly for CMIP6 and earlier, where really only a handful of different grids seem to be in use. I fear that this approach will become less useful in the future as unstructured grids and perhaps even dynamic grids become more commonplace.

[BrendanMD]

Survey completed. Thanks for your responses.