CESM History File Parser Issue - CISM component

[mgrover1]

When the file parser attempts to read in the CISM component of CESM, the following error is returned when it reads in the file in xarray

unable to decode time units 'common_year since 1-1-1 0:0:0' with "calendar 'noleap'".

Using decode_times = false in xarray, where the time values are now a float n equal to "common_year since 0000-01-01 0:0:0"

Would it make sense to add a try, except block, and allow the parser to feed "decode_times = false" in? I am wondering if this error would propagate downstream in the analysis...

[mgrover1]

So it looks like the parser is using decode_times = False, the problem comes with the bounds... there are two time variables with CISM.

• time which is the model time, in common_year since 000-01-01 0:0:0
• internal_time which is the model time - internal representation in common_year since 1-1-1 0:0:0

The data starts with year 2, where for year 2 the time variable is equal to 2, and internal_time is equal to 1.

The data appear to instantaneous as opposed to representative over some time period... would it be best to skip this then? It wouldn't have time bounds.

[andersy005]

unable to decode time units 'common_year since 1-1-1 0:0:0' with "calendar 'noleap'".

It's my understanding the calendar units used here are also part of the problem (if you wanted to open the dataset with decode_times=True)... I don't think common_year is a standard reference that cftime understands.

[andersy005]

@mgrover1, is there any explanation as to why they use this reference instead of say days since ....?

[mgrover1]

unable to decode time units 'common_year since 1-1-1 0:0:0' with "calendar 'noleap'".

It's my understanding the calendar units used here are also part of the problem (if you wanted to open the dataset with decode_times=True)... I don't think common_year is a standard reference that cftime understands.

So I tried manually changing the calendar units from common_year to years since, writing out a netcdf file then reading it back in, but the issue still persists...

@mgrover1, is there any explanation as to why they use this reference instead of say days since ....?

I am not sure why they do not use a cf-standard unit like days since or years since. Within the CESM-postprocessing repo @sherimickelson included a way of dealing with this when reading files in using the netcdf4 library. But this might be a model-output issue.

[sherimickelson]

@Katetc or @billsacks might be able help with input on this issue as it might be best to coordinate with them on the solution.

[billsacks]

I dug back into some emails from 2016, when this issue first arose. A number of us spent a HUGE amount of time on this 5 years ago, with many back and forths about what is and isn't acceptable, given what units are and aren't supported by various tools.

There are a few separate but related issues here:

1. year vs. day, second, etc.: From skimming back through the relevant email threads (I have close to 100 old emails on this topic, so I haven't read back through everything carefully), I believe that – at the time – the CISM developers preferred to stick with a unit of years, if possible. The rationale was that there were various post-processing scripts that assumed units of years, and the time scale of CISM runs can be so long (tens or hundreds of thousands of years) that units of years can be easier to work with. That said, my impression is that they were willing to switch if it would help; it's partly that the conversations at that time came at a bad time for them as they were in crunch mode trying to get a release out the door. Revisiting that conversation could be reasonable now.
2. If we use units of years, whether to use "years" or "common_years". My recollection is that "common_years" is cf-compliant, even though some python packages don't support it. @kmpaul opened an issue to get it supported (Full support for UDUNIT years units Unidata/cftime#5), but it looks like there hasn't been any movement there. I don't think the CISM developers care one way or another, but we were asked to change from "years" to "common_years" because some tools don't know how to deal with a unit of "years", or maybe the definition of "years" differs from what you would expect. Ah, there's some information if you search for "common_year" here: http://cfconventions.org/cf-conventions/cf-conventions.html which I think is what drove us to use "common_year" instead of "year".
3. I forget the details of why I needed to introduce a separate "internal_time" variable, but I found an email explaining that this was, indeed, needed (at least if I wanted to avoid completely overhauling CISM's time management). For the purpose of post-processing, use "time", not "internal_time".

[andersy005]

@billsacks, thank you for the clarification, and pointers!

. @kmpaul opened an issue to get it supported (Unidata/cftime#5), but it looks like there hasn't been any movement there.

I just submitted a PR upstream Unidata/cftime#246 that tries to address the issue @kmpaul opened.

[billsacks]

Awesome, thank you @andersy005 !

All: Please let us know if CISM's time units are causing any additional issues.

[andersy005]

@mgrover1, Unidata/cftime#246 got merged. This issue should be resolved in the next release of cftime which I think is going to be v1.5.1