Using cfgrib to open Stage-IV rainfall data

[edougherty32]

Hi,

I was wondering if it's possible to use cfgrib to open Stage-IV GRIB rainfall data (downloaded here). I attempted this in Jupyter notebook:

cfgrib.open_dataset('/glade/scratch/doughert/stage_iv/2008/ST4.2008031900.01h')

And received the following error:

EcCodesError: ('Key/value not found (-10).', -10)

During handling of the above exception, another exception occurred:
..
KeyError: 'latitudes

I am guessing this is due to the structure of the Stage-IV data, which is seen by opening the dataset via the usual xarray function:

xr.open_dataset('/glade/scratch/doughert/stage_iv/2008/ST4.2008031900.01h', engine='pynio')

<xarray.Dataset>
Dimensions:                 (g5_x_0: 881, g5_y_1: 1121)
Coordinates:
    g5_lat_0                (g5_x_0, g5_y_1) float32 ...
    g5_lon_1                (g5_x_0, g5_y_1) float32 ...
Dimensions without coordinates: g5_x_0, g5_y_1
Data variables:
    g5_rot_2                (g5_x_0, g5_y_1) float32 ...
    VAR_237_GDS5_SFC_acc1h  (g5_x_0, g5_y_1) float32 ...

Where the coordinates and variables aren't the usual "latitude" and "longitude" I've seen in the examples of cfgrib so far. So does this mean this package would be unusable for Stage IV data?

[rabernat]

Perhaps related to #17?

There are many different flavors of grib files out there in the wild. It would be good to be clear about which ones that cfgrib is committed to supporting.

[alexamici]

@edougherty32 looks like I assumed ecCodes could provide geographic coordinates for gridType==polar_stereographic but didn't test it. And obviously it doesn't.

In current master the "Precipitation NCEP/EMC 4KM Gridded Data (GRIB) Stage IV Data" GRIB files now don't crash cfgrib, but they are opened with no geographic support 😞. Please test them and report if it fixes your problem. I will probably add a different issue to track gridType support by ecCodes but that would be a wontfix for cfgrib as we intend to rely on the underlying library for these things.

@rabernat regarding coordinate systems support the commitment is the one in the README:

Limitations:

• ...
• relys on ecCodes for the gridType handling.

More in general we aim of cfgrib is quite simple: to support all GRIB files that

• can be decoded by ecCodes (back-end)
• can be represented in xarray (front-end)

[alexamici]

@edougherty32 as a quick update: I had a quick conversation with the ecCodes developers and looks like polar_stereographic is supported, just not by the interface I am using. We are assessing what is the best way to proceed.

[edougherty32]

@alexamici Thanks for the update. For now, I will stick with the

xarray.open_dataset()

function, setting the engine to 'pynio' to read in the Stage-IV rainfall.

However, I look forward to any developments on the use of cfgrib to work with Stage-IV files, as it might be more efficient than just using xarray.

[alexamici]

Next release of ecCodes will enable polar_stereographic for cfgrib. We still do not have a strategy on what to do on system with old ecCodes but we track progress in #28 .

Closing the issue.