new compression AEC

[edwardhartnett]

Hi Edward and Eric,

I got a bounce back from Boi's email address.

Do either of you know who is the current maintainer of the NCEPLIBS-g2c library?

Thanks
Arthur

On 6/28/2023 10:18 AM, Arthur Taylor wrote:
Hi Boi,

Are you still in charge of the NCEPLIBS-g2c library?

It looks like ECMWF is using a new compression scheme. Do you know if the g2c library supports it? If not, do you know if there are plans to support it?

Thanks,
Arthur

-------- Forwarded Message --------
Subject: ECMWF Changes
Date: Wed, 28 Jun 2023 14:39:10 +0100
From: Karl Devine karl@netweather.tv
To: Arthur.Taylor@noaa.gov, Michael.Allard@noaa.gov

Hi,
Sorry to be a pain but ECMWF have made some big changes so degrib no longer works with those files:

$filename="$dbdate$dbtime"."0000-$ext"."h-oper-fc.grib2";
$ftpdir="https://data.ecmwf.int/forecasts/$dbdate/$dbtime"."z"."/0p4-beta/oper";

The changes are detailed at:
https://confluence.ecmwf.int/display/FCST/Implementation+of+IFS+Cycle+48r1

I think it's mostly compression related but i get the following error:

getdrstemplate: DRS Template 5.42 not defined.
ERROR: In call to Grib2Convert.
ERROR: In call to ReadGrib2Record.
ERROR: Unpack library error code (500 2)

I do hope it's possible to apply these changes to degrib and wgrib2 .

Many thanks

Karl De Vine

[edwardhartnett]

@EricEngle-NOAA is this the compression you desire in the C library?

Of course this should not be implemented in Fortran, but in C. Once present in the C library, we can have wrapper functions in Fortran to make it available there.

[edwardhartnett]

OK, this is implemented in the C library. In the 3.5.0 release of g2, we will allow for this compression to be used from Fortran.

[edwardhartnett]

Sorry, this has to be bumped to the next release...

[EricEngle-NOAA]

@edwardhartnett - I can work on implementing AEC compression in g2.

IMO the way this should be done is to interface to the g2c functions which would make g2c a required dependency for g2. I know that is what we want to do anyways down the road, but I do not if that is something you want to do at this time.

The work I am doing on NOAA-EMC/NCEPLIBS-g2c#461 is basically a primer to this.

[AlexanderRichert-NOAA]

For what it's worth, this is something I was looking at a few weeks back, specifically, interfacing to the g2c aec functionality. The tricky part is getting the C/fortran bindings right in terms of pointer handling etc. I've gotten as far as having a buildable draft of the code but haven't had time to set up a compression/decompression test.

[EricEngle-NOAA]

Here is the interface that was working in my test program:

NOAA-EMC/NCEPLIBS-g2c#461 (comment)

[edwardhartnett]

Guys,

Do we have a PR for this?

Or can I ask @AlysonStahl-NOAA to take a look?

[EricEngle-NOAA]

@edwardhartnett - From previous conversations, it seems @AlexanderRichert-NOAA has done some work and I have done some work as well, but no formal PR at this time it seems.

[AlexanderRichert-NOAA]

See #739 for work in progress