1-01-01 Addition of new Level I observed variables

[JohnEyre]

Proposal Summary

A class of variables is missing from WMDR. This is the class of Level I observed variables, such as radiances, brightness temperatures, reflectivities, backscatter coefficients, bending angles, etc. Observations of these variables are used directly in NWP and other applications within WMO programmes. As a consequence, WMO messages containing such data are widely disseminated. (In fact, in terms of data volumes, they dominate the observational data on the GTS). They do not appear in OSCAR/Requirements because requirements are stated in terms of equivalent Level II geophysical variables. In OSCAR/Space, the capabilities are stated both in these Level I terms and also in Level II terms for comparison with OSCAR/Requirements.

Although these variables do not appear in WMDR, similar variable names appear (to some extent) in other parts of the WMO codes database – in /grib2, /bufr4, etc.

The proposal, therefore, is to include Level I observed variables in WMDR. To contain the scope of the change, the proposal is to include only those variables that appear in transmitted WMO observation messages and in OSCAR. The list below is provisional and not exhaustive.

Summary and Purpose

Add new Level I observed variables

Proposal

Notation	Name	Description	Tags
12252	Radiance	The radiant flux emitted, reflected, transmitted or received by a given surface, per unit solid angle per unit projected area. It is a directional quantity. Note: Historically, radiance was called 'intensity'.
12253	Spectral radiance	The radiance of a surface per unit frequency or wavelength, depending on whether the spectrum is taken as a function of frequency or of wavelength. It is a directional quantity. Note: 'spectral radiance' is also used for measurements by radiometers and spectrometers, where it means the spectral radiance incident on the instrument integrated over the normalised spectral response function of the instrument. Historically, spectral radiance was called spectral intensity.
12254	Brightness temperature	Also called radiance temperature or equivalent black body temperature. The temperature of a black body emitting the spectral radiance equivalent to the observed spectral radiance at the observed frequency or wavelength.
12255	Spectral bidirectional reflectance	Ratio of reflected to incident radiation as a function of incident angle, reflected angle and wavelength. For a satellite measurement, these angles are the zenith and azimuth angles of the sun and the zenith and azimuth angle of the viewing instrument at the feature (surface and/or atmospheric column) being observed.
12256	Radar backscatter coefficient	Normalised radar cross section (NRCS) backscatter coefficient, also called sigma-0 ('sigma-nought'). The normalised measure of the radar return, measured in dB.
12257	Bending angle	The angle through which an electromagnetic wave is refracted when passing along an atmospheric path, used particularly for GNSS radio occultation observations.	Radio occultation
12258	Refractivity	The refractive index (of air) minus one. Particularly used for GNSS radio occultation observations.
12259	Slant total delay (STD)	The time delay of a GNSS radio signal passing along an atmospheric path from transmitter to receiver compared with the same path in vacuum.
12260	Zenith total delay (ZTD)	The time delay of a GNSS radio signal passing along an atmospheric path from transmitter to receiver compared with the same path in vacuum, and scaled to refer to the equivalent vertical path.
12261	Delay Doppler map	A 2D array of backscatter coefficients as a function of Doppler shift and signal delay, derived from GNSS reflectometry.
12262	Impact parameter	Minimum distance between the direction vector of the ray (prior to bending) and the centre of the curvature of the ellipsoid describing the Earth, used particularly for GNSS radio occultation observations.	Radio occultation

Reason

Addition of missing variables

Stakeholder(s)

@JohnEyre (JET-EOSDE)

Consultations

Chair of ET-SSU (Fiona Smith, BoM, Australia)

Context

[include references to manuals or guides that are reviewed to ensure alignment, if proposal differs then document how and why]

Expected Impact of Change

MEDIUM

[JohnEyre]

Revise "Backscatter coefficient" to read "Backscatter coefficient - Normalised radar cross section (NRCS) backscatter coefficient, also called sigma-0 (Units: dB)."

Add: "nnnnn - Impact parameter - (Radio occultation). Minimum distance between the direction vector of the ray (prior to bending) and the centre of the Earth." [The definition will need some revision because, for an ellipsoidal Earth, the exact definition should refer to the distance from the centre of curvature, not from the centre of the Earth.]

[meulenvd]

May be of interest w/r requirements for Level I data, as published in the WMO Tech Regs, (WMO-No. 49):

2.4.1.3 Members shall report observational data as observed variables, defined in Annex VIII (Manual on the WMO Integrated Global Observing System (WMO-No. 1160)). The data shall be expressed in environmental quantities in accordance with the standards of the International System of Units (SI).

Note: While observations are Level II data, in some circumstances it may also be valuable to record and share Level I data, from which the observations were derived. A further description of some of these circumstances is provided in
Annex VIII (Manual on the WMO Integrated Global Observing System (WMO No. 1160)) and related guidance material.

and Manual on the WIGOS (WMO-No. 1160):

2.4.4.7 Members should record and retain all Level I data used when making observations available internationally.

[amilan17]

https://github.com/wmo-im/tt-wigosmd/wiki/Meeting-2022.09.22 notes:

• @joergklausen will consolidate and update a branch

[amilan17]

https://github.com/wmo-im/tt-wigosmd/wiki/2022.10.06-TT-WIGOSMD meeting notes:

no progress yet

[joergklausen]

Branch created https://github.com/wmo-im/wmds/tree/414-1-01-01-addition-of-new-level-i-observed-variables.

@JohnEyre Please complete definition of 12250.

[joergklausen]

May be of interest w/r requirements for Level I data, as published in the WMO Tech Regs, (WMO-No. 49):

2.4.1.3 Members shall report observational data as observed variables, defined in Annex VIII (Manual on the WMO Integrated Global Observing System (WMO-No. 1160)). The data shall be expressed in environmental quantities in accordance with the standards of the International System of Units (SI).

Note: While observations are Level II data, in some circumstances it may also be valuable to record and share Level I data, from which the observations were derived. A further description of some of these circumstances is provided in Annex VIII (Manual on the WMO Integrated Global Observing System (WMO No. 1160)) and related guidance material.

and Manual on the WIGOS (WMO-No. 1160):

2.4.4.7 Members should record and retain all Level I data used when making observations available internationally.

2.4.1.3 doesn't seem to be in conflict with what we are doing here, all these level 1 measurements can be expressed in SI units (including 1 for ratios). The term 'environmental quantities' is weird, though.

[amilan17]

https://github.com/wmo-im/tt-wigosmd/wiki/2022.11.03-TT-WISOSMD notes:

• @JohnEyre - update definition for 12250 and 12258
• Then we get a review by satellite community.
• @amilan17 - email Heikki when ready

[joergklausen]

@JohnEyre Please reach out to Fiona Smith for external review and confirmation.

[JREyre]

@amilan17 As discussed, please could you make the following updates:
12250 – change “[Attn: parts missing]” to “spectral intensity”.
12258 – change to “A 2D array of backscatter coefficients as a function of Doppler shift and signal delay, derived from GNSS reflectometry.”

[JREyre]

@amilan17
Proposal referred for comment to Chair of ET-SSU (Fiona Smith, BoM, Australia) on 7 Nov 2022.

[amilan17]

@joergklausen @JREyre I updated the branch with changes that were recently adopted during FT2022-2 and changed the notation numbers. However, I did not update the descriptions to align with the issue summary yet. 

I think we should align some of the information in the descriptions with the units from our units table. For example, "The SI unit of radiance is watt per steradian per square metre (W·sr-1·m-2)." is in the description for radiance and it looks like the same unit as below from the [units](https://github.com/wmo-im/wmds/blob/414-1-01-01-addition-of-new-level-i-observed-variables/tables_en/1-02.csv) table, but slightly different. Noting in particular the order of the name and the use of a period dot "w.m" instead of the middle dot "w·m". 

W.m-2.sr-1

watt per square metre per steradian

SI derived unit of radiance; SI base unit equivalent: kg.s-3

There are other references to units that don't exist in our units table, such as "watt per steradian per square metre per hertz (W·sr-1·m-2·Hz-1)". Should we add units as part of these additions?

[JREyre]

@amilan17 @joergklausen
Thanks, Anna, for the link to the table of units. There are some problems here:

• 185 for radiance is OK.
• spectral radiance is often called "radiance" by practitioners - most of the data on the GTS are called "radiances" - but spectral radiance is more correct.
• 186 for spectral radiance per wavenumber is OK. (This is the one used by thermal infra-red people - people like me!) However it is normal to write the units as W.m-2.sr-1.(cm-1)-1, to make it clear that this is inverse wavenumbers (rather than centimetres).
• 187 should be spectral radiance per wavelength with units W.m-2.sr-1.m-1 = watts per square metre per steradian per metre. I think this makes the SI unit: kg.m-1.s-3. This one is more commonly used by people who work in the short wave, although they would normally use W.m-2.sr-1.nm-1 (nm=nanometer) because it gives more convenient values).
• 188 is then the same as the corrected 187, although you would never see it written like this - always W.m-2.sr-1.m-1 or W.m-2.sr-1.nm-1.

[joergklausen]

Proposal ready for external feedback.

[amilan17]

https://github.com/wmo-im/tt-wigosmd/wiki/2022.11.22-TT-WIGOSMD notes:

• @joergklausen will update parameter name in proposal (done), @JREyre will confirm
• @JREyre will open up new issues for the units table as needed
• @JREyre will remind Fiona to review

[JREyre]

@amilan17 @joergklausen
Here are the comments from ET-SSU via its Chair, Fiona Smith, [together with my comments]:

• Lidar return seems to be missing? I think this should be distinct from radar backscatter, though maybe it was consider that this covers all backscattered active sensing? If that is the case it should be made clear it covers both. [Reply: Yes, it is missing. Please could you supply a proposed name and definition for the appropriate variable(s).]
• Fluxes are missing, surely we want these listed here too? [Reply: Fluxes are already in WMDR. There are some problems of consistency and definitions which we are sorting out under a separate Issue 1-01-01 Align radiation variables requirements #389].
• However I did notice that 12258 "Refractivity" and 12261 "Delay Doppler map" have exactly the same definition "A 2D array of backscatter coefficients as a function of Doppler shift and signal delay, derived from GNSS reflectometry." Which seems a bit odd to me. [Reply: Thanks for spotting this - it’s an editorial problem – the definition for DDM has been incorrectly replicated under Refractivity, which be “The refractive index (of air) minus one. Particularly used for GNSS radio occultation observations.”]
• GRIB codes for synthetic satellite imagery potentially overlap with this (synthesised values from NWP vs actual observed values). Possibly relevant GRIB codes ... (from eccodes 4.2.3.1.table) [Reply: Thanks for alerting to these WMO GRIB codes. WMDR should certainly respond to these. TT-WIGOSMD will need to decide whether “cloud”, “clear-sky” and “scaled” variables need to be separate entries or whether the underlying variables of “radiance” and “brightness temperature” are sufficient for WMDR.]
  14 14 Cloudy brightness temperature (K)
  15 15 Clear-sky brightness temperature (K)
  16 16 Cloudy radiance (with respect to wave number) (W m-1 sr-1)
  17 17 Clear-sky radiance (with respect to wave number) (W m-1 sr-1)
  27 27 Bidirectional reflectance factor (numeric)
  28 28 Brightness temperature (K)
  29 29 Scaled radiance (numeric)

Some Notes from the WMO documents:

• Entry 27 (Bi-directional reflectance) "1. The ratio of the radiant flux reflected by a surface to that reflected into the same reflected-beam geometry and wavelength range by an ideal (lossless) and diffuse (Lambertian) standard surface, irradiated under the same conditions." Not sure what the relationship is between this and 12255 [Reply: Thanks. I will compare this definition with the current one and propose a resolution.]
• Entry 29 (Scaled radiance) "Top of atmosphere radiance observed by a sensor, multiplied by pi and divided by the in-band solar irradiance." [Reply: To be considered in the context of WMDR Issue 1-01-01 Align radiation variables requirements #389]

[JREyre]

@amilan17 @joergklausen
Some comments on "spectral directional reflectance" and "bidirectional reflectance".
I've reviewed the comments on this from ET-SSU. I don't think this proposed definition (from the GRIB documentation) nor our own currently proposed definition is satisfactory. A "bidirectional reflectance" is a particular realisation of a "bidirectional reflectance function" (BDRF). The BRDF is a ratio of reflected to incident radiation as a function of the incident angle (a vector) and the reflected angle (another vector). The bidirectional reflectance is a realisation of this function (a scalar value) for particularly values of both the incident and reflected angles. For a measurement by a satellite instrument, these angles are the zenith and azimuth angles of the sun and the zenith and azimuth angle of the viewing instrument at the feature (surface and/or atmospheric column) being observed.
So I suggest that the variable should be renamed "spectral bidirectional reflectance" and defined consistently with the above explanation. I suggest that the references to lossless and Lambertian surfaces are unnecessary (but not wrong), but I'd welcome advice on this.

[joergklausen]

As I understand @JREyre , the 12255 should be changed as follows:

notation	current name	new name	new description
12255	spectral directional reflectance	spectral bidirectional reflectance	Ratio of reflected to incident radiation as a function of incident angle, reflected angle and wavelength. For a satellite measurement, these angles are the zenith and azimuth angles of the sun and the zenith and azimuth angle of the viewing instrument at the feature (surface and/or atmospheric column) being observed.

[amilan17]

https://github.com/wmo-im/tt-wigosmd/wiki/2023.01.12-TT-WIGOSMD notes:

• John agrees with Joerg's update (see last comment)
• @joergklausen will update proposal and branch

[amilan17]

@joergklausen when branch is ready for my review, please open a PR to merge into the FT2023-1 branch.

[JREyre]

@amilan17 @joergklausen
Re bullet 4 above (GRIB codes): we agreed (provisionally) that we do not need to add the terms "cloudy", "clear-sky' and "scaled" to the vocabulary. So, I suggest we proceed with this Issue on this basis. However, I also agreed to refer this question back to ET-SSU (chair: Fiona Smith). If ET-SSU think that these terms should be included in the vocabulary, then I will raise this as a new Issue.

[joergklausen]

@amilan17 As to content, I believe, we are there and the proposal is complete, but, as discovered earlier today, there is a problem with the numbering (=notation) in the dedicated branch that you promised to look into. The existing code list in the Master branch ends at 12251, which looks okay. If you could update the branch, I can review and create a PR.

[amilan17]

@joergklausen branch is updated and PR is ready for your review