Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

EDR CoverageJSON conformance fixes: time axis and demo data #1814

Open
wants to merge 29 commits into
base: master
Choose a base branch
from
Open

EDR CoverageJSON conformance fixes: time axis and demo data #1814

Show file tree
Hide file tree
Changes from 14 commits
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
5cf2ee3
clean axis y order
pzaborowski Aug 12, 2024
1b5432c
remove comments
pzaborowski Aug 12, 2024
3fc3b5f
fixed axis from data/bbox
pzaborowski Aug 12, 2024
2feee54
bbox read from already calculated metadata
pzaborowski Sep 18, 2024
d583d32
Merge branch 'upsorg' into edr_fixed_axis
pzaborowski Sep 19, 2024
f0c2dd0
fix
pzaborowski Sep 19, 2024
d3c6726
whitespaces
pzaborowski Sep 19, 2024
dfce634
Merge pull request #11 from pzaborowski/edr_fixed_axis
pzaborowski Sep 19, 2024
00ea4c4
EDR covjson time axis compliance with stadnard
pzaborowski Sep 19, 2024
bc7623e
coveragejson compliance
pzaborowski Sep 20, 2024
8531e40
Update coads_sst.nc
pzaborowski Sep 20, 2024
baf9c7c
Merge branch 'EDR-time-axis-values' into edr_fixed_axis
pzaborowski Sep 20, 2024
9c0be8b
formatting
pzaborowski Sep 20, 2024
57fb90f
Merge pull request #12 from pzaborowski/edr_fixed_axis
pzaborowski Sep 20, 2024
76fd130
Update Ubuntu Jammy docker base image to 20240911.1 (#1815)
kalxas Sep 21, 2024
52bec0f
docs: update compliance for OGC API - Processes (#1817)
tomkralidis Sep 21, 2024
6b91024
Zoom to first layer on EDR (#1819)
webb-ben Sep 26, 2024
83ef1ac
update release version
kalxas Sep 27, 2024
b3a7071
back to dev
kalxas Sep 27, 2024
474cb60
fix item queryables provider handling (#1820)
tomkralidis Sep 29, 2024
d240a82
Improvements for xarray provider (#1800)
barbuz Sep 30, 2024
e736fa3
Custom esri token service (#1813)
colin-geo Oct 1, 2024
179c90f
Fixes a memoryfile issue with rasterio. When using f=json, it doesn't…
Alex-NRCan Oct 3, 2024
3bdeefe
Properly support int variables of any width (#1829)
barbuz Oct 15, 2024
c38ad33
tests aligned with data, unneccesary time properties removed
pzaborowski Oct 23, 2024
368fc34
Merge branch 'upsorg'
pzaborowski Oct 23, 2024
1431963
Merge branch 'edr_fixed_axis'
pzaborowski Oct 23, 2024
e6436eb
Merge branch 'EDR-time-axis-values'
pzaborowski Oct 23, 2024
12f9c5d
time dimension as first and with label from configuration
pzaborowski Nov 1, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
49 changes: 20 additions & 29 deletions pygeoapi/provider/xarray_.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -92,6 +92,8 @@ def __init__(self, provider_def):
self.axes = [self._coverage_properties['x_axis_label'],
self._coverage_properties['y_axis_label'],
self._coverage_properties['time_axis_label']]
self.time_axis_covjson = provider_def.get('time_axis_covjson') \
or self.time_field
Comment on lines +103 to +104
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is the point of adding this? Can we add a test case that uses this field. Suggest to potentially rename in pygeoapi, but will defer to @tomkralidis. This addition should also be noted in the docs (OACoverages and OAEDR

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

used it to configure the time axis label (see #1833, coverage schema expect 't' as the time axis) and keep an option for backward compatibility in case the implemented client needs the other one.


self.get_fields()
except Exception as err:
Expand All @@ -105,7 +107,7 @@ def get_fields(self):
LOGGER.debug('Adding variable')
dtype = value.dtype
if dtype.name.startswith('float'):
dtype = 'number'
dtype = 'float'

self._fields[key] = {
'type': dtype,
Expand Down Expand Up @@ -248,48 +250,36 @@ def gen_covjson(self, metadata, data, fields):
"""

LOGGER.debug('Creating CoverageJSON domain')
minx, miny, maxx, maxy = metadata['bbox']
startx, starty, stopx, stopy = metadata['bbox']
mint, maxt = metadata['time']

selected_fields = {
key: value for key, value in self.fields.items()
if key in fields
}

try:
tmp_min = data.coords[self.y_field].values[0]
except IndexError:
tmp_min = data.coords[self.y_field].values
try:
tmp_max = data.coords[self.y_field].values[-1]
except IndexError:
tmp_max = data.coords[self.y_field].values

if tmp_min > tmp_max:
LOGGER.debug(f'Reversing direction of {self.y_field}')
miny = tmp_max
maxy = tmp_min

cj = {
'type': 'Coverage',
'domain': {
'type': 'Domain',
'domainType': 'Grid',
'axes': {
'x': {
'start': minx,
'stop': maxx,
'start': startx,
'stop': stopx,
'num': metadata['width']
},
'y': {
'start': maxy,
'stop': miny,
'start': starty,
'stop': stopy,
'num': metadata['height']
},
self.time_field: {
'start': mint,
'stop': maxt,
'num': metadata['time_steps']
self.time_axis_covjson: {
'values': [str(i) for i in
data.coords[self.time_field].values],
#'start': mint,
#'stop': maxt,
#'num': metadata['time_steps']
}
},
'referencing': [{
Expand All @@ -307,7 +297,7 @@ def gen_covjson(self, metadata, data, fields):
for key, value in selected_fields.items():
parameter = {
'type': 'Parameter',
'description': value['title'],
'description': {'en': value['title']},
'unit': {
'symbol': value['x-ogc-unit']
},
Expand All @@ -330,11 +320,12 @@ def gen_covjson(self, metadata, data, fields):
'type': 'NdArray',
'dataType': value['type'],
'axisNames': [
'y', 'x', self._coverage_properties['time_axis_label']
self.time_axis_covjson, 'y', 'x'
],
'shape': [metadata['height'],
metadata['width'],
metadata['time_steps']]
'shape': [metadata['time_steps'],
metadata['height'],
metadata['width']
]
}
cj['ranges'][key]['values'] = data[key].values.flatten().tolist() # noqa
except IndexError as err:
Expand Down
2 changes: 1 addition & 1 deletion pygeoapi/templates/collections/edr/query.html
View file
Open in desktop
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be already addressed in upstream pygeoapi

Original file line number Diff line number Diff line change
Expand Up @@ -139,7 +139,7 @@
if (!firstLayer) {
firstLayer = layer;
layer.on('afterAdd', () => {
zoomToLayers([layers])
zoomToLayers([layer])
if (!cov.coverages) {
if (isVerticalProfile(cov) || isTimeSeries(cov)) {
layer.openPopup();
Expand Down
38 changes: 24 additions & 14 deletions tests/api/test_environmental_data_retrieval.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -123,7 +123,7 @@ def test_get_collection_edr_query(config, api_):
# bounded date range
req = mock_api_request({
'coords': 'POINT(11 11)',
'datetime': '2000-01-17/2000-06-16'
'datetime': '2000-01-17/2000-08-16'
})
rsp_headers, code, response = get_collection_edr_query(
api_, req, 'icoads-sst', None, 'position')
Expand All @@ -132,14 +132,17 @@ def test_get_collection_edr_query(config, api_):
data = json.loads(response)
time_dict = data['domain']['axes']['TIME']

assert time_dict['start'] == '2000-02-15T16:29:05.999999999'
assert time_dict['stop'] == '2000-06-16T10:25:30.000000000'
assert time_dict['num'] == 5
assert time_dict['values'] == ['2000-06-16T10:25:30.000000000',
'2000-07-16T20:54:36.000000000',
'2000-08-16T07:23:42.000000000']
# assert time_dict['start'] == '2000-06-16T10:25:30.000000000'
# assert time_dict['stop'] == '2000-08-16T07:23:42.000000000'
# assert time_dict['num'] == 3
Comment on lines +138 to +140
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
# assert time_dict['start'] == '2000-06-16T10:25:30.000000000'
# assert time_dict['stop'] == '2000-08-16T07:23:42.000000000'
# assert time_dict['num'] == 3


# unbounded date range - start
req = mock_api_request({
'coords': 'POINT(11 11)',
'datetime': '../2000-06-16'
'datetime': '../2000-08-16'
})
rsp_headers, code, response = get_collection_edr_query(
api_, req, 'icoads-sst', None, 'position')
Expand All @@ -148,29 +151,36 @@ def test_get_collection_edr_query(config, api_):
data = json.loads(response)
time_dict = data['domain']['axes']['TIME']

assert time_dict['start'] == '2000-01-16T06:00:00.000000000'
assert time_dict['stop'] == '2000-06-16T10:25:30.000000000'
assert time_dict['num'] == 6
assert time_dict['values'] == ['2000-06-16T10:25:30.000000000',
'2000-07-16T20:54:36.000000000',
'2000-08-16T07:23:42.000000000']
# assert time_dict['start'] == '2000-06-16T10:25:30.000000000'
# assert time_dict['stop'] == '2000-08-16T07:23:42.000000000'
# assert time_dict['num'] == 3
Comment on lines +157 to +159
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
# assert time_dict['start'] == '2000-06-16T10:25:30.000000000'
# assert time_dict['stop'] == '2000-08-16T07:23:42.000000000'
# assert time_dict['num'] == 3


# unbounded date range - end
req = mock_api_request({
'coords': 'POINT(11 11)',
'datetime': '2000-06-16/..'
'datetime': '2000-08-16/..'
})
rsp_headers, code, response = get_collection_edr_query(
api_, req, 'icoads-sst', None, 'position')
assert code == HTTPStatus.OK

data = json.loads(response)
time_dict = data['domain']['axes']['TIME']

assert time_dict['start'] == '2000-06-16T10:25:30.000000000'
assert time_dict['stop'] == '2000-12-16T01:20:05.999999996'
assert time_dict['num'] == 7
assert time_dict['values'] == ['2000-08-16T07:23:42.000000000',
'2000-09-15T17:52:48.000000000',
'2000-10-16T04:21:54.000000000',
'2000-11-15T14:51:00.000000000',
'2000-12-16T01:20:05.999999996']
# assert time_dict['start'] == '2000-08-16T07:23:42.000000000'
# assert time_dict['stop'] == '2000-12-16T01:20:05.999999996'
# assert time_dict['num'] == 7
Comment on lines +177 to +179
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
# assert time_dict['start'] == '2000-08-16T07:23:42.000000000'
# assert time_dict['stop'] == '2000-12-16T01:20:05.999999996'
# assert time_dict['num'] == 7


# some data
req = mock_api_request({
'coords': 'POINT(11 11)', 'datetime': '2000-01-16'
'coords': 'POINT(11 11)', 'datetime': '2000-06-16'
})
rsp_headers, code, response = get_collection_edr_query(
api_, req, 'icoads-sst', None, 'position')
Expand Down
Binary file modified tests/data/coads_sst.nc
View file
Open in desktop
Binary file not shown.