inconsistency between requested/returned time extent in case of summertime

[veenstrajelmer]

When requesting data from the ddl, we always get timestamps/extent in UTC+1 back (MET). This makes sense since the data in DONAR is also in MET. However, something perculiar happens when requesting data for summertime dates. The requested timestamp is shifted with one hour. In the example below I am retrieving data from 01:00-04:00 hours (MET), but I get one additional hour of data (01:00-05:00, MET). The summertime starts at 03:00 this night.

Reproducible example:

import requests
import pandas as pd

# general
start_date_str = '2022-03-27T01:00:00.000+01:00'
end_date_str = '2022-03-27T04:00:00.000+01:00'

aquometadata_dict = {'Eenheid': {'Code': 'cm'},
                      'Grootheid': {'Code': 'WATHTE'},
                      'Hoedanigheid': {'Code': 'NAP'},
                      'Groepering': {'Code': 'NVT'}}
locatie_dict = {'X': 576917.669784491, 
                'Y': 5759136.15818497,
                'Code': 'HOEKVHLD'}

# meas slice
request = {
    "AquoPlusWaarnemingMetadata": {
        "AquoMetadata": aquometadata_dict
        },
    "Locatie": locatie_dict,
    "Periode": {"Begindatumtijd": start_date_str, 
                "Einddatumtijd": end_date_str},
}

url_ophalen = "https://waterwebservices.rijkswaterstaat.nl/ONLINEWAARNEMINGENSERVICES_DBO/OphalenWaarnemingen"
resp = requests.post(url_ophalen, json=request)
result = resp.json()
if not result["Succesvol"]:
    raise Exception(result.get("Foutmelding", "No error returned"))

rows = []
for waarneming in result["WaarnemingenLijst"]:
    for row in waarneming["MetingenLijst"]:
        rows.append(row)
df = pd.json_normalize(rows)

print(df["Tijdstip"])

This prints:

0     2022-03-27T01:00:00.000+01:00
1     2022-03-27T01:10:00.000+01:00
2     2022-03-27T01:20:00.000+01:00
3     2022-03-27T01:30:00.000+01:00
4     2022-03-27T01:40:00.000+01:00
5     2022-03-27T01:50:00.000+01:00
6     2022-03-27T02:00:00.000+01:00
7     2022-03-27T02:10:00.000+01:00
8     2022-03-27T02:20:00.000+01:00
9     2022-03-27T02:30:00.000+01:00
10    2022-03-27T02:40:00.000+01:00
11    2022-03-27T02:50:00.000+01:00
12    2022-03-27T03:00:00.000+01:00
13    2022-03-27T03:10:00.000+01:00
14    2022-03-27T03:20:00.000+01:00
15    2022-03-27T03:30:00.000+01:00
16    2022-03-27T03:40:00.000+01:00
17    2022-03-27T03:50:00.000+01:00
18    2022-03-27T04:00:00.000+01:00
19    2022-03-27T04:10:00.000+01:00
20    2022-03-27T04:20:00.000+01:00
21    2022-03-27T04:30:00.000+01:00
22    2022-03-27T04:40:00.000+01:00
23    2022-03-27T04:50:00.000+01:00
24    2022-03-27T05:00:00.000+01:00
Name: Tijdstip, dtype: object

When requesting '2022-03-27T04:00:00.000+01:00' to '2022-03-27T08:00:00.000+01:00', I get 05:00-09:00 (MET)

[TvLoon-RWS]

The following body gives 4 hour of data, where 3 hours is expected;

{
    "Locatie": {
        "X": "576917.669784491",
        "Y": "5759136.15818497",
        "Code": "HOEKVHLD"},
    "AquoPlusWaarnemingMetadata": {
        "AquoMetadata": {
            "Grootheid":{
                "Code": "WATHTE" }, 
            "Parameter":{
                "Code": "NVT"
            }
        }
    },
    "Periode": {
        "Begindatumtijd": "2022-03-27T01:00:00.000+01:00",
        "Einddatumtijd": "2022-03-27T04:00:00.000+01:00"
    }
    ```

[TvLoon-RWS]

Check if the same issue arises at Wadar WaterWebservices

[veenstrajelmer]

With the new Wadar WaterWebservices the problem does not occur:

import requests
import pandas as pd

# general
start_date_str = '2022-03-27T01:00:00.000+01:00'
end_date_str = '2022-03-27T04:00:00.000+01:00'

aquometadata_dict = {'Eenheid': {'Code': 'cm'},
                      'Grootheid': {'Code': 'WATHTE'},
                      'Hoedanigheid': {'Code': 'NAP'},
                      'Groepering': {'Code': ''}}
locatie_dict = {'Code': 'ameland.nes'}

# meas slice
request = {
    "AquoPlusWaarnemingMetadata": {
        "AquoMetadata": aquometadata_dict
        },
    "Locatie": locatie_dict,
    "Periode": {"Begindatumtijd": start_date_str, 
                "Einddatumtijd": end_date_str},
}

url_ophalen = "https://waterwebservices.beta.rijkswaterstaat.nl/test/ONLINEWAARNEMINGENSERVICES/OphalenWaarnemingen"
resp = requests.post(url_ophalen, json=request)

if resp.status_code==204:
    # this error is raised here, but catched in ddlpy.ddlpy.measurements() so the process can continue.
    raise Exception(resp.reason)

result = resp.json()
if not result["Succesvol"]:
    raise Exception(result.get("Foutmelding", "No error returned"))

rows = []
for waarneming in result["WaarnemingenLijst"]:
    for row in waarneming["MetingenLijst"]:
        rows.append(row)
df = pd.json_normalize(rows)

print(df["Tijdstip"])

Results in:

0     2022-03-27T01:00:00.000+01:00
1     2022-03-27T01:10:00.000+01:00
2     2022-03-27T01:20:00.000+01:00
3     2022-03-27T01:30:00.000+01:00
4     2022-03-27T01:40:00.000+01:00
5     2022-03-27T01:50:00.000+01:00
6     2022-03-27T02:00:00.000+01:00
7     2022-03-27T02:10:00.000+01:00
8     2022-03-27T02:20:00.000+01:00
9     2022-03-27T02:30:00.000+01:00
10    2022-03-27T02:40:00.000+01:00
11    2022-03-27T02:50:00.000+01:00
12    2022-03-27T03:00:00.000+01:00
13    2022-03-27T03:10:00.000+01:00
14    2022-03-27T03:20:00.000+01:00
15    2022-03-27T03:30:00.000+01:00
16    2022-03-27T03:40:00.000+01:00
17    2022-03-27T03:50:00.000+01:00
18    2022-03-27T04:00:00.000+01:00
Name: Tijdstip, dtype: object

The start/stoptime are now the same as the requested start/stoptime