CSIS URI variables in praxis 2 [HC-LE]: Local Effects for Heatwaves #45

DenoBeno · 2019-09-16T14:19:56Z

I have implemented a "Local Effects or Heatwaves" resource following the instructions from the https://github.com/clarity-h2020/csis/wiki/Services-endpoints-(used-by-CSIS).

The result is https://csis.myclimateservice.eu/node/1272 and now I wonder if this is aright way to do it.

For a start, this is how URIs look: https://service.emikat.at/EmiKatTst/api/scenarios/${emikat_id}/feature/view.2974/table/data?rownum=1000&filter=PROJECT%3D${study_variant}&filter=PERIOD%3D${time_period}&filter=RCP%3D${emissions_scenario}&filter=FREQUENCE%3D${event_frequency}&filter=SZ_ID%3D$emikat_id

That is, this resource has following variables: ${emikat_id}, ${study_variant}, ${time_period}, ${emissions_scenario}, and ${event_frequency}

And this are the allowed values for the variables:

Study_variant can be either isn't known until the user decides on some adaptation options.

DenoBeno · 2019-09-16T14:30:21Z

Potential Issues:

Study_variant isn't known until the user decides on some adaptation options. Or we start with "baseline" and "adaptation-01" being the same?
From the wiki, I understand that we also need a "baseline" time period. OK, this can be easily added.

Wiki also claims the need for the baseline RPC, but this makes no sense to me. At baseline time period all RPCs must deliver the same result and there is AFAIK no scenario claiming that the future climate will remain the same. Do we need this?

Question: is this the way to go?

humerh · 2019-09-17T07:31:50Z

Please remark:
I have now adapted the name of the FILTER Column names:
STUDY_VARIANT
TIME_PERIOD
EMISSIONS_SCENARIO
EVENT_FREQUENCY

See also discussions in #42
and the definition in the wiki page: https://github.com/clarity-h2020/csis/wiki/Services-endpoints-(used-by-CSIS)

p-a-s-c-a-l · 2019-09-17T12:05:58Z

@humerh: I replaced the template URLs listed under Local Effects or Heatwaves with some variables and got the following errors:

p-a-s-c-a-l · 2019-09-18T12:38:54Z

Ignoring STUDY_VARIANT and EMISSION_SCENARIO (which is not needed for time period baselkine), I get an empty result for https://service.emikat.at/EmiKatTst/api/scenarios/2846/feature/view.2974/table/data?rownum=1000&filter=TIME_PERIOD%3D%27Baseline%27&filter=EVENT_FREQUENCY%3D%27rare%27&filter=SZ_ID%3D2846

humerh · 2019-09-18T14:10:50Z

You have to use the correct spelling:
https://service.emikat.at/EmiKatTst/api/scenarios/2846/feature/view.2974/table/data?rownum=1000&filter=TIME_PERIOD%3D%27Baseline%27&filter=EVENT_FREQUENCY%3D%27Rare%27&filter=SZ_ID%3D2846

But this was not only your error, I had some errors to correct. I hope, it is now ok!

humerh · 2019-09-18T14:33:05Z

The correct spelling is also neccessary for WMS call:

https://service.emikat.at/geoserver/clarity/wms?service=WMS&version=1.1.0&request=GetMap&layers=clarity%3Aview.2974&bbox=4672000.0%2C1979500.0%2C4687500.0%2C1988000.0&width=768&height=421&srs=EPSG%3A3035&format=image%2Fgif&CQL_FILTER=STUDY_VARIANT%3D%27BASELINE%27%20and%20TIME_PERIOD=%27Baseline%27%20AND%20EMISSIONS_SCENARIO=%27Baseline%27%20AND%20EVENT_FREQUENCY=%27Rare%27%20AND%20SZ_ID=2846&styles=T_A

p-a-s-c-a-l · 2019-09-18T15:32:21Z

@humerh OK, great. This works:

Could you please rename Ocassional to Occasional?

humerh · 2019-09-18T20:54:30Z

Could you please rename Ocassional to Occasional?
Done

DenoBeno · 2019-09-19T10:09:18Z

Hehe, I'm soooo dyslexic. This time I managed to put a correct value in taxonomy (hail spellchecker!) without even noticing the issue....

DenoBeno · 2019-09-19T10:45:05Z

You have to use the correct spelling:
https://service.emikat.at/EmiKatTst/api/scenarios/2846/feature/view.2974/table/data?rownum=1000&filter=TIME_PERIOD%3D%27Baseline%27&filter=EVENT_FREQUENCY%3D%27Rare%27&filter=SZ_ID%3D2846

But this was not only your error, I had some errors to correct. I hope, it is now ok!

This returns empty set atm :-(

By the way: according to this discussion thread, single quotes should not be used in URIs. However, when I try removing them, the call fails completely. replacing %3D with = is OK, replacing %27 with single quotes causes the call to fail completely. It would be good if we could dispose with %27, to make the URIs more readable => less probable to make an error.

DenoBeno · 2019-09-19T11:14:30Z

The correct spelling is also necessary for WMS call:

https://service.emikat.at/geoserver/clarity/wms?service=WMS&version=1.1.0&request=GetMap&layers=clarity%3Aview.2974&bbox=4672000.0%2C1979500.0%2C4687500.0%2C1988000.0&width=768&height=421&srs=EPSG%3A3035&format=image%2Fgif&CQL_FILTER=STUDY_VARIANT%3D%27BASELINE%27%20and%20TIME_PERIOD=%27Baseline%27%20AND%20EMISSIONS_SCENARIO=%27Baseline%27%20AND%20EVENT_FREQUENCY=%27Rare%27%20AND%20SZ_ID=2846&styles=T_A

Based on this, I have entered this link for the WMS in https://csis.myclimateservice.eu/node/1272

https://service.emikat.at/geoserver/clarity/wms?service=WMS&version=1.1.0&request=GetMap&layers=clarity%3Aview.2974&bbox=4672000.0%2C1979500.0%2C4687500.0%2C1988000.0&width=768&height=421&srs=EPSG%3A3035&format=image%2Fgif&CQL_FILTER=STUDY_VARIANT%3D%27${study_variant}%27%20and%20TIME_PERIOD=%27${time_period}%27%20AND%20EMISSIONS_SCENARIO=%27${emissions_scenario}%27%20AND%20EVENT_FREQUENCY=%27${event_frequency}%27%20AND%20SZ_ID=${emikat_id}&styles=T_A

The problem is that this isn't working now. I'm not going to update other links until I can test them. :-(

DenoBeno · 2019-09-19T11:23:04Z

Ignoring STUDY_VARIANT and EMISSION_SCENARIO (which is not needed for time period baselkine), I get an empty result for https://service.emikat.at/EmiKatTst/api/scenarios/2846/feature/view.2974/table/data?rownum=1000&filter=TIME_PERIOD%3D%27Baseline%27&filter=EVENT_FREQUENCY%3D%27rare%27&filter=SZ_ID%3D2846

One can indeed ignore some filters, but I would advise against it in production system.

For the time period "baseline", all emission scenarios should indeed return the same, so ignoring this one may be safe, as long as EMIKAT isn't producing any unnecessary data... Ideal solution would be if EMIKAT would ignore this filter for baseline timeframe and always return the same data but I suppose this is overkill...
For the time period baseline, the user could be interested to see what happens when adaptation options are applied and EMIKAT might be actually producing the data. Ignoring this parameter could therefore be dangerous.

It's IMO better to always explicitly set a filter and check if anything is returned than not to set it and risk getting double the amount of the data...

p-a-s-c-a-l · 2019-09-19T11:28:36Z

Now, the following URLs return an empty results: :-( Yesterday, they used to work.

@humerh Can you please fix this ASAP as I need a dependable API for client development. Thanks.

DenoBeno · 2019-09-19T13:07:00Z

This works again now. Thanks, Heinrich.

humerh · 2019-09-19T13:09:20Z

I fixed the problem.

DenoBeno · 2019-09-19T14:56:19Z

OK, adding entries to the resource:

this works:
https://service.emikat.at/EmiKatTst/api/scenarios/2846/feature/view.2974/table/data?rownum=1000&filter=STUDY_VARIANT%3D%27BASELINE%27&filter=TIME_PERIOD%3D%27Baseline%27&filter=EMISSIONS_SCENARIO%3D%27Baseline%27&filter=EVENT_FREQUENCY%3D%27Rare%27&filter=SZ_ID%3D2846
thus:
https://service.emikat.at/EmiKatTst/api/scenarios/${emikat_id}/feature/view.2974/table/data?rownum=1000&filter=STUDY_VARIANT%3D%27${study_variant}%27&filter=TIME_PERIOD%3D%27${time_period}%27&filter=EMISSIONS_SCENARIO%3D%27${time_period}%27&filter=EVENT_FREQUENCY%3D%27${event_frequency}%27&filter=SZ_ID%3D${emikat_id}

and same for geojson and csv should be OK. Problem: I can add the csv one to "download path" and I would like to add the others to "service paths (references)", but what should be the type?

For now, I have added all three of them as @resource, but this doesn't make sense. We need more types.

And https://service.emikat.at/EmiKatTst/api/scenarios/2846/feature/view.2974/table/geojson?rownum=1000&filter=STUDY_VARIANT%3D%27BASELINE%27&filter=TIME_PERIOD%3D%27Baseline%27&filter=EMISSIONS_SCENARIO%3D%27Baseline%27&filter=EVENT_FREQUENCY%3D%27Rare%27&filter=SZ_ID%3D2846

is taking forever, I thought that it's broken but it did work in the end.

DenoBeno · 2019-09-19T15:09:14Z

Status:

https://csis.myclimateservice.eu/node/1272 was updated with correct URIs.
all three service paths (references) for the csv/json/geojson have the same type "@resource". We need a way to differentiate here.
mapview and especially the geojson take too long to render.
with time_period='Baseline', one has to use EMISSIONS_SCENARIO='baseline'
replacing %3D with = in the URI works, but %27 can't be replaced by ´ (or by ` or by '). Ah wait: replacing it with ' does work, just can't be done in MS word.

So this works:
https://service.emikat.at/EmiKatTst/api/scenarios/2846/feature/view.2974/table/data?rownum=1000&filter=STUDY_VARIANT='BASELINE'&filter=TIME_PERIOD='20110101-20401231'&filter=EMISSIONS_SCENARIO='rcp26'&filter=EVENT_FREQUENCY='Rare'&filter=SZ_ID=2846

I'll change it all in the https://csis.myclimateservice.eu/node/1272 so that the URIS are easier to read.

DenoBeno · 2019-09-19T15:22:47Z

The "Baseline" emission scenario has been missing. I have added it now and gave it a good explanation, but this means that our software will need to know that baseline scenario can only be combined with baseline time. Hard-coded in the logic. :-(

https://csis.myclimateservice.eu/taxonomy/term/2256

I have also removed all %XX signs from the links used. E.g. the one used for the map now is:

https://service.emikat.at/geoserver/clarity/wms?

plus:

service=WMS&version=1.1.0&request=GetMap&layers=clarity:view.2974&bbox=4672000.0,1979500.0,4687500.0,1988000.0&width=768&height=421&srs=EPSG:3035&format=image/gif&CQL_FILTER=STUDY_VARIANT='${study_variant}' and TIME_PERIOD='${time_period}' AND EMISSIONS_SCENARIO='${emissions_scenario}' AND EVENT_FREQUENCY='${event_frequency}' AND SZ_ID=${emikat_id}&styles=T_A

(formatting breaks here, so I took it apart)
If this works - great, it's easier to read. It does work when I use such URIs in a browser, but browser automatically translates URIs as necessary...

p-a-s-c-a-l · 2019-09-20T15:02:52Z

There are even more pitfalls: ${emikat_id} in the URL is sometimes replaced to $%7Bemikat_id%7D(by the browser?).

DenoBeno · 2019-09-21T05:28:52Z

That looks really promising now. I looked at maximal damage and damage probability and it changes with period and scenario as one would expect. :-)

DenoBeno · 2019-09-21T07:35:52Z

Oh, what a happy day! :-( Is this affecting us when we use these links from JavaScript programs? And if yes: how to handle this?

…

On Fri, 20 Sep 2019, 17:02 Pascal Dihé, ***@***.***> wrote: There are even more pitfalls: $*{emikat_id}* in the URL is sometimes replaced to $%7Bemikat_id%7D(by the browser?). — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#45?email_source=notifications&email_token=AAWTC7U254O2VLAZPA5BYXDQKTQZ3A5CNFSM4IXC6P3KYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD7G7EAQ#issuecomment-533590530>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAWTC7WAIE6D66ME2YYY3O3QKTQZ3ANCNFSM4IXC6P3A> .

DenoBeno · 2019-09-21T13:24:23Z

I just tried the test table introduced by @p-a-s-c-a-l in #46 (comment) on local effects of heatwaves. It's very easy, just do this:

Log in on the EMIKAT on the browser.
follow the link https://csis.myclimateservice.eu/apps/simple-table-component/build/?emikat_id=2846
change the "view.2975" to "view.2974" in the uri at the top of the table and the result is:

DenoBeno · 2019-10-04T15:26:10Z

Heinrik has removed the "&filter=SZ_ID=${emikat_id}" part from the table URIs - I'll rewrite the resource now. (Someone already did this, except or the "download path")

Less good news is that we need to define a way to handle three different temperatures that are returned by this view: T_A, T_MRT, T_UTCI

styles=T_A for Apparent temperature
styles=T_TMRT for Mean Radiant Temperature
styles=T_UTCI for UTCI Temperature

In fact, this is also delivering the "discomfort level"...

For now, it is only possible to differentiate between these three cases (and discomfort?) in the maps, so I will add two more resources. But beware: the table data is exactly the same for al three at the moment, this needs fixing.

DenoBeno · 2019-10-04T16:12:31Z

OK, done. I have also removed the surplus hazards tags from all three, to avoid duplication in the data tab.

Beware: since the tables are the same for all three atm., I haven't duplicated the related yet. This needs to be done once these references can be split.

Oh jee. Map component is not showing this properly. Opening a new issue report: :-(

p-a-s-c-a-l · 2019-10-30T10:33:32Z

Implemented and working.

DenoBeno assigned humerh, patrickkaleta, therter and p-a-s-c-a-l Sep 16, 2019

DenoBeno added help wanted Extra attention is needed question Further information is requested specification Data Package Specification Issues labels Sep 16, 2019

This was referenced Sep 16, 2019

CSIS URI variables in praxis 4 [RA/IA]: Heat mortality risk/impact #46

Closed

Which EMIKAT-generated resources do we need to include in the screening Data Package(s)? #42

Closed

CSIS URI variables in praxis 1 [HC]: heat hazard indices #47

Closed

DenoBeno mentioned this issue Sep 17, 2019

How to choose which "variable combinations" to use in a study? clarity-h2020/csis#101

Closed

DenoBeno mentioned this issue Sep 19, 2019

CSIS URI variables in praxis 3 [EE]: Population Exposure #49

Closed

p-a-s-c-a-l changed the title ~~CSIS URI variables in praxis 1: Local Effects or Heatwaves~~ CSIS URI variables in praxis 1: Local Effects for Heatwaves Sep 26, 2019

p-a-s-c-a-l changed the title ~~CSIS URI variables in praxis 1: Local Effects for Heatwaves~~ CSIS URI variables in praxis 2 [HC-LE]: Local Effects for Heatwaves Sep 26, 2019

p-a-s-c-a-l mentioned this issue Sep 26, 2019

1st Data Package: 02 Hazard Local Effects Layers for Heat Waves #11

Closed

6 tasks

DenoBeno removed help wanted Extra attention is needed question Further information is requested labels Oct 4, 2019

DenoBeno mentioned this issue Oct 4, 2019

Map component doesn't set the temperature type in EMIKAT local temperature layers. clarity-h2020/map-component#62

Closed

p-a-s-c-a-l added the BB: Catalogue of ER & AO Catalogue of Elements at Risk and Adaptation Options Building Block label Oct 30, 2019

p-a-s-c-a-l added this to the 1st Data Package milestone Oct 30, 2019

p-a-s-c-a-l closed this as completed Oct 30, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

CSIS URI variables in praxis 2 [HC-LE]: Local Effects for Heatwaves #45

CSIS URI variables in praxis 2 [HC-LE]: Local Effects for Heatwaves #45

DenoBeno commented Sep 16, 2019 •

edited

Loading

DenoBeno commented Sep 16, 2019 •

edited

Loading

humerh commented Sep 17, 2019

p-a-s-c-a-l commented Sep 17, 2019 •

edited

Loading

p-a-s-c-a-l commented Sep 18, 2019

humerh commented Sep 18, 2019

humerh commented Sep 18, 2019

p-a-s-c-a-l commented Sep 18, 2019

humerh commented Sep 18, 2019

DenoBeno commented Sep 19, 2019

DenoBeno commented Sep 19, 2019 •

edited

Loading

DenoBeno commented Sep 19, 2019

DenoBeno commented Sep 19, 2019

p-a-s-c-a-l commented Sep 19, 2019 •

edited

Loading

DenoBeno commented Sep 19, 2019

humerh commented Sep 19, 2019

DenoBeno commented Sep 19, 2019 •

edited

Loading

DenoBeno commented Sep 19, 2019 •

edited

Loading

DenoBeno commented Sep 19, 2019 •

edited

Loading

p-a-s-c-a-l commented Sep 20, 2019

DenoBeno commented Sep 21, 2019

DenoBeno commented Sep 21, 2019 via email

DenoBeno commented Sep 21, 2019

DenoBeno commented Oct 4, 2019 •

edited

Loading

DenoBeno commented Oct 4, 2019

p-a-s-c-a-l commented Oct 30, 2019

CSIS URI variables in praxis 2 [HC-LE]: Local Effects for Heatwaves #45

CSIS URI variables in praxis 2 [HC-LE]: Local Effects for Heatwaves #45

Comments

DenoBeno commented Sep 16, 2019 • edited Loading

DenoBeno commented Sep 16, 2019 • edited Loading

humerh commented Sep 17, 2019

p-a-s-c-a-l commented Sep 17, 2019 • edited Loading

p-a-s-c-a-l commented Sep 18, 2019

humerh commented Sep 18, 2019

humerh commented Sep 18, 2019

p-a-s-c-a-l commented Sep 18, 2019

Could you please rename Ocassional to Occasional?

humerh commented Sep 18, 2019

DenoBeno commented Sep 19, 2019

DenoBeno commented Sep 19, 2019 • edited Loading

DenoBeno commented Sep 19, 2019

DenoBeno commented Sep 19, 2019

p-a-s-c-a-l commented Sep 19, 2019 • edited Loading

DenoBeno commented Sep 19, 2019

humerh commented Sep 19, 2019

DenoBeno commented Sep 19, 2019 • edited Loading

DenoBeno commented Sep 19, 2019 • edited Loading

DenoBeno commented Sep 19, 2019 • edited Loading

p-a-s-c-a-l commented Sep 20, 2019

DenoBeno commented Sep 21, 2019

DenoBeno commented Sep 21, 2019 via email

DenoBeno commented Sep 21, 2019

DenoBeno commented Oct 4, 2019 • edited Loading

DenoBeno commented Oct 4, 2019

p-a-s-c-a-l commented Oct 30, 2019

DenoBeno commented Sep 16, 2019 •

edited

Loading

DenoBeno commented Sep 16, 2019 •

edited

Loading

p-a-s-c-a-l commented Sep 17, 2019 •

edited

Loading

DenoBeno commented Sep 19, 2019 •

edited

Loading

p-a-s-c-a-l commented Sep 19, 2019 •

edited

Loading

DenoBeno commented Sep 19, 2019 •

edited

Loading

DenoBeno commented Sep 19, 2019 •

edited

Loading

DenoBeno commented Sep 19, 2019 •

edited

Loading

DenoBeno commented Oct 4, 2019 •

edited

Loading