Weather binding

[lolodomo]

We don't have an equivalent of the OH 1.x weather binding in Eclipse smarthome or OH2. We only have the Yahoo weather binding but it provides apparently less data than the 1.x binding.
Any reason for that ?
Any work in progress ?
Any problem with the proposal of a general weather binding in Eclipse smarthome with several possible providers that will be a migration of the 1.x binding ?
I am a candidate to work on this new binding.

[lolodomo]

Here is my proposal:

• Bridges:
  • weather:geoLocation:locationId with config parameters: latitude, longitude
  • weather:yahooLocation:locationId with config parameter: woeid
• Things:
  • weather:yahooData:locationId:id with config parameter: updateInterval
  • weather:hamData:locationId:id with config parameters: clientId, clientSecret, updateInterval
  • weather:providerData:locationId:id with config parameters: provider, apikey, updateInterval, language
• Binding configuration parameters: speedUnit, temperatureUnit, rainUnit, snowUnit, pressureUnit, lenghtUnit
• Channels:
  • channel groups atmosphere_today, atmosphere_forecast1, atmosphere_forecast2, ... with channels humidity, visibility, pressure, pressureTrend, ozone, uvIndex
  • channel groups clouds_today, clouds_forecast1, clouds_forecast2, ... with channel percent
  • channel groups condition_today, condition_forecast1, condition_forecast2, ... with channels text, observationTime, id, lastUpdate, commonId
  • channel groups precipitation_today, precipitation_forecast1, precipitation_forecast2, ... with channels rain, snow, probability, total
  • channel groups temperature_today, temperature_forecast1, temperature_forecast2, ... with channels current, min, max, feel, dewpoint, minMax
  • channel groups wind_today, wind_forecast1, wind_forecast2, ... with channels speed, direction, degree, gust, chill
• General idea is to re-use what can be re-used from the 1.x binding, that is mainly all provider data retrieval and internal data storing (most of the binding code in fact)
• what would be lost (at least in a first time)
  • round/scale for numeric values
  • WEB view
• this binding will entirely cover the Yahoo Weather binding

Any comment ?

[watou]

I think each weather provider should have its own Bridge Thing handler implementation, hiding all provider specific details there, but feed data to a common set of Things like Conditions, Forecast, etc. The binding itself should have no configuration. The YahooWeather binding was meant mainly as a code example, iirc.

[kaikreuzer]

I think each weather provider should have its own Bridge Thing handler implementation, hiding all provider specific details there

👍 . This would imho be the correct approach, following the ESH design idea.

but feed data to a common set of Things like Conditions, Forecast, etc.

Not even that - only the channels should have the same tagging - imho the clean way is to define a weather-related tag library in the context of #1093.

[lolodomo]

Ok to have a bridge thing for each provider. The bridge thing will be in fact a mix of provider and location. We will provide no channels on the bridge things but the bridge thing will retrieve data from provider and dispatch the result to the things.

Then I think it is a good idea to have one weather thing for the current weather data, another one (same think type) for the 1 day forecast, another one for the 2 days forecast,, ... With this structure, it will be possible to have an automatic discovery of these things.

@watou: I am not sure to be ok with your idea to dispatch all the weather data on different thing types like one for wind, another one for precipitation, another one for temperatures, ... This will produce a lot of things. We can have only one thing type for weather data and we can group the channels in channel groups like one group for temperatures, one group for precipitation, one group for wind, .... Isn't it a better idea ?

@kaikreuzer : I have to read about your tag concept. I just hope I have not (again) to wait for a new concept implementation to make progress on this binding.

[lolodomo]

Regarding the tagging, that is not clear at all for me what is requested.
Do you want that I define new channel categories ? Define system channels rather than simple channels for all weather individual data ?
I understand more or less that the concept is to have some standard channels but it makes sense mainly for enum channels. All weather data are mainly numeric values except the weather condition and the wind direction.

[lolodomo]

Is Eclipse SmartHome the right place for this binding or should I consider an openHAB 2 binding ?
Can I take source files from OH 1.x binding and just rename the package/includes and the licence header to be compliant with Eclipse SmartHome ?

[kaikreuzer]

I do not know the details of the OH1 weather binding, but as discussed above, I doubt that it at all fits into the architecture (if my understanding is right that it internally uses yet another abstraction to be able to plug in different providers). There should be dedicated bindings for the different providers instead (and if there is code in common, we can discuss the creation of a shared bundle that all are using).

[maggu2810]

I agree that it would be "better" to create different bindings for different weather data providers and if there is code to share, we can create a bundle that contains all that shared code.

[lolodomo]

It will be a pity to not re-use the good work done with the 1.x weather binding.
There is not really an abstraction but the data parser for each provider is storing the retrieved data in a Java structure common to all providers. That is a good idea and looks like a good implementation. And thiese parsers are working well I suppose.

If we create a binding for each provider, we will have a lot of duplicated code because each binding will in fact do the same thing, that is start a scheduled job that will request data from the provider, then parse the data and finally update the channels of the binding. Each one will have to implement its unit converting for example. With several bindings, it will be more difficult to maintain something consistent betwwen the bindings.

That being said, we could try to create a shared bundle that will deal with provder request and result parsing. But then the weather bindings will be more or less duplicates of each other.

[maggu2810]

@lolodomo WRT commercial use of ESH bundles: If you use one binding for all weather providers and a customer owns only the agreement of one provider (e.g. he pays for) to use that weather data, how could the other ones removed from its product?
Perhaps the data parser itself could be implemented using OSGi services, so one bundle contains the generic implementation and the other bundles contains one service per provider.

[lolodomo]

We can also have very similar bindings, each one with its own code.
In this case, I will first implement the Wunderground Weather binding.

[lolodomo]

If we have one binding per provider, I propose to put the API key as a binding parameter. The binding will handle only one thing type: a weather location.

[lolodomo]

And we can copy a large part of the YahooWeather binding. What needs to be specific to each provider is the method updateWeatherData and of course all the getXXX methods to get individual information from the data returned by the provider.

[watou]

I propose to put the API key as a binding parameter.

Why not a property of the Thing? It makes more sense to me to address each weather providing API on its own terms, and supply as many channels as available from that provider, instead of abstracting above them all, and then making exceptions for each. Over time I've seen many problem reports with the OH1 weather binding where identifying the root cause was more difficult due to the abstractions internal to the binding.

[lolodomo]

We could put the API key on the thing but it is something that is attached to the provider (and so the binding if we decide to have one binding per provider). Putting it on the thing means that the user will have to enter the API key for each thing (location) he wants to define, with no real reason.

The thing type I imagine could even be shared by all weather bindings but I don't think the Eclipse SmartHome architecture allows this kind of things ? This thing type would have as parameters the location identifier and as channels all weather information that a weather provider can provide.

[lolodomo]

In fact, the ideal would be an abstract weather binding lol

[lolodomo]

What is the recommended JSON parser and mapper to Java object for Eclipse SmartHome and openHAB 2 ? Is it Google Gson ?

[watou]

Is it Google Gson ?

Yes.

[lolodomo]

Another reason to not re-use 1.x code ;)
So finally I will implement a new specific binding for Weather Underground that will look like the Yahoo Weather binding. This new binding could then be a good start point for any other weather bindings that requires JSON parsing.

[lolodomo]

The advantage of a specific binding for each provider is that we can provide specific provider data.
I will try to provide a first version of the Weather Underground binding before the end of the year (I have now already something very basic running).

[lolodomo]

Weather Underground provides icon sets for current weather conditions: https://www.wunderground.com/weather/api/d/docs?d=resources/icon-sets&MR=1
I am already afraid by your possible answer to the next question lol but what would be the correct way to provide this icon in Eclipse SmartHome ?
Shall I provide a channel with the URL ?
Shall I download the different icon sets and include them in Eclipse SmartHome ? I am not clear how to build and provide such a new incon set in Eclipse SmartHome.

[lolodomo]

In case I should consider iconset, should I simply add WU icon set in org.eclipse.smarthome.ui.iconset.classic as wu1-<state>.png for first WU icon set, wu2-<state>.png for second WU icon set, ... ? This would require that I convert original GIF images into PNG (and/org SVG images.

[hakan42]

There is a nice project over at https://github.com/ghys/org.openhab.ui.iconset.climacons that provides just icons. I was thinking of using that project as a template to create one of my own that provides the netatmo icons.

Perhaps something like that could be a basis for a seperate "binding" that provides just the icons, as I am not sure what the license situation with the WU icons are and whether they would be compatible with eclipse/smarthome ...

[lolodomo]

Do you know if such icon set bundle after deployment (put in addons directory or included in the distro) would be directly available in all UI ? I don't know if UIs consider only the "classic" icon set or all icon set bundles ? @kaikreuzer : you certainly have the answer ?
By the way, I am not sure at all that we have the right to download and provide these WU icon sets in any product ?
But I could imagine a new icon set provider that will provide the icon URL from WU, meaning we will not copy/provide the icons directly in our distro ?

[lolodomo]

No need to proxy, the icon set provider just needs to open an URL so a new icon set provider could provide the WU URL. The only problem would be the WU icon format (GIF) that is not supported by Eclipse SmartHome ! Would it be easy to add GIF format in Eclipse SmartHome ?

For Netatmo, I could imagine a similar icon set provider. (Note that a mapping could be required).

But still the same question: will such additional icon set provider be directly considered by all Eclipse SmartHome UIs ? How are defined the icon sets to be used ?

[hakan42]

I would expect the icons to be available to any and all consumers (bindings) in the same installation.

And about the gif format, the climacons project has a script inside it that converts some images automatically to svgs. A similiar build step could convert the WU icons to SVG or JPG, and maybe even provide both?

[lolodomo]

Maybe the mapping should be done in the binding itself rather than the icon set provider ?
I could even imagine to use the clamicons provider in the Weather Underground binding through an appropriate mapping.

[cdjackson]

Just for completeness, there's another nice weather icons here.

[lolodomo]

@cdjackson : interesting, there is even a proposed mapping for few weather providers including Weather Underground. But that is not clear for me how we could integrate that in Eclipse SmartHome.

[cdjackson]

Agreed - these are font icons - so they are used in CSS classes and not served up like images - many modern UIs are using them now as they are typically more theme-able… I just thought I’d throw the link into the mix ;)

[mhilbush]

Weather Underground binding should be ready in the following days

@lolodomo I'd be interested in testing this when it's ready -- either in the distro, or as a jar I can drop into addons.

[lolodomo]

@mhilbush : I will push my work in my Git fork and then submit a PR. I don't know if the PR will be merged or not but at least my binding will then be available to anybody. I will also put the jar file somewhere for downloading and communicate the link.

[lolodomo]

My current work is available here: https://github.com/lolodomo/smarthome/tree/wu/extensions/binding/org.eclipse.smarthome.binding.weatherunderground
I will probably push a request today.

For the about.html, where can I get the reference file ? I copied the file from the YahooWeather binding, updated the date and of course the part relative to third-party services. Is it the right way to provide this file ?

For the WU service, I played with "autoip" as location but it seems to be not reliable at all because I got data from a location at several hundreds kilometers from my location ! I have not documented this feature.

Then I tried to play with "country/city" as location and I discovered that for certain cities it does not provide weather data but just a list of available weather stations. That is the case for example for "France/Paris" or "Italia/Roma". In this case, the thing is OFFILENE. I will update the code to provide more information in the log about the available stations.

Using "latitude,longitude" as location, I discovered that I got weather data from a station that is several kilometers from my location, while the WU map is showing several stations just around me that are Netatmo personal stations. Does it mean that by default, WU is relying only on a few official stations (in particular airport stations) ?

[watou]

For Weather Underground, I use the OH1 HTTP binding and an XSLT transformation against a URL like this:

http://api.wunderground.com/api/b1bxxxx4981578f/conditions/q/pws:XCLOXXXI4.xml

So your binding ought to allow a thing to specify a pws parameter so the user can choose his or her preferred weather station.

[lolodomo]

Yes, you can enter a PWS too. Just enter "pws:XCLOXXXI4" in the location setting. Take a look at my README.md file.

[lolodomo]

Interesting, I started again (using lat,lon) and this time I got another station.
For the same lat/lon, WU does not always provide the data for the same station ?

[hakan42]

Nice, I logged in to wunderground to obtain an API key and search for a provider for my location, and it promptly offers me my own netatmo station 😄

I guess I have to search a bit more to find an "official" station

[lolodomo]

@hakan42 : I don't know if there is a way to identify and request professional stations ?

[hakan42]

I am looking over the web interface but did not find anything yet. The netatmo devices seem to have "netatmo" as their "software" tag, but I could not find this information in the response json.

All I could imagine right now would be to differientate between "airport" and "pws" nodes in the response.

[lolodomo]

The final question is how to identify a reliable and professional station.
I was surprised to see the "netatmo network" available in WU. I should compare the stations map from Netatmo and WU to check if all netatmo stations appear in WU.

[hakan42]

http://help.wunderground.com/knowledgebase/articles/838122-how-do-i-link-my-netatmo-station-with-weather-unde

"Due to a recent partnership between Weather Underground and Netatmo, weather data from all Netatmo stations that are registered through Netatmo appear on Weather Underground."

I read a few blog entries from people complaining about all the noise coming in from the badly calibrated netatmo devices, but wunderground seems to be adamant about keeping that data in their database.

[kaikreuzer]

I will also put the jar file somewhere for downloading and communicate the link.

I think this is the best for now. If you check the queue of bindings that has to be reviewed, it definitely won't be merged soon, especially as there are still questions open.
I agree with @cdjackson that we should not merge something now, only to completely change its use a short while later when having a solution for handling units.

For the about.html, where can I get the reference file ?

This is documented here: https://www.eclipse.org/legal/epl/about.php

[lolodomo]

PR #2748 pushed.

[pinkfish]

I think the way units should work, perhaps should pull that piece in a different issue? Is that the internals of the system should all use one unit. All the bindings use Celsius, use metric everywhere. In the ux you can set the unit you want as the display unit and the ux does the conversion on the edge to display it. We provide a nice helper class that does the conversions for you... Otherwise you end up with inconsistent values in a channel, one could be C and one could F so you cannot compare them properly. Or you end up with a whole whack of channels that have both C and F in them, which is also less than desirable. If the engine all uses SI units then all the automation and rules have a consistent way of interpreting the data. We could also put in some edge conversions in the rules code to handle F.

[kaikreuzer]

@pinkfish Please also note #1857, which will add the unit to the state. So in general I think it is ok if bindings provide the value in the unit that the device is actually providing - this way, we do not have any rounding issues and do not end up with "odd" numbers. Using the uom library would nonetheless allow us to do automatic conversion to any other unit that the user might want to use/see in UIs and rules.

[lolodomo]

We have now two weather bindings in ESH, the Yahoo weather binding and the WeatherUnderground binding.

As a reminder, here are the general opened discussions for all weather bindings:

• defining an iconset and mapping weather condition to this iconset in each weather binding
• using the unit management system still in development
• defining system channels that could be used by any weather binding

[9037568]

So, is it in the plan to implement the other weather bindings in ESH?

[lolodomo]

Once stuff relative to unit conversion is sorted, I will probably implement myself another weather binding.

[cweitkamp]

@lolodomo Any ideas which weather provider you prefer for your next implementation? I started work on my own version of an ESH binding for OpenWeatherMap. I could contribute if progress is sufficient.

The reason why I started this private project is to connect my Netatmo weather station to OWM using the recently introduced Station API.

[lolodomo]

Do you know that netatmo stations can already be available through WU ?

I will choose one free provider using JSON format as output to be able to mostly reuse the WU binding artefact.

[cweitkamp]

Yes, I read something about that the other day. Do you have experience with combining them? Is it possible to access my Netatmo station data with WU binding? I think we should resume this discussion in the OH2 community, shouldn't we? Is there a related topic?

[lolodomo]

To access your Netatmo station from WU, I think you simply have to setup your station to publish data and then select this station in the WU binding.

If you are working on a OpenWeatherMap binding, I could work myself on a World Weather Online binding. Normally, adding such a binding is just few days of work and probably even only few hours to get first data.

[cweitkamp]

Sounds easy. Maybe I try it later.

Your WU binding is a very good example for others to start. I follow your code base myself.

[lolodomo]

The 1.x version of the Weather binding provides a Common Id. Is it something we should consider through a channel for all 2.x weather bindings ? WDYT ?

[kaikreuzer]

I'd suggest to close this issue as we have dedicated bindings for different providers now.