[HueEmulation] Support Coloured bulbs & other improvements

[davidgraeff]

• Use OSGI annotations
• Use config holder class
• Move code into own subclasses instead of having a giant HueEmulationServlet.class:
  • Introduce UDN class
  • Introduce user management class
  • Introduce light items management class
• Watch the item registry for changes and keep a cache of published items,
  instead of quering all items from the item registry for each API call.
• Implement /config endpoint. Necessary for Hue App etc.
• Implement dummy entpoints for nowadays Hue bridges (/group, /scenes etc).

Signed-off-by: David Graeff david.graeff@web.de

Fixes #3753
Fixes #3754
Fixes #3118
Fixes #2858
Fixes #2005
Fixes #1317
Fixes #1311
Fixes #1257
Fixes #2881
Fixes #2936
Fixes #3171

[kaikreuzer]

Thanks @davidgraeff, just a little cosmetic left.

[davidgraeff]

@kaikreuzer Ready for another review round

[kaikreuzer]

Thanks!

[openhab-bot]

This pull request has been mentioned on openHAB Community. There might be relevant details there:

https://community.openhab.org/t/items-with-switchable-tag-are-not-discovered/58250/1

[gdgeist]

#3171 is not fixed by this. Setting to 1 still results in an effective off.
public class HueStateBulb extends HueStatePlug { // https://github.com/openhab/openhab2-addons/issues/2881 // Apparently the maximum brightness is 254 public static int MAX_BRI = 254;
yes its 0-254 but you dont normalize that when you do a later calculation for the 1-100 resulting in a fraction thats rounded down i.e. 1 is always 0.

And a hearty thanks for this work HSBtype support ---woot!

[5iver]

Details in #4293 and in the forum post... just noting it here to bring attention to the issues before the release. IMO, removing the ability to use groups needs to be resolved or communicated as a breaking change. There have been several reports of the inability to discover Items tagged as Switchable. I suggest these get remedied before 2.4 is released.

[davidgraeff]

This is not a regression, as the hue emulation never supported groups before. @openhab-5iver Could you please open an issue for group support :)

Regarding switchables. I'm having a look at this right. My plan is to pair an osram plug to an original hue bridge and looking at the rest api response to spot any differences in how switchables are exposed.

Before my changes everything was exposed as a light. I don't thing this is the way to go for switchables.

If certain Alexa firmware versions or devices (personally I do have dots and echos with the German firmware) are not able to recognise Osram plugs, there is not much that can be done.

[davidgraeff]

Ok this is the response of the hue bridge for a paired osram plug that works with my local Alexa setup with 1 x 2gen dot and 2 x 2gen Echoes:

{
    "lights": {
        "1": {
            "state": {
                "on": true,
                "alert": "none",
                "mode": "homeautomation",
                "reachable": true
            },
            "swupdate": {
                "state": "notupdatable",
                "lastinstall": null
            },
            "type": "On/Off plug-in unit",
            "name": "On/Off plug 1",
            "modelid": "Plug 01",
            "manufacturername": "OSRAM",
            "productname": "On/Off plug",
            "capabilities": {
                "certified": false,
                "control": {
                },
                "streaming": {
                    "renderer": false,
                    "proxy": false
                }
            },
            "config": {
                "archetype": "classicbulb",
                "function": "functional",
                "direction": "omnidirectional"
            },
            "uniqueid": "7c:b0:3e:aa:00:a6:fc:3a-03",
            "swversion": "V1.04.12"
        }
    },
    "groups": {
    },
    "config": {
        "name": "Philips hue",
        "zigbeechannel": 11,
        "bridgeid": "-removed-",
        "mac": "-removed-",
        "dhcp": true,
        "ipaddress": "192.168.0.178",
        "netmask": "255.255.255.0",
        "gateway": "192.168.0.1",
        "proxyaddress": "none",
        "proxyport": 0,
        "UTC": "2018-12-02T16:11:20",
        "localtime": "2018-12-02T17:11:20",
        "timezone": "Europe/Berlin",
        "modelid": "BSB002",
        "datastoreversion": "74",
        "swversion": "1810251352",
        "apiversion": "1.28.0",
        "swupdate": {
            "updatestate": 0,
            "checkforupdate": false,
            "devicetypes": {
                "bridge": false,
                "lights": [
                ],
                "sensors": [
                ]
            },
            "url": "",
            "text": "",
            "notify": true
        },
        "swupdate2": {
            "checkforupdate": false,
            "lastchange": "2018-12-02T16:10:18",
            "bridge": {
                "state": "noupdates",
                "lastinstall": "2018-12-02T15:33:46"
            },
            "state": "noupdates",
            "autoinstall": {
                "updatetime": "T14:00:00",
                "on": false
            }
        },
        "linkbutton": false,
        "portalservices": true,
        "portalconnection": "connected",
        "portalstate": {
            "signedon": true,
            "incoming": false,
            "outgoing": true,
            "communication": "disconnected"
        },
        "internetservices": {
            "internet": "connected",
            "remoteaccess": "connected",
            "time": "connected",
            "swupdate": "connected"
        },
        "factorynew": false,
        "replacesbridgeid": null,
        "backup": {
            "status": "idle",
            "errorcode": 0
        },
        "starterkitid": "",
        "whitelist": {
            RfnIhUKHNmTknBRSSIwXcmz1JCjPAVNtpsFoC5FI":{
"last use date": "2018-12-02T16:17:49",
"create date": "2018-12-02T16:16:55",
"name": "Echo"
}
        }
    },
    "schedules": {
    },
    "scenes": {
    },
    "rules": {
    },
    "sensors": {
        "1": {
            "state": {
                "daylight": null,
                "lastupdated": "none"
            },
            "config": {
                "on": true,
                "configured": false,
                "sunriseoffset": 30,
                "sunsetoffset": -30
            },
            "name": "Daylight",
            "type": "Daylight",
            "modelid": "PHDL00",
            "manufacturername": "Philips",
            "swversion": "1.0"
        }
    },
    "resourcelinks": {
    }
}

The hue bridge additionally sends "mode": "homeautomation", "capabilities" and "config". I'll add those keys to the hue emulation response.

[davidgraeff]

You are basically saying that I have not performed a test run before publishing. That is harsh criticism, I'm sorry but there's no other way to interpret that.

And you did it again in your last post. Please just accept the fact that hue essentials works on my phone in my local network and all my Alexa's are happily talking to my openHAB via the hue emulation.

And yes parsing of this specific username message is performed by string comparisons in a few hue rest implementations, and that's why it slipped my eyes here.

And we totally disagree about coding style apparently. I like a well commented and extendable DTO class, were fields can be updated if required, that is converted to a JSON on demand and you prefer string building which of course also works. But that doesn't reduce maintenance complexity. Have you seen the config class with a dozen fields? That will not look pretty with your string builder pattern.

I'm lacking time right now, but I will fix the most crucial bugs like the wrong response.

[Flole998]

I am saying that it looks like a not well tested PR, I never said you didn't do a test run. I have tested this on 3 independent devices, all of them didn't work. There's no way to say that it's been well tested as then the official app and the hue essentials app would work. But if you did indeed a complete test run on a fresh install and this doesn't work on any of the apps right now on my side your or my test scenario seems to be a little odd or the code was changed somewhere along the way.

It might work right now for you but if you have to re-setup it you will most likely not be able to, at least not with the hue essentials app (if you are that's really interesting and I'll definitely do further debugging to figure out how that json parser is able to accept something like that and find the correct data). I don't know what the echo devices are doing in this case, I would have to look that up but if I remember correctly we have a proper parser in there that'll fail in this case aswell.

I am also not preferring string building but json arrays/objects that get arrays/objects "put'ed" into them and in the end a toString call if it's necessary. To be honest I have no clue right now how i would fix the broken response in your code, I'm definitely looking forward to the new PR to see how that's done.

Also I've written in one of the issues that Philips devices (like my TV) don't care about the path advertised for the description.xml file, so they're requesting /description.xml everytime instead of /api/description.xml . It would be great if you could do that small change aswell, I have already looked into that and I could do it aswell but as I said I would prefer to not mess with your code until it's working properly as then neither you nor I know what's going on in there which makes it more difficult to find issues.

Thanks for taking time to fix this, I'm looking forward to this!

[davidgraeff]

There is one way to catch those errors. A test bundle. If you have some time on your hands, but don't want to fiddle with the emulation code right now, you could help a lot by setting up a test bundle :)

I will tackle item group support (in a hacky way for this release only) and this wrong response. I still don't know why switchable itema are not working for some people. That would need to be sorted as well before a release.

[5iver]

@openhab-5iver Could you please open an issue for group support :)

#4307

[davidgraeff]

Also I've written in one of the issues that Philips devices (like my TV) don't care about the path advertised for the description.xml file, so they're requesting /description.xml everytime instead of /api/description.xml . It would be great if you could do that small change aswell, I have already looked into that and I could do it aswell but as I said I would prefer to not mess with your code

@Flole998 This is not possible afaik. The hue emulation registers a java servlet on /api and can only serve content beneath this path. If you know a framework api call to register a single file on the root topic, please propose a PR with that change.

[davidgraeff]

I have made a fix for the wrong response in #4311

[xsnrg]

If it helps, Hue Emulation no longer discovers devices on my network either. I have tried using both Switchable and Lighting (I had always used Lighting previously) in my otherwise unchanged insteon configuration. This is a version 1 binding, so it only uses flat files. I cleared out all of my devices from the Alexa app, have tried setting the binding to try the discovery fix and without. I also changed the timeout to 5 minutes, though this was only for my sanity of not going back into the binding configuration to re-enable it after every test.

Using wireshark I am seeing the request to GET /api/description.xml and a 200 is returned, this all being after the ssdp traffic of course. The data returned from the GET is fairly sparse. It contains only information about the emulated hub and a couple icon entries. The very next request looks like this:
GET /api/<random string key>/lights but it always returns a 404 with the description "Hue resource not available"

If the pairing button has been pushed (the binding is in pairing mode), it still gets that error, but then goes on and posts the devicetype "Echo". This returns a 200 with the username, a UUID. It then tries a GET like the previous one only with a new key GET /api/<new string key>/lights. This is a 404 again, as is every request after.

I am running the latest SNAPSHOT, build 1448, and have about 16 devices that worked previously. The bridge is returning success with the username, so that verifies the code is recent. What else can I provide to be helpful?

[davidgraeff]

Thank you for your report. After one of the last commits the emulation stopped working for me as well. I'm on this right now.

[Flole998]

Updated to M8 today and what should I say: Amazing...... The description.xml made it to /description.xml, so my TV found the OpenHAB Hue Bridge, the lights got detected correctly and I am able to use them as an Ambilight. Unfortunately, there's one small issue: The color does not work, as the xy stuff is not implemented (see https://gist.github.com/popcorn245/30afa0f98eea1c2fd34d for some explanation on how that works) yet, I am looking into this right now to see how this could be done easily (and hopefully that still makes it into the 2.4.0 release then). That's the only thing that I'm missing right now, other than that this is absolutely great!! Thanks for all the work, I really appreciate it!!

[Flole998]

@davidgraeff If you have a second, could you please check my implementation at Flole998/openhab2-addons@b3ea45a and Flole998/openhab2-addons@6f06291 and let me know if that's how it is intended (and maybe also check if it works for you), and if it is feel free to pull it over so we can get this merged as quick as possible :)

A valid request looks like this

PUT /api/apikey/lights/1/state HTTP/1.1
Content-type: application/json
Accept: application/json
User-Agent: SomeAgent
Host: my.openhab.local
Connection: Keep-Alive
Accept-Encoding: gzip
Content-Length: 64

{"transitiontime":5,"xy":[0.3002627491950989,0.317052960395813]}

[Flole998]

Actually my changes are dumb/unnecessary in some parts, we already have fromXY(x, y) in HSB type, so all that is really necessary is "return HSBType().fromXY(xy[0], xy[1]);" and not copying all that stuff over.... That basically means: Disregard the second commit for now, I'll fix that one.

EDIT: Fixed and Updated the previous comment

[davidgraeff]

Those commits are awesome. I'm not sure if Kai will accept feature commits at this point in time anymore. The release notes have been written already. I'll try anyway.

[Flole998]

Someone just brought to my attention that things are not quite right yet:

• no Answer is being sent, just a json sucess.
• Warm colors are slightly inaccurate

[davidgraeff]

Could you elaborate on the first list item?
What endpoint responds without an actual message.

[Flole998]

I've taken some time to work on the slightly inaccurate colors today (aswell as my first xy_inc implementation was wrong), my repo has that fixed now. Technically all this is not "new", but fixed from your incomplete implementation now. The changelog doesn't need to get rewritten, and I'd say even an "untested" implementation is better than none.

The first Problem is specific to my implementation (sorry if that wasn't clear): In your Response Handler you have several responses for different instances, the Problem with xy is, that it is a list and there is no case for that (and it is very difficult to implement). I haven't found a way for that yet.

(And maybe someone wants to look at my calculations and simplify that a little, I'm sure there's room for optimization....)

[davidgraeff]

Can you make a PR out of your changes? If you include a test case for xy for either the RestAPI unit test or the OSGI full implementation test, we can get this in super fast.

[Flole998]

PR created, I also made a testcase based on one of yours, I am not checking the results of my color conversion though, hope this is okay like this.

[openhab-bot]

This pull request has been mentioned on openHAB Community. There might be relevant details there:

https://community.openhab.org/t/alexa-cannot-find-any-devices-anymore-with-the-hueemulation-service/61400/9