Integrate with Ikea Trådfri Gateway as bridge to communicate and control connected Ikea's ZigBee based smart lights, switches, and sensors #570
Comments
Hedda
changed the title
|
@Hedda Those are really affordable! Pretty much $17 USD for each item. So, Ikea would need to publish an API that can be called for the ha-bridge to connect. We will need to keep looking into that. |
|
From this link: Zigbee lights should also work with other bridges like Philips Hue, Osram Lightify or Vera Plus |
|
Cool. The more companies building smart home items the more it should drive down competition. I do wish these were zwa e but I have a very plus so zigbee will be just fine for now. |
|
While it is cool that Ikea have released a gateway/hub on their own I would personally much rather skip buying the Ikea Trådfri Gateway and instead buy and install an ZigBee USB-adapter in my mini-PC that I currently use for as my DIY home automation hub with Linux to control each Ikea Tråd Lightbulb and Lightpanel directly. Problem with that simplter concept is that right now I don't know of which if any ZigBee USB-adapters and software libraries available that will work the ZLL (ZigBee Light Link) protocol which the Ikea Trådfri Lightbulbs and Lightpanels. Anyone here got some advice there? |
|
There's a user on the hue developers forum that said it works with the old hub 1.0. |
|
@emiliosic, Ikea Trådfri sensors and devices uses ZLL (ZigBee Light Link) protocol so will probably be made to work sooner or alter with hubs such as Samsung SmartThings Hub and Vera Plus. That is because those hubs are made by companies who made it the main concept for those hubs as commercial products to work with as many third-party devices as possible. That is, their goal is to make the hubs compatible with third-party sensors and devices. Samsung and Vera does officially support third-party devices. But that is not the goal of the Philips Hue and Osram Lightify hubs. Thier goal is only for those hubs to work with their own devices, which means that they do not do compatibility testing with third-party ZigBee devices connected to their hubs. So while some third-party sensors and devices might work because they too use the ZLL (ZigBee Light Link) protocol, is not the goal of Philips or Osram to make third-party devices work stable with their hubs, as that is not in their interest. Philips and Osram want people to buy their own devices, so officially they only support their own devices connected to their hubs. |
|
If the lights follow the standard there shouldn't be any development needed. Same goes for many Z-Wave devices which just work as generic. |
|
I believe what the OP intended is, will the ha-bridge support the Tradfri Gateway. This is a specific hub and will need an API to talk to. This Gateway would be on the level of the vera, wink hub or smartthings hub. |
|
@bwssytems That is exactly what I mean. Idea is that ha-bridge should be bridge to Ikea Trådfri Gateway. I did not mean that ha-bridge should communicate using ZigBee directly to the devices. Amazon Echo => ha-bridge => Ikea Trådfri Gateway => Ikea Trådfri Lightbulbs/Lightpanels. Google Home => ha-bridge => Ikea Trådfri Gateway => Ikea Trådfri Lightbulbs/Lightpanels. Harmony Hub => ha-bridge => Ikea Trådfri Gateway => Ikea Trådfri Lightbulbs/Lightpanels. Main problem with this is that we don't know if the Ikea Trådfri Gateway will offer a (documented) open API on the day-one of release. However I an interview with Ikea smart home development team from 6-months ago where they first leaked the news of plans to release Ikea Trådfri Gateway and in that article the Ikea developers specifically said that they do plan for the Ikea Trådfri Gateway to have an open API. So if it an open API not available in the initial firmware pre-installed day-one of release, they might release an over-the-air update for the firmware sooner or later which will offer an open API. And even if they might not officially offer support to third-parties that uses that API once available, the Ikea developers of this product have at least stated they do want interoperability with third-party smart home solutions. One possible option if the Ikea Trådfri Gateway does not offer an open API in the initial firmware pre-installed day-one of release is to hack the network communication between Ikea Trådfri Android/iOS application and the Ikea Trådfri Gateway, and then make library for ha-bridge software that can emulate an Ikea Trådfri Android/iOS app, similar to how ha-bridge software today emulate a Philips Hue Hub. That would of course be much more difficult and might not something that the developers of ha-bridge have any personal interest to dvelve into, but with the Ikea Trådfri series gateway and devices being so cheap and sold worldwide it will surley not be long before some other hacker hacks the communication between the Ikea Trådfri Android/iOS apps and the Ikea Trådfri Gateway and then publish that information or even code/library publicly for how to achieve exactly this. At it is several hackers usually have more insentive to hack stuff like this when a company does not offer an open API. Again, just look at the ha-bridge software and the Philips Hue Hub API. Here are some more decriptive diagrams with paths to put these concepts in perspective of ha-bridge: Amazon Echo => Hue API in ha-bridge => API by Ikea Trådfri Gateway => Ikea Trådfri device to control Google Home => Hue API in ha-bridge => API by Ikea Trådfri Gateway => Ikea Trådfri device to control Harmony Hub => Hue API in ha-bridge => API by Ikea Trådfri Gateway => Ikea Trådfri device to control |
Hedda
changed the title
|
The app is now downloadable on the google play store and apple app store , in case anyone wants to analyse it. I have tried to start it in a virtual machine but it just complains that im not connected to wifi, then i tried to start it in |
|
Has any one tried to dump the network packets leaving the trådfri app? (i don't have access to a physical android phone) |
|
@AnderssonPeter Tried to capture the packages from the app on my Android but tPacketCapture did not allow the Trrådfri app to work so I need another way to capture packages.. Also neither the iOS or Android app use any connection over HTTP what I could see when trying to capture req/resp. |
|
@AnderssonPeter maybe try loading Ikea Trådfri app in the Android Emulator? https://developer.android.com/studio/run/emulator.html You of course need to have to have a physical Ikea Trådfri Gateway pre-installed. But not sure if can enable WiFi in Android Emulator to local network though? https://www.techwalla.com/articles/how-to-enable-wifi-on-an-android-emulator If that works then should be able to use Wireshark to capture all IP packages That would perhaps be simplest, but again don't know if WiFi works in emulator. |
|
@Hedda i don't have the official android emulator installed as that requires java and i don't want all those nagging java update popups. I have tried with the visual studio android emulator, but none of the package capturing apps seem to work there sadly. I have also tried android x86, but there i ran into the wifi problem and when i tried a solution to add wifi or |
|
@AnderssonPeter You can install Android x86 in a virtual machine on VirtualBox or KVM https://www.howtogeek.com/164570/how-to-install-android-in-virtualbox/ |
|
@Hedda i had it up and running on hyper-v and it worked until i managed to brick it when trying to fake the WIFI. But ill try with VirtualBox as i have that installed on one of my PC's. |
|
Think you'll find Tradfri uses ZHA not ZLL. when IKEA stated they were Zigbee compliant they were, just using 'the other' protocol. Most of the other hubs (like the hue hub 2.0) use ZLL though i think home hub 1 can use ZHA with an old firmware. All it really means is don't rush out and buy some bulbs and expect them to work with your hue hub 2.0. (yet) there is a big post on the hue developers forum about it here: https://developers.meethue.com/content/philips-hue-and-ikea-tr%C3%A5dfri On the plus side, it sounds like IKEA are aware of this (post #45) and will remedy at some future point. |
|
From what I could see (proxy:ing the app traffic) it's using DTLS v1.2. I can see a "Hello" from the client and then an encryption request. After that all data is encrypted. If I understand it all correctly it's using a PSK. I.e. we'd need to extract that to be able to talk to the gateway. I'm in no way good at network communication so I might be completely wrong on this. |
|
@stenehall you are correct it is using DTLS, some analysis has started here: https://community.home-assistant.io/t/ikea-tradfri-gateway-zigbee/14788/8 |
|
Someone now looks to have figured out how exactly Android/iOS communicates with the Ikea Trådfri Gateway - Turns out it communicates using OMA LwM2M (Lightweight machine-to-machine) security model for IoT device management based on CoAP (Constrained Application Protocol) encrypted with DTLS (Datagram Transport Layer Security) using the PSK (Pre-Shared Key) written on under the physicial Ikea Trådfri Gateway box. https://bitsex.net/software/2017/coap-endpoints-on-ikea-tradfri/ https://bitsex.net/software/2017/ikea-tradfri-zigbee-lights/ This Norwegian guy as linked in two blog posts above have already figured out how monitor and send control commands to the Ikea Trådfri Gateway using standard CoAP to send and recieve commands to its end devices (using DTLS based encrypted communication to the Ikea Trådfri Gateway over network). https://tools.ietf.org/html/rfc7252 Ikea Trådfri Android application appears to be using multicast (224.0.0.1) to find the Ikea Trådfri Gateway, and then communicates using encrypted CoAP (coaps). Also, it does not look like the Trådfri Gateway attempts to talk to the Internet (as the device looks to have has no outgoing connections). The default Ikea Trådfri Android app is fairly basic, where it currently only let you create schedules for turning on and off, and you can control lights and create zones, and control zones. Looks like Ikea might have conformed with OMA LightweightM2M (LwM2M) Object IDs and Resource Registry ID as unique identifier For those who don’t know, LWM2M is a protocol built around CoAP and use for managing devices. So things like firmware upgrades, error reports, etc. Apart form the management interfaces, LWM2M also adds a very simple Object Model for managing those devices. IPSO expands that set of Objects so that you can have application information too (e.g. sensor readings, commands, etc). IPSO defines objects and resources that map to device properties. Particular pay attenton to the IPSO Light Control objects: https://github.com/IPSO-Alliance/pub/blob/master/reg/xml/3311.xml IPSO (IP for Smart Objects) Smart Object Guidelines provide a common design pattern: If you want to know more about the wealth of data models out there you can check the IoTSI Workshop as a reference. https://www.iab.org/activities/workshops/iotsi/ OMA LightweightM2M (LWM2M) standard: Interestingly is that CoAP (Constrained Application Protocol) is the protocol which the OCF (Open Connectivity Foundation) backed by most industry gians is promoting to become the standard for IoT. As reference in IoTivity and AllJoyn projects which guidelines one can suspect that Ikea have followed when they choose to go with standard CoAP in the Ikea Trådfri Gateway and its apps. |
|
I'm that norwegian guy. I was not aware of the standard, and I'm happy that you brought it to my attention. I was not aware of ha-bridge either, but it seems interesting. So far I've been a bit stumped by that zero of the COAP libraries for python supports dTLS, and I don't want to go the java route... I got a couple more bulbs today, and confirm that they add the same place as the previous. If I can be of any assistance, I'm happy to. If wanted I can provide a linux box with access to the gateway for anyone wanting to play. The key that is used for communication is printed beneath the gateway, so there's no need for any cracking; it's perfectly open, albeit using a not-yet-so-common protocol. |
|
Hi guys, You can easily talk to the gateway if you build the "dtls" branch of https://github.com/obgm/libcoap And then enter examples and do stuff like:
You can set the dimmer to something between 0-255. You can also query all the available endpoints:
|
|
@vidarlo Sound as Ikea have choosen to base their implementation on OMA (Open Mobile Alliance) and Eclipse recommended IoT protocol standards of those three logical components; CoAP, and DTLS layers of the LwM2M (Lightweight M2M) protocol stack. That is, looks as if the communication between Android/iOS app and the Gateway takes place via OMA Lightweight M2M (LwM2M) wrapped in CoAP with DTLS. "Lightweight M2M (LWM2M) is a system standard in the Open Mobile Alliance. It includes DTLS, CoAP, Block, Observe, SenML and Resource Directory and weaves them into a device-server interface along with an Object structure." http://openmobilealliance.org/data-models-for-the-internet-of-things/ https://connect2.io/open-mobile-alliance-lightweightm2m-oma-lwm2m/ http://openmobilealliance.org/constrained-application-protocol-coap-is-iots-modern-protocol/ http://openmobilealliance.org/data-models-for-the-internet-of-things/ https://connect2.io/open-mobile-alliance-lightweightm2m-oma-lwm2m/ https://iot.eclipse.org/standards/ https://eclipse.org/community/eclipse_newsletter/2014/february/article2.php The Wakaama project covers the LWM2M Protocol, CoAP, and DTLS layers of the LwM2M protocol stack for all three logical components. Wakaama is not a library but files to be built with an application. The Eclipse Wakaama project provides a C portable framework for building LWM2M clients and/or servers. The source code of Wakaama is available from the project webpage. It is written in C and designed to be portable on POSIX compliant systems. http://www.eclipse.org/wakaama/ You can also build the "dtls" branch of libcoap from: https://github.com/obgm/libcoap/tree/dtls Anjay - C implementation of the client-side OMA LwM2M protocol have very good documentation: https://avsystem.github.io/Anjay-doc/ Californium is another CoAP client from Eclipse (programmed in Java) which also supports DTLS https://eclipse.org/californium/ And the Eclipse Leshan project provides a Java implementation of LwM2M, allowing to build LwM2M servers and clients. The source code of Leshan is available from the project webpage. |
|
@arturo182 tinydtls-coap is another attempt to integrate the libcoap and tinydtls client-server https://github.com/thecodemaiden/tinydtls-coap You are then missing implementation of the LWM2M standard an top of CoAP. |
|
@Hedda, yes I have tried that one, it has the psk hardcoded and even when I changed that I still couldn't get a DTLS handshake, the dtls branch of libcoap works very well. |
|
Bought my Trådfri starter kit today (in Germany where it was launched shortly ago) and have been following this thread attentively, nice insights so far :) I would recommend doing: Then you can get available endpoints (from @arturo182): or use dimming: this dims one of my two bulbs (they are in different groups). ProcotolI think the communication takes place via OMA Lightweight M2M wrapped in CoAP with DTLS.
A library listed under https://en.wikipedia.org/wiki/OMA_LWM2M#Implementations may be a better approach than writing LwM2M by hand, I'm currently looking into that... |
|
@UserXYZ I believe it's an issue with libcoap, as we don't experience the issue with aiocoap over at ggravlingen/pytradfri |
|
Well, I guess then that the developer of libcoap made a bug that needs to be fixed, or he chose such behavior of coap-client deliberately, since that is supposed to be a test tool... What surprises me is that no other contributor decided to fix it, or at least make subscribe parameter with value of zero work as normal Observe:0 option, which would actually be a regular no-timeout subscription. Since I can't use Python 3 (still being under much of development etc), aiocoap is not a solution atm... So, anybody knows of another software that can be used instead of libcoap? Should be written in C/C++ (simplest to build), have dtls using tinydtls (gnutls is pain to include and I'm not sure if it works with keys besides certs) and pretty much everything else that libcoap has...? |
|
@UserXYZ You are always welcome to open issues on the libcoap bugtracker or discuss these on the libcoap mailing list. It is difficult to fix your issues if nobody knows about them. You can adapt the amount of output produced by coap-client on stdout with the option '-v'. Regarding the timeout: '-B' controls the time until coap-client terminates. By default, this is 90 seconds as documented in the manual page. The option -s controls the timeout until coap-client cancels the established observe relationship. Regarding your question about alternatives to libcoap: There is Californium which has DTLS support, and AFAIK, there is a version of node-coap with DTLS support. |
|
I came a cross a similar problem : Getting something not supported by ha-bridge working with ha-bridge. (Denon Amplifier, Samsung TV, 433Mhz based HA kit and more. Rather than delving into ha-bridge's code, I ended up writing my own php broker and ha-bridge just flings the http command at it and the broker sorts out the rest. Alexa / Google => ha-bridge => My Broker => Insert Weird Gateway => Weird Kit here. It appears from the above a similar approach is being attempted by some of you. The work carried out by Adreas Spiess may help : https://www.youtube.com/watch?v=wS8Vj0ba4ic. |
|
@davetaz I have exactly the same problem, my gateway is stuck on 1.0.0008 and says there are no updates. I tried reseting with the hole button but it stays in the same version. How did you make it to boot to the other image? |
|
Here are some of the things I did.
|
|
@lwis can you tell me how do you use aiocoap with ggravlingen/pytradfri code? In asynchronous mode, I suppose, because in synchronous mode it is just calling coap-client in the background (saw that in my process list) which kinda defeats the purpose of aiocoap... |
|
@lwis Well, that didn't quite answer my question, but based on your link, I'd say async mode... |
|
@UserXYZ Oh, you mean observation? Yes, coap-client has issues with observation, but aiocoap does not. You can observe until the connection is stopped at either end. |
|
@lwis Sweet, that's what I need...now just to figure out aiocoap's API to reimplement my code or switch totally to pytradfri as a base...oh, and learn Python3 alongside Python2...such fun... |
|
@UserXYZ obviously I'd recommend using pytradfri 😉 |
|
@lwis Weeell, that would be easier way to do it, for sure, but again it's also adding another layer. |
|
I found my GW, but it is not open on port 5684 The gateway is running version 1.2.42, claiming to be hue-compatible. I'm currently scanning all ports; none seem to be open. |
|
With 1.2.42 IKEA changed the authentication. See the following link: https://community.openhab.org/t/ikea-tradfri-gateway/26135/148?u=kai |
|
Where do you find the coap-client? I got the one from libcoap-1-0-bin in debian. ` ` I'm trying to access the hub from node-red, and it will not authenticate.
|
|
For testing of the gateway purpose i'm using the californium.tools (https://github.com/eclipse/californium.tools) and in my program i use the coap client implementation of the californium library. |
|
Hi, I've been following this (nice) discussion from the very beginning. The thing is that I would be interested in running some (source) code directly (bypassing the gateway) on the bulbs. Do you think it would be possible? (a piece of software running continuously in the bulb... if the temperature of the bulb falls down, then change the color to blue) |
|
Hello! Tons of useful info and links in the thread, thanks! I have been playing with the new Trådfri Color bulb trying to figure out why it would not show the whole spectrum and found the answer by cracking one open. It does not have green LEDs. I have put some code, info and pictures here: https://github.com/ffleurey/ThingML-Tradfri so that you do not have to vut your own bulb :-) Cheers! |
|
@ffleurey Really nice library. Great that you have put the effort in to make the UI, really fun and works great. |
|
Very informative thread. I'm trying to generalize some of the findings above into a handy CoAP Shell (https://github.com/tzolov/coap-shell) with some initial Ikea gateway support as well. |
|
With the implementation of OpenHAB, Which has Tradfri connectivity, I will suggest that is the path to implement this. |
|
^ Please report @SamSpringPackaging for spam (go to the profile and click 'Block or report user') |
|
@vandenberghev Thanks! He is reported and banned from BWS Systems |
and others
Ikea have just released a new app-controlled network-attached home automation hub which will serve as a Gateway to control its new "Trådfri" series of affordable smart lights / lightbulbs, switches / remotes, and sensors, which in turn so far all uses ZigBee based protocols. These products are set to be released on the 31st of March 2017 in selected countries around the world.
https://www.cnet.com/news/ikeas-rolling-out-a-brand-new-smart-home-lineup/
http://www.ikea.com/ms/sv_SE/customer-service/about-our-products/smart-lighting/index.html
"Trådfri" means 'wireless' in Swedish, and Ikea have so far announced this very aggresivly low-priced network-attached (Ethernet) "Ikea Trådfri Gateway" home automation hub in their "Tradfri" series, as well as a wireless Motion Sensor Kit (that have integrated light sensor too), a wireless Dimmer Remote (which is accelerator-based), a wireless multi-switch remote, and several smart light bulbs of different formats and even a few unique panel lights. All these products will then be released in most other contries worldwide too as Ikea steps up manufacturing (and irons our the initial software bugs I guess).
Ikea had already leaked news about this upcoming gateway/hub more than 6-months ago, during the summer or 2016, and at that time they also revealved that they will use ZigBee and keep validated access to the gateway/hub as open as possible, including providing an open API for this network-attached home automation hub.
Ikea in Sweden are first to post this news about the network-connected Home Automation Gateway / Hub, but again these products will become available worldwide. Here is link to the Swedish links:
http://www.ikea.com/se/sv/catalog/categories/departments/living_room/36812/
http://www.ikea.com/se/sv/catalog/products/40337806/
http://www.ikea.com/se/sv/catalog/products/80338960/
http://www.ikea.com/se/sv/catalog/products/80338941/
http://www.ikea.com/se/sv/catalog/products/80349888/
Link are in Swedish for Ikea Sweden site, but the PDF manuals on each page are available in English and many more languages, however they don't say much other than mounting instructions.
http://www.ikea.com/ms/en_US/img/buying_guides/fy17/april/Home_Smart_lighting_Buying_guide_APRIL1.pdf
Reason why I think that this news being interested is Ikea's aggresive pricing might them the first to make two-way communication home automation really affordable for almost everyone while still following all the electrical safety and wireless communications regulations in all countries, as they are today well known to have very low prices yet good manufacturing quality items.
UPDATE 1: Sound as Ikea Trådfri Gateway software uses the Cypress WICED IoT platform SDK (formerly Broadcom WICED IoT platform before acquired by Cypress http://www.cypress.com/internet-things-iot ) and have choosen to base their implementation on OMA (Open Mobile Alliance) and Eclipse recommended IoT protocol standards of those three logical components; CoAP (coaps) and DTLS layers of the LwM2M (Lightweight machine-to-machine) security model for IoT device management and protocol stack, using IPSO (IP for Smart Objects) Smart Object Guidelines provide a common design pattern. That is, looks as if the communication between Android/iOS app and the Gateway takes place via OMA Lightweight M2M (LwM2M) wrapped in CoAP with DTLS.
http://openmobilealliance.org/data-models-for-the-internet-of-things/
Update 2: Jaime Jiménez (from the company Ericsson) who is an active member of the IPSO Alliance’s working group and part of the team that published the IPSO Smart Object Guidelines, posted this great teardown of the Ikea Trådfri implementation:
http://jaimejim.github.io/tradfri/
For those who don’t know, LWM2M is a protocol built around CoAP and use for managing devices. So things like firmware upgrades, error reports, etc. Apart form the management interfaces, LWM2M also adds a very simple Object Model for managing those devices. IPSO expands that set of Objects so that you can have application information too (e.g. sensor readings, commands, etc). IPSO defines objects and resources that map to device properties.
Particular pay attention to the IPSO Light Control objects:
https://github.com/IPSO-Alliance/pub/blob/master/reg/xml/3311.xml
If you want to know more about the wealth of data models out there you can check the IoTSI Workshop as a reference.
https://www.iab.org/activities/workshops/iotsi/
LwM2M (Lightweight machine-to-machine) meanwhile is a system standard in the Open Mobile Alliance. It includes DTLS, CoAP, Block, Observe, SenML and Resource Directory and weaves them into a device-server interface along with an Objects structure based on IPSO Smart Object Guidelines.
https://connect2.io/open-mobile-alliance-lightweightm2m-oma-lwm2m/
https://www.eclipsecon.org/na2014/sites/default/files/slides/Eclipsecon%20NA14%20-%20One%20protocol%20to%20rule%20them%20all-%20(1).pdf
http://openmobilealliance.org/constrained-application-protocol-coap-is-iots-modern-protocol/
IPSO provide common object model for interoperability of IoT Devices and Applications.
https://github.com/IPSO-Alliance/pub/blob/master/reg/README.md
https://github.com/IPSO-Alliance/pub/tree/master/reg/xml
https://github.com/IPSO-Alliance/pub/blob/master/reg/xml/3311.xml
https://github.com/IPSO-Alliance/pub
The text was updated successfully, but these errors were encountered: