groups/0 (all devices) not working

[popy2k14]

Describe the bug

Yesterday i have updated to 2.05.88 from 02.05.84 and now my groups/0 doesnt work at all.
I cant set it with phoscon "all off switch" or over rest api.
There is a 404 error with HUEEssentials.

Steps to reproduce the behavior

Switch the group 0 as stated above.

Expected behavior

All lights should be controlable with group 0.

Screenshots

Environment

02.05.84 (downgraded again)
26660700

• Host system: Raspberry Pi
• Running method: Raspbian
• Firmware version: 26660700
• deCONZ version: 2.05.84 and 02.05.88
• Device: ConBee II
• Do you use an USB extension cable: yes
• Is there any other USB or serial devices connected to the host system? If so: Which? yes SSD

deCONZ Logs

Additional context

When ill do a get on group "0" manuall with the browser the answer is:

GET: http://192.168.0.9:8080/api/XXXXXXXXXX/groups/0
Answer: [{"error":{"address":"/groups/0","description":"resource, /groups/0, not available","type":3}}]

With other groups all is working.

ANy hints how i can get my group/0 working again?

[MattWestb]

Try with 0xFFF0 = 65520 that is the "new" groupe Al

[popy2k14]

Thank you for your help.
Sadly, not working on 2.05.84 after downgrade from 02.05.88.
result is:

[{"error":{"address":"/groups/65520","description":"resource, /groups/65520, not available","type":3}}]

Any hints?
Do i have to upgrade to 02.05.88 again to get it working?

PS.: why should the groups/0 be changed?

[MattWestb]

Then taking on look in zigbee.db in the table group wot number "All" is having but not the one that is marked "deleted"

[MattWestb]

About group 0 is in Zigbee 3 reserved for global scene commands and not as one trashcan for HA devices.

[popy2k14]

ok, can you please guide me in the right direction regarding zigbee.db.
Dont want to mess something up.

I just updated deconz yesterday to 02.05.88 and now my groups/0 is gone :-(

[MattWestb]

Sorry I was having wrong with the name but on my Pi its installed in
/home/pi/.local/share/dresden-elektronik/deCONZ/zll.db

I have trying activating group 0 but deCONZ is marking deleted and automatic adding the 0xfff0 after restart for some month ago.

The 0xff09 is used by IKEAs 5 button remote scenes.

[popy2k14]

I just have these ones...
No group 0 deleted, but an 0xfff0 one named "All" which is marked as deleted.

How can i get it back to a working state? just set it to normal?

Would it be accessible with groups/0 again over the rest api?

[popy2k14]

Now when i'll do an:

http://192.168.0.9:8080/api/XXXXXXXXXX/groups/65520

i'll get:

{"action":{"alert":"none","bri":127,"colormode":"hs","ct":0,"effect":"none","hue":0,"on":false,"sat":127,"scene":null,"xy":[0,0]},"devicemembership":[],"etag":"xxxxxxxxxxxxxxxxxxxxxxxxxxx","id":"65520","lights":["3","2","1","4","7","6","5","8","9","10","11","12","13","14","16","15","19","18","17","20","21","25","22","23","24","26","27","28","29","30","31","32","33","34","35","36","37","38","39","40","42","43","44","45","46","48","49","50","51","52","53","41"],"name":"All","scenes":[],"state":{"all_on":false,"any_on":true},"type":"LightGroup"}

So it seems the all group is working under id 65520 now.

Sadly the "Power off" button in phoscon, HUEEssentials ... does not work because they will switch "0" instead of 65520.

Does it work when i just rename it to "0" in the sql table?

[MattWestb]

Its looks little normal deCONZ.
You can trying changing "deteletd" to "normal" but I deCONZ is marking it deleted after next reboot.
My installation have doing the 0xfff0 and giving it the name All.
You can doing one new groupe "3" and adding all lights you like in it (its not adding new lights in it) or manual adding 0xfff0 and se wath deCONZ is doing on the next boot.

One warning only saving the DB then deCONZ is not running or it is overwriting you changes.

I think the system was little sleepy adding it after restart :-))

The last Q you can try but me experience is that deCONZ is redoing it to deleted after next boot.

[MattWestb]

Then you is having around 50 lights you should using real light groupe that is bounded to your remotes like the HUE dimmer switch so they is working if deCONZ is offline or having problems !!!

[manup]

Found the related commit which deletes group 0x0000 and adds a new non zero group which mostly happens to be 0xfff0:
590a8f1

Devices are added to the new group 0xfff0.

The REST-API group "0" is mapped to Zigbee group 0xfff0.

I need to find the discussion behind this, not using (Zigbee) group 0x0000 has reasons but adding routers to that new 0xfff0 group has shown it's side effects and problems recently. I think it's best to use broadcast instead group cast when controlling REST-API group "0".

[popy2k14]

@MattWestb:
Thanks for all your help.
Now i am back at 02.05.88.

I have now changed the 0xfff0 to 0x0000, stopped deconz before and than restarted.
Group 0 was working for a short while. A few minutes later -> group 0 gone again (marked as deleted).
And also 0xfff0 was added, marked as deleted.
So now i have no zigbee "All" group at all.

@manup
Thanks for stepping in.
Is there a workaround how i can get group "0" to work again and not have to change it everywhere (fhem, HUE Essentials...)
Maybe downgrade to 02.05.84, change the zll.db?

thx

[manup]

Actual group Zigbee group 0x0000 was never used, the REST-API group "0" was transformed into a broadcast to all routers, which I think is good, but likely there were other problems leading to the change.

Note that change was made over a year ago long before v2.5.84.

[popy2k14]

thx, got you.
So now i have done the following to get REST-API group "0" working again:

• let the 0x0000 zigbee group in the zll.db marked as "deleted"
• deconz now added the 0xfff0 zigbee group with name "all" (dont know why it was no present after upgrading the first time)
  This group also was makred as deleted in the DB.
• i have now stopped deconz, set it to "normal" and restarted it

Until now (~5 Minutes) REST-API group "0" is working again.

Now its gone again :-( REST-API group "0" not working.

thx

[popy2k14]

Now both are deleted.
Seems deconz running an "job" which deletes my ALL groups.
After starting deconz, the group is working (all lights are switching) for a few minutes until it returns:

[{"error":{"address":"/groups/0","description":"resource, /groups/0, not available","type":3}}]

After that the DB is looking like this:

How can i get it to work again?

[MattWestb]

Great that Manup was not sleeping ;-))

In the latest Zigbee 3 papers is groupe 0 is reserved for scene command (globale one i think) and it was added then LL (light and occupancy) was added to ZCL. Its more zigbee groupes that is reserved at 0xffxx somthing and to 0xffff but i cant finding the paper its written for the moment (its about using touchlink in zigbee3 from NXP and sirlabs).

[MattWestb]

@popy2k14 If i reading the code change and the discussion is the groupe 0xfff0 in the database linked to the "group 0" in the API. The groupe 0 is inactivated and 0xfff0 added if not there after restart.
From the discussion for the PR: Pore Philips HUE is not working with zigbee groupe 0 Little bad but OK in ZLL and ZB3) and OSRAM is not working with groups over 0xfffX (also OK in ZLL and ZB3) -> 0xfff0.

deCONZ can taking little time doing some magic after restart but if all is working the groupe 0 sould working in the API that hue essential is using (i dont like tapping on the switch in my handy then its being little dark all over then so i have not testing if its working).

My feeling is that the adding and deleting is working in the DB but perhaps not the linking for the API calls. "Some kind of magic (Queen)"

Then you is restarting deCONZ open the new debug window and you can see how the system is reading the groups from the DB and adding the lights in it.

[popy2k14]

yeah i know, but the issue is that it "deletes" zigbee 0x0000 (which is ok because 0xfff0 is used) and 0xfff0 after a few minutes of deconz start.

So i can not use REST-API groups/0 (which should be internally 0xfff0).

How can i disable/prevent this behaviour?

[MattWestb]

If you having windows: Format C:

Sorry I think its one bug because its written in the discussion to the PR how it is working but perhaps is not tested in real then the DB part is working.
I think @manup must taking one look on it.

As work around you can do one new normal groupe and adding all your (around 50) lights in it and rewriting your automations AND hope its works OK.

I was finding theise behaviour for 3 weeks ago then sniffing IKEAs remote and the scene select function that is not working. Was deleting all groups in one light and some seconds the 0xfff0 was automatically added by deCONZ all the time and i was hunting the groupe 0xff09. The DB i have seeing the deleting / cleaning the DB from crap after moving devices to ZHA.

[MattWestb]

@popy2k14 Pylips looks very interesting if having one Philips TV !!!

[KodeCR]

That's my commit. Sorry I broke things for you.

There is a config setting which stores in the database which zigbee groupid (normally 65520) is used for rest api group 0. This is in the database as under table "config2", variable "group0". If "group0" is 0, the zigbee group is deleted and a new one is created. Sounds like something is going wrong there. Could you find out what you have in the database for config2/group0?

[popy2k14]

@KodeCR no problem :-) i can help debug
It seems that your code is always triggered but also deleting 65520?
Here is the db entry:

[MattWestb]

I dont have that proble then i dont using API (Only HUE Essentials) bit this os the record:

"14" "group0" "65520"

I think the DB part is working but not the linking to API.

[popy2k14]

api linking is working! i can access REST-API "groups/0" after fixing the DB (set deleted to normal) and starting deconz.
But after a few minutes REST-API "groups/0" isnt accessible again and DB flag is set to "deleted" on group 0x0000 and 0xfff0.

Currently fetching an debug log....

[MattWestb]

In HA i have the IKEA scene defalt groupe "Group 65289" from deCONZ but no 0 or 0xfff0 perhaps is hidden or the linking is not working to API.

[KodeCR]

Thanks, I can't see in the commit I made how it would be deleted. In the original commit that only happened if group0 from the database was 0.

I did commit a "fix", but that one deleted the group only if the zigbee address is 0.

So it's working right after starting deCONZ?

[popy2k14]

After "fixing" DB (set deleted to normal) on 0xfff0 entry and starting deconz it works for a few minutes.
Should ill set the config2/group0 entry to "0" and the All group to 0x0000 and try again?

[popy2k14]

i think i found the issue, see here from the debug log:

22:01:57:017 [INFO] - No button handler for: FYRTUR block-out roller blind endpoint: 0x01 cluster: BASIC (0x0000) command: 0x0A payload[0]: 000
22:01:57:017 ZCL attribute report 0x680AE2FFFE36D57F for cluster: 0x0000, ep: 0x01, frame control: 0x08, mfcode: 0x0000
22:01:57:017 payload: 00404207322e322e303039
22:01:57:031 Node data 0x680ae2fffe36d57f profileId: 0x0104, clusterId: 0x0000
22:01:57:032 Update Sensor 0x680AE2FFFE36D57F Basic Cluster
22:01:57:032 0x680AE2FFFE36D57F: update ZCL value 0x01/0x0000/0x4000 after 0 s
22:01:57:032 [INFO] - No button handler for: FYRTUR block-out roller blind endpoint: 0x01 cluster: BASIC (0x0000) command: 0x0A payload[0]: 000
22:01:57:032 ZCL attribute report 0x680AE2FFFE36D57F for cluster: 0x0000, ep: 0x01, frame control: 0x08, mfcode: 0x0000
22:01:57:032 payload: 00404207322e322e303039
22:01:57:034 Websocket 192.168.0.1:57850 send message: {"config":{"group":"65520","on":true,"reachable":true},"e":"changed","id":"77","r":"sensors","t":"event","uniqueid":"68:0a:e2:ff:fe:36:d5:7f-01-0001"} (ret = 32410024)
22:01:57:035 Websocket 192.168.0.9:33928 send message: {"config":{"group":"65520","on":true,"reachable":true},"e":"changed","id":"77","r":"sensors","t":"event","uniqueid":"68:0a:e2:ff:fe:36:d5:7f-01-0001"} (ret = 32410024)
22:01:57:037 attach group 65520 to sensor 77
22:01:57:077 Node data 0x680ae2fffe36d57f profileId: 0x0104, clusterId: 0x0000
22:01:57:077 Update Sensor 0x680AE2FFFE36D57F Basic Cluster
22:01:57:078 0x680AE2FFFE36D57F: update ZCL value 0x01/0x0000/0x4000 after 0 s
22:01:57:078 [INFO] - No button handler for: FYRTUR block-out roller blind endpoint: 0x01 cluster: BASIC (0x0000) command: 0x0A payload[0]: 000
22:01:57:078 ZCL attribute report 0x680AE2FFFE36D57F for cluster: 0x0000, ep: 0x01, frame control: 0x08, mfcode: 0x0000
22:01:57:078 payload: 00404207322e322e303039
22:01:57:080 Websocket 192.168.0.1:57850 send message: {"config":{"group":"14","on":true,"reachable":true},"e":"changed","id":"77","r":"sensors","t":"event","uniqueid":"68:0a:e2:ff:fe:36:d5:7f-01-0001"} (ret = 32410024)
22:01:57:080 Websocket 192.168.0.9:33928 send message: {"config":{"group":"14","on":true,"reachable":true},"e":"changed","id":"77","r":"sensors","t":"event","uniqueid":"68:0a:e2:ff:fe:36:d5:7f-01-0001"} (ret = 32410024)
22:01:57:082 delete old group 65520 of sensor Rollo Julian
22:01:57:082 attach group 14 to sensor 77

The interesting part is:

22:01:57:082 delete old group 65520 of sensor Rollo Julian

Thats an ikea fyrtur rollo and deconz deletes the 0xfff0 (65520) group here.

Any hints why?

[KodeCR]

Having config2/group0 set to 65520, and having the 0xfff0 group entry as "normal" is the default behaviour and how it is currently working for me. I need to figure out why the group is deleted after a while. Perhaps the debug log gives some information. I'm afraid I'll have to look at it further tomorrow though.

[KodeCR]

Ok, I know what is going on now, and I've changed the code to only add lights to the all group, hence the blinds will never be added. Could you please test the following plugin based on the 2.8.0 release:
libde_rest_plugin.so.zip
(sudo cp ../libde_rest_plugin.so /usr/share/deCONZ/plugins)

I expect that the group0 will be removed one more time removing the group from the blinds, and then will stay after that.

Sorry for the wait.

[popy2k14]

@KodeCR thx a lot. Don't worry about the delay in this crazy times.

Can I test/use this plugin on my raspbian 02.07.01 installation? (because it's based on 2.8.0)

Thx

[popy2k14]

@KodeCR using your rest plugin now for 15 minutes in my 02.07.02 system and group 0 was not deleted.
I would say issue fixed, thx a lot!!!!
Will take a look in a few hours again and come back here.

Will the PR be merged into the next release?

[popy2k14]

Seems good for me also after ~3h.
thx a lot.

[KodeCR]

Great, thanks for testing and confirming.

The PR will need to be reviewed before it's merged, so assuming there aren't any unforeseen issues, probably next release, or the one after.

[popy2k14]

Nice, thx for the info.
Keep up the good work.

[stale]

As there hasn't been any response in 21 days, this issue has been automatically marked as stale. At OP: Please either close this issue or keep it active It will be closed in 7 days if no further activity occurs.

[KodeCR]

Waiting on merge of PR #3966

[popy2k14]

Just to let you know guys, the test build from above running without an issue over a month now.
Hope this gets merged anytime soon.

[github-actions]

As there has not been any response in 28 days, this issue will be closed. @ OP: If this issue is solved post what fixed it for you. If it is not solved, request to get this opened again.

[popy2k14]

Not solved!
Waiting on merge of PR #3966

[github-actions]

As there has not been any response in 21 days, this issue has been automatically marked as stale. At OP: Please either close this issue or keep it active It will be closed in 7 days if no further activity occurs.

[popy2k14]

Not solved yet

[github-actions]

As there has not been any response in 21 days, this issue has been automatically marked as stale. At OP: Please either close this issue or keep it active It will be closed in 7 days if no further activity occurs.