docker setup, Raspi 4B, Ubuntu server, "device state timeout ignored in state" [2|3]

[AnotherDaniel]

Describe the bug

I'm setting up a fresh deconz setup on a Raspi 4B, running Ubuntu Server, using marthoc/deconz:stable. Below I'll add the docker-compose file I'm using, as well as bespoke configuration.

The issue I'm seeing is that I cannot get deconz to connect to the RaspBee device (properly). By trying out suggestions and configuration hints from all the threads I could find, I can boil down the behavior I can get into these two scenarios:
Startup of application, log outputs etc, ending with endless loops of either

1. "device state timeout ignored in state 2", "try to reconnect to network", or
2. "device state timeout ignored in state 3", "invalid current channel 0 (TODO)"

I have tried to nail down when I'm getting which, without success so far - it doesn't make a difference whether I start with a clean deconz config or an existing one.

I did not try to update the device firmware in any way - but this is the first attempt to get a setup going with Ubuntu and docker. Previously I ran deconz as a system service on Raspian on the same Raspi hardware, which worked fine. But there is no native deconz package available for Ubuntu on arm, plus I'd really like to get deconz into a docker environment like every other workload on that machine.

Steps to reproduce the behavior

Run docker-compose up in the directory with following docker-compose.yml

version: "3"
services:
deconz:
image: marthoc/deconz:stable
container_name: deconz
network_mode: host
restart: always
volumes:
- /opt/deconz:/root/.local/share/dresden-elektronik/deCONZ
devices:
- /dev/ttyAMA0
- /dev/ttyS0
environment:
- TZ=Europe/Berlin
- DECONZ_DEVICE=/dev/ttyAMA0
- DECONZ_DEVICE=/dev/ttyS0
- DECONZ_WEB_PORT=8090
- DECONZ_WS_PORT=8443
- DECONZ_VNC_PORT=5900
- DECONZ_NOVNC_PORT=0
- DEBUG_INFO=1
- DEBUG_APS=0
- DEBUG_ZCL=0
- DEBUG_ZDP=0
- DEBUG_OTAU=0
cap_add:
- ALL
privileged: true

Expected behavior

I'd like deconz to connect to my RaspBee device properly

Screenshots

n/a

Environment

• Host system: Raspberry Pi 4B 4Gb
• Running method: Ubuntu Server, Marthoc Docker container
• Firmware version: (0x26390500)
• deCONZ version: (2.10.04)
• Device: RaspBee I ("RaspBee Premium", bought November 2019)
• Is there any other USB or serial devices connected to the host system? none
• wiringpi package is installed and in latest version

deCONZ Logs

Log output scenario 1:

deconz | [marthoc/deconz] Starting deCONZ...
deconz | [marthoc/deconz] Current deCONZ version: 2.10.04
deconz | [marthoc/deconz] Web UI port: 8090
deconz | [marthoc/deconz] Websockets port: 8443
deconz | [marthoc/deconz] VNC Disabled
deconz | QStandardPaths: XDG_RUNTIME_DIR not set, defaulting to '/tmp/runtime-root'
deconz | libpng warning: iCCP: known incorrect sRGB profile
deconz | 19:03:33:064 HTTP Server listen on address 0.0.0.0, port: 8090, root: /usr/share/deCONZ/webapp/
deconz | 19:03:33:094 CTRL. 3.27.219:03:33:613 dev /dev/ttyAMA0
deconz | 19:03:33:613 dev /dev/ttyS0
deconz | 19:03:33:613 COM: /dev/ttyS0 / serialno: , RaspBee
deconz | 19:03:33:614 ZCLDB init file /root/.local/share/dresden-elektronik/deCONZ/zcldb.txt
deconz | 19:03:33:936 parent process /bin/sh
deconz | 19:03:33:936 gw run mode: docker
deconz | 19:03:33:936 GW sd-card image version file does not exist: /root/.local/share/dresden-elektronik/deCONZ/gw-version
deconz | 19:03:33:937 sd-card cid: 035344534e36344780ffffffff013a00
deconz | 19:03:33:959 DB sqlite version 3.27.2
deconz | 19:03:33:961 DB PRAGMA page_count: 0
deconz | 19:03:33:961 DB PRAGMA page_size: 4096
deconz | 19:03:33:961 DB PRAGMA freelist_count: 0
deconz | 19:03:33:961 DB file size 0 bytes, free pages 0
deconz | 19:03:33:962 DB PRAGMA user_version: 0
deconz | 19:03:33:962 DB upgrade to user_version 1
deconz | 19:03:34:234 DB write sqlite user_version 1
deconz | 19:03:34:245 DB PRAGMA user_version: 1
deconz | 19:03:34:246 DB upgrade to user_version 2
deconz | 19:03:34:277 DB write sqlite user_version 2
deconz | 19:03:34:288 DB PRAGMA user_version: 2
deconz | 19:03:34:288 DB upgrade to user_version 6
deconz | 19:03:34:328 DB write sqlite user_version 6
deconz | 19:03:34:337 DB PRAGMA user_version: 6
deconz | 19:03:34:337 DB upgrade to user_version 7
deconz | 19:03:34:369 DB write sqlite user_version 7
deconz | 19:03:34:380 DB PRAGMA user_version: 7
deconz | 19:03:34:380 DB cleanup
deconz | 19:03:34:381 DB create temporary views
deconz | 19:03:34:394 started websocket server at port 8443
deconz | 19:03:34:397 create default username and password
deconz | 19:03:34:408 [INFO] - Found file containing button maps. Parsing data...
deconz | 19:03:34:415 [WARNING] - Button map 'sunricherCCTMap' in JSON file has no assigned ModelIDs. Skip loading button map...
deconz | 19:03:34:424 [INFO] - Button maps loaded.
deconz | 19:03:34:426 dlg action: Read binding table
deconz | 19:03:34:426 found node plugin: libde_rest_plugin.so - REST API Plugin
deconz | 19:03:34:431 found node plugin: libde_signal_plugin.so - Signal Monitor Plugin
deconz | 19:03:34:477 found node plugin: libstd_otau_plugin.so - STD OTAU Plugin
deconz | 19:03:34:884 dev /dev/ttyAMA0
deconz | 19:03:34:884 dev /dev/ttyS0
deconz | 19:03:34:884 COM: /dev/ttyS0 / serialno: , RaspBee
deconz | This plugin does not support propagateSizeHints()
deconz | This plugin does not support propagateSizeHints()
deconz | 19:03:36:294 dev /dev/ttyAMA0
deconz | 19:03:36:294 dev /dev/ttyS0
deconz | 19:03:36:294 COM: /dev/ttyS0 / serialno: , RaspBee
deconz | 19:03:36:317 COM check bootloader
deconz | 19:03:38:330 COM timout query bootloader, assume application
deconz | 19:03:39:695 dev /dev/ttyAMA0
deconz | 19:03:39:695 dev /dev/ttyS0
deconz | 19:03:39:894 Announced to internet https://phoscon.de/discover
deconz | 19:03:39:895 discovery server date: Fri, 16 Apr 2021 17:03:39 GMT
deconz | 19:03:39:896 local time seems to be ok
deconz | 19:03:39:907 device state timeout ignored in state 2
deconz | 19:03:40:475 Skip idle timer callback, too early: elapsed 779 msec
deconz | 19:03:42:975 device state timeout ignored in state 2
deconz | 19:03:44:735 New websocket 192.168.0.36:59509 (state: 3)
deconz | 19:03:45:975 device state timeout ignored in state 2
deconz | 19:03:48:975 device state timeout ignored in state 2
deconz | 19:03:50:386 dev /dev/ttyAMA0
deconz | 19:03:50:386 dev /dev/ttyS0
deconz | 19:03:51:975 device state timeout ignored in state 2
deconz | 19:03:54:975 device state timeout ignored in state 2
deconz | 19:03:55:475 start reconnect to network
deconz | 19:03:57:975 device state timeout ignored in state 2

Log output scenario 2:

deconz | [marthoc/deconz] Starting deCONZ...
deconz | [marthoc/deconz] Current deCONZ version: 2.10.04
deconz | [marthoc/deconz] Web UI port: 8090
deconz | [marthoc/deconz] Websockets port: 8443
deconz | [marthoc/deconz] VNC Disabled
deconz | QStandardPaths: XDG_RUNTIME_DIR not set, defaulting to '/tmp/runtime-root'
deconz | libpng warning: iCCP: known incorrect sRGB profile
deconz | 19:16:10:007 HTTP Server listen on address 0.0.0.0, port: 8090, root: /usr/share/deCONZ/webapp/
deconz | 19:16:10:021 CTRL. 3.27.219:16:10:503 dev /dev/ttyAMA0
deconz | 19:16:10:503 dev /dev/ttyS0
deconz | 19:16:10:503 COM: /dev/ttyS0 / serialno: , RaspBee
deconz | 19:16:10:504 ZCLDB init file /root/.local/share/dresden-elektronik/deCONZ/zcldb.txt
deconz | 19:16:10:668 parent process /bin/sh
deconz | 19:16:10:669 gw run mode: docker
deconz | 19:16:10:669 GW sd-card image version file does not exist: /root/.local/share/dresden-elektronik/deCONZ/gw-version
deconz | 19:16:10:669 sd-card cid: 035344534e36344780ffffffff013a00
deconz | 19:16:10:670 DB sqlite version 3.27.2
deconz | 19:16:10:671 DB PRAGMA page_count: 0
deconz | 19:16:10:671 DB PRAGMA page_size: 4096
deconz | 19:16:10:671 DB PRAGMA freelist_count: 0
deconz | 19:16:10:671 DB file size 0 bytes, free pages 0
deconz | 19:16:10:671 DB PRAGMA user_version: 0
deconz | 19:16:10:671 DB upgrade to user_version 1
deconz | 19:16:10:928 DB write sqlite user_version 1
deconz | 19:16:10:947 DB PRAGMA user_version: 1
deconz | 19:16:10:948 DB upgrade to user_version 2
deconz | 19:16:10:981 DB write sqlite user_version 2
deconz | 19:16:10:992 DB PRAGMA user_version: 2
deconz | 19:16:10:992 DB upgrade to user_version 6
deconz | 19:16:11:035 DB write sqlite user_version 6
deconz | 19:16:11:045 DB PRAGMA user_version: 6
deconz | 19:16:11:046 DB upgrade to user_version 7
deconz | 19:16:11:078 DB write sqlite user_version 7
deconz | 19:16:11:088 DB PRAGMA user_version: 7
deconz | 19:16:11:088 DB cleanup
deconz | 19:16:11:089 DB create temporary views
deconz | 19:16:11:095 started websocket server at port 8443
deconz | 19:16:11:097 create default username and password
deconz | 19:16:11:100 [INFO] - Found file containing button maps. Parsing data...
deconz | 19:16:11:102 [WARNING] - Button map 'sunricherCCTMap' in JSON file has no assigned ModelIDs. Skip loading button map...
deconz | 19:16:11:109 [INFO] - Button maps loaded.
deconz | 19:16:11:111 dlg action: Read binding table
deconz | 19:16:11:111 found node plugin: libde_rest_plugin.so - REST API Plugin
deconz | 19:16:11:114 found node plugin: libde_signal_plugin.so - Signal Monitor Plugin
deconz | 19:16:11:145 found node plugin: libstd_otau_plugin.so - STD OTAU Plugin
deconz | 19:16:11:552 dev /dev/ttyAMA0
deconz | 19:16:11:552 dev /dev/ttyS0
deconz | 19:16:11:552 COM: /dev/ttyS0 / serialno: , RaspBee
deconz | This plugin does not support propagateSizeHints()
deconz | This plugin does not support propagateSizeHints()
deconz | 19:16:12:962 dev /dev/ttyAMA0
deconz | 19:16:12:962 dev /dev/ttyS0
deconz | 19:16:12:962 COM: /dev/ttyS0 / serialno: , RaspBee
deconz | 19:16:12:988 COM check bootloader
deconz | 19:16:15:002 COM timout query bootloader, assume application
deconz | This plugin does not support propagateSizeHints()
deconz | 19:16:15:050 Device firmware version 0x26390500 RaspBee
deconz | 19:16:15:071 unlocked max nodes: 200
deconz | 19:16:16:527 dev /dev/ttyAMA0
deconz | 19:16:16:527 dev /dev/ttyS0
deconz | 19:16:16:529 GW update firmware found: /usr/share/deCONZ/firmware/deCONZ_Rpi_0x26390500.bin.GCF
deconz | 19:16:16:529 GW firmware version: 0x26390500
deconz | 19:16:16:529 GW firmware version is up to date: 0x26390500
deconz | 19:16:16:722 Announced to internet https://phoscon.de/discover
deconz | 19:16:16:723 discovery server date: Fri, 16 Apr 2021 17:16:16 GMT
deconz | 19:16:16:723 local time seems to be ok
deconz | 19:16:19:577 device state timeout ignored in state 3
deconz | 19:16:21:076 invalid current channel 0 (TODO)
deconz | 19:16:22:576 device state timeout ignored in state 3
deconz | 19:16:25:577 device state timeout ignored in state 3
deconz | 19:16:28:576 device state timeout ignored in state 3
deconz | 19:16:31:076 invalid current channel 0 (TODO)
deconz | 19:16:31:576 device state timeout ignored in state 3

Additional context

I have tried all the configuration tricks and tipps I could find in other issues and web search hits, and I've tried variations of the following configuration entries in /boot/firmware/config.txt:

dtoverlay=disable_bt
dtoverlay=disable_wifi
dtoverlay=miniuart-bt
#dtoverlay=pi3-miniuart-bt

Alternating the last to config lines, with or without disabling bt and wifi... no joy. I've also tried the ttyAMA0 and ttyS0 devices individually of course, no joy.
The user I'm running docker-compose/docker as is part of the tty and dialout groups, to provide access to the /dev nodes. Currently (with the above config.txt settings), my /dev nodes look like this:

crw-rw---- 1 root dialout 4, 64 Mar 10 22:03 /dev/ttyS0
crw--w---- 1 root tty 204, 64 Apr 16 19:24 /dev/ttyAMA0

(ttyAMA0 is used by the agetty process, with different config.txt settings it's the other way round and S0 is used by agetty)

I really like my RaspBee, and am getting a bit desparate here with the home automation blocked-broken. Any help I can get would be much appreciated!

[Mimiix]

@phdelodder

[phdelodder]

I notice in your docker compose file that you have 2 devices, is there a reason for that? By passing the environment variable DECONZ_DEVICE twice it's most likely you use the last entry. So clean that up and have a look at:

#3974 (comment) perhapse update your firmware to the latest version?

[phdelodder]

you should check https://www.raspberrypi.org/documentation/configuration/uart.md as well for the config.txt

Did you check: https://phoscon.de/en/raspbee2/install#docker ?

in my /boot/config.txt I have these entries:

enable_uart=1
dtoverlay=disable-bt

NOTE: I'm using an RPI 3B!

[AnotherDaniel]

Guys you rock! It wasn't any the specific hints you gave above, but a follow-up from one of these... on Ubuntu there's no raspi-config to disable the serial console. However it's still activated by the kernel cmdline - as I just figured out. Disabling that gets me deconz running properly on ttyS0!

Two more things:

1. Would deconz-RaspBee profit from the better specs of PL011, vs the miniUART? (i.e. I'll try to get it configured to run on ttyAMA0 if that should be the case)
2. I would do a little paragraph-writeup of the steps to enable deconz on an Ubuntu Server [arm64] system with what I learned, to go into the deconz docker installation docs - if you want me to and would put it in...

[phdelodder]

Documentation is always welcome!

[AnotherDaniel]

Allright. It became more than a paragraph - feel free to put this whereever you think it makes sense, in whichever form:

---

Setup dockerized deconz in Ubuntu, connecting to a RaspBee board

Ubuntu provides a server (headless) version that also is available for arm devices - this is a great option if you want a 64bit OS on your Raspberry 3, and of course it works equally well on 4th gen Raspberry boards.

There aren't any arm-architecture deconz packages available for Ubuntu - but this is where Docker steps in, with the images provided by marthoc!

If you want to get Docker deconz running on Ubuntu, you will not have the raspi-config tool available to turn off the kernel serial console on the Raspis's GPIO ports, like described in marthoc's Readme. But we still need to 'free up' these GPIO ports for use by the RaspBee board.

To prepare your Ubuntu system for running dockerized deconz, you can follow these steps:

Turn off linux kernel serial console

Edit /boot/firmware/cmdline.txt and remove these two parameters console=serial0,115200 console=tty1. My cmdline.txt looks like this afterwards:

dwc_otg.lpm_enable=0 root=LABEL=writable rootfstype=ext4 elevator=deadline rootwait fixrtc quiet splash

Tell the kernel device tree which UART to use

You can either map the Raspi's mini UART device for communication with the RaspBee, or it's first PL011 device - this page is a good resource to learn about the Raspberry's serial interfaces. Configuration happens in /boot/firmware/config.txt

To drive the RaspBee board via mini UART, add this line to config.txt:

dtoverlay=disable_bt

After a reboot, the RaspBee will be available on /dev/ttyS0

To make RaspBee available via PL011, you should add this to config.txt:

dtoverlay=miniuart-bt

After a reboot, the RaspBee will be available on /dev/ttyAMA0

You will find information about the differences between these two UART options on the raspberrypi documentation page referenced above.

Once you have decided on your setup, put in these configuration changes and rebooted your Raspberry, you should be good to go! Just remember to

• install the wiringpi package,
• add your user account to the dialout group,
• set your docker cmdline (or docker-compose file) to use /dev/ttyS0 or /dev/ttyAMA0, depending on which option you chose above.

These steps are described in marthoc's documentation, as referenced above.

[SwoopX]

Hey, thanks for crafting some documentation which probably will help quite some people out there. However, I feel the passage about configuring the dtoverlay is "a bit dangerous", so please allow me to explain why I think so. No matter how often I read the referenced raspberry documentation in the past, I still kinda shocked how complicated they make it.

First of all, the RaspBee (I or II) must be run on the PL011, as only this is fullfeatured while the mini UART is stripped in capabilities. That being said, it of course also matters which type of RPi you have, as PL011 is not automatically the primary UART as the documentation explains. To add more complexity to this, it also matters if you desire/require BT to run or not. If yes, the required configuration might vary as well.

However, there seems to be a savior in form of the overlays. Documentation states:

Primary UART         | Default state of enable_uart flag
---------------------|----------------------------
mini UART            | 0
first PL011 (UART0)  | 1

As I recall, upon deCONZ installation, enable_uart is always set to 1 in config.txt, which should ensure the primary UART is PL011. However, not sure if that could interfere in any way with the below.

"disable-bt disables the Bluetooth device and makes the first PL011 (UART0) the primary UART. You must also disable the system service that initialises the modem, so it does not connect to the UART, using sudo systemctl disable hciuart."

This is what we want if we don't want BT running in parallel.

"miniuart-bt switches the Bluetooth function to use the mini UART, and makes the first PL011 (UART0) the primary UART. Note that this may reduce the maximum usable baud rate (see mini UART limitations below). You must also set the VPU core clock to a fixed frequency using either force_turbo=1 or core_freq=250."

This is what we want if we want BT running in parallel.

So TLDR, as you might have recognized, it is more the wording in your guide that might need to be amended. For the BT to remain active, it depends on the RPi which additional parameter needs to be added to the configuration.

Please note that the above is based on my current understanding of this topic and must not necessarily be correct. But as my RPi is running without any issues while BT is enabled, it cannot be that wrong 😉

[phdelodder]

I've copied to a wiki page: https://github.com/dresden-elektronik/deconz-rest-plugin/wiki/Setup-dockerized-deconz-in-Ubuntu,-connecting-to-a-RaspBee-board

Feel free to further improve it.

@AnotherDaniel please close the issue if it's solved.

[AnotherDaniel]

Solved, see here for resolution: https://github.com/dresden-elektronik/deconz-rest-plugin/wiki/Setup-dockerized-deconz-in-Ubuntu,-connecting-to-a-RaspBee-board