New advertiser: Avahi/D-Bus API #918

adriancable · 2021-12-30T01:21:01Z

♻️ Current situation

HAP-NodeJS currently doesn't work on some set-ups ('No Response'!) due to some router bugs:

Example: Arris G36 won't forward IPv4 multicast between wired and wireless interfaces. Bonjour-hap and Ciao advertisers won't advertise on IPv6. So you get no working advertisements if your HAP-NodeJS machine is connected via Ethernet instead of Wi-Fi.

HAP-NodeJS also currently doesn't work on other set-ups, for software-related reasons:

Example: since on Linux Avahi is usually enabled by default, and HAP-NodeJS usually runs after Avahi, it cannot listen for unicast requests on 5353 even with SO_REUSEADDR. There are at least some routers which (even though they shouldn't) recreate multicast packets directed to a multicast group as unicast packets which are sent to connected devices joined to that group. Example: Arris RAC2V1A. So you get no working advertisements.

HAP-NodeJS a very long time ago used node-mdns as an advertiser which uses the Avahi Bonjour compatibility API, but:

This required node-gyp to build as it used C/C++ modules, so it didn't work on some platforms
The Bonjour compatibility API wasn't ever fully implemented, and throws up all sorts of warnings when you try to use it
The Bonjour compatibility library was optional and some systems with otherwise complete Avahi don't have it

Also:

Avahi has circa 10000X+ the user base of bonjour-hap/ciao and so probably has 10000X+ the hours of testing and therefore broader compatibility in general

💡 Proposed solution

Added a 3rd advertiser (in addition to, not replacing, bonjour-hap and ciao) which talks to Avahi over the D-Bus API. Because it's pure JavaScript (no node-gyp) it works on any system with Avahi/D-Bus. If there is no Avahi/D-Bus on the system just don't use it, and continue with ciao or bonjour-hap 'según el gusto'.

Testing

Not done a lot. Seems to work on Linux! Tested IPv6 announcements, and they work.

coveralls · 2021-12-30T01:51:40Z

Pull Request Test Coverage Report for Build 1638863641

11 of 35 (31.43%) changed or added relevant lines in 2 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage decreased (-0.1%) to 48.948%

Changes Missing Coverage	Covered Lines	Changed/Added Lines	%
src/lib/Accessory.ts	2	4	50.0%
src/lib/Advertiser.ts	9	31	29.03%

Totals
Change from base Build 1632296833:	-0.1%
Covered Lines:	5189
Relevant Lines:	9400

💛 - Coveralls

Supereg

Thanks for you contribution. Especially for taking the time to illustrate the situation around mdns advertising with HAP-NodeJS. 🚀

I very much like the idea of getting back to interface with "standard" mdns implementations. This was and is a reoccurring pain point of HAP-NodeJS. Having a way to interface with avahi in a non-complex way is definitely a huge win 🎉

I know this is not part of the PR, but is there an easy way of checking if dbus and avahi is available on the currently running platform?

On a similar topic, I already though about interfacing with mDNSResponder on macOS. To my knowledge, the easiest approach would be to interface via the dns-sd CLI. Though, doing txt updates isn't really possible without completely readvertising, but that might be a sensible addition for the future as well.

src/lib/Advertiser.ts

src/lib/Accessory.ts

src/lib/Advertiser.ts

adriancable · 2021-12-30T21:40:37Z

I know this is not part of the PR, but is there an easy way of checking if dbus and avahi is available on the currently running platform?

Yes. If D-Bus is not available, then this line will throw an exception (tested on my Mac which can have D-Bus but does not have it installed):

this.bus = dbus.systemBus();

I can easily catch this, but not sure what the right way of dealing with it is. (in the sense of what would "fit in" best with the rest of HAP-NodeJS's operation)

If D-Bus is available but Avahi is not, then the avahiInvoke calls should throw an exception. (hard for me to test this)

Supereg · 2021-12-30T21:49:21Z

I know this is not part of the PR, but is there an easy way of checking if dbus and avahi is available on the currently running platform?

Yes. If D-Bus is not available, then this line will throw an exception (tested on my Mac which can have D-Bus but does not have it installed):

this.bus = dbus.systemBus();

I can easily catch this, but not sure what the right way of dealing with it is. (in the sense of what would "fit in" best with the rest of HAP-NodeJS's operation)

If D-Bus is available but Avahi is not, then the avahiInvoke calls should throw an exception. (hard for me to test this)

Okay good to know. I think it is fine if we keep the "auto configuration" separate from this PR for now.

If D-Bus is available but Avahi is not, then the avahiInvoke calls should throw an exception. (hard for me to test this)

I think the easiest way to test this, is to boot into a barebones docker container. Those typically ship without avahi.

adriancable · 2021-12-30T21:50:05Z

I think the easiest way to test this, is to boot into a barebones docker container. Those typically ship without avahi.

Even easier is probably just to systemctl stop avahi-daemon!

Supereg

🚀

Supereg · 2021-12-31T01:45:04Z

The PR is live with 0.10.0-beta.4

EDIT: live with homebridge v1.4.0-beta.0. The UI still needs to be updated to be able to configure the advertiser.

ebaauw · 2021-12-31T04:11:18Z

Thank you both! I’ll be testing this on my Raspberry Pi installations. Looking good so far (in Discovery app) on a test installation with five child bridges.

ebaauw · 2021-12-31T05:18:45Z

Hm, getting a fatal error on Raspberry Pi Zero:

Dec 31 06:14:40] [31/12/2021, 06:14:40] [HB Supervisor] Restarting Homebridge...
[Dec 31 06:14:40] [31/12/2021, 06:14:40] [HB Supervisor] Starting Homebridge with extra flags: -T -D
[Dec 31 06:14:40] [31/12/2021, 06:14:40] [HB Supervisor] Starting Homebridge with custom env: {"NODE_OPTIONS":"--trace-warnings"}
[Dec 31 06:14:40] [31/12/2021, 06:14:40] [HB Supervisor] Started Homebridge v1.4.0-beta.1 with PID: 32243
[Dec 31 06:14:56] #
[Dec 31 06:14:56] # Fatal error in , line 0
[Dec 31 06:14:56] # Liftoff bailout should not happen. Cause: Armv6 not supported
[Dec 31 06:14:56] #
[Dec 31 06:14:56] #
[Dec 31 06:14:56] #
[Dec 31 06:14:56] #FailureMessage Object: 0xb49faf50
[Dec 31 06:14:57] [31/12/2021, 06:14:57] [HB Supervisor] Homebridge Process Ended. Code: null, Signal: SIGTRAP

I also though I saw node-gyp being invoked while installing the Homebridge beta?

donavanbecker · 2021-12-31T05:37:27Z

The PR is live with 0.10.0-beta.4

EDIT: live with homebridge v1.4.0-beta.0. The UI still needs to be updated to be able to configure the advertiser.

@oznu

donavanbecker · 2021-12-31T05:39:55Z

getting the following on M1 Mac:

Error: connect ENOENT /var/run/dbus/system_bus_socket
    at PipeConnectWrap.afterConnect [as oncomplete] (node:net:1161:16)

ebaauw · 2021-12-31T05:43:09Z

I also though I saw node-gyp being invoked while installing the Homebridge beta?

Yes, looking at htop, while installing es-abstract after installing hap-nodejs. I don’t think npm saved a log file, as the install (eventually) succeeds. node-gyp is not invoked when installing the latest version of Homebridge.

Because it's pure JavaScript (no node-gyp) it works on any system with Avahi/D-Bus.

Apparently not?

ebaauw · 2021-12-31T05:46:33Z

getting the following on M1 Mac

That’s to be expected. No dbus nor Avahi on Mac. I hope only when setting advertiser to avahi?

donavanbecker · 2021-12-31T05:51:20Z

getting the following on M1 Mac

That’s to be expected. No dbus nor Avahi on Mac. I hope only when setting advertiser to avahi?

yes.

adriancable · 2021-12-31T06:40:04Z

Because it's pure JavaScript (no node-gyp) it works on any system with Avahi/D-Bus.

Apparently not?

dbus-native uses abstract-socket as an optionalDependency, which does use node-gyp. But as far as I can tell this is just to improve performance. If node-gyp isn't found or fails, everything still works. Alternatively, you can skip installing it using npm i --no-optional, node-gyp isn't called, and everything also still works.

oznu · 2021-12-31T06:46:02Z

Looks good. I'm no where near a computer right not to actually test however.

Before this goes into a Homebridge releases I'd like to see auto-fallback to a valid advertiser if Avahi is not available on the system (assuming this is not already the case?), and the default to remain as Bonjour for existing installs for at least for the initial release, (I can make it so new installs have the advertiser set to Avahi if available).

adriancable · 2021-12-31T06:46:32Z

[Dec 31 06:14:56] # Liftoff bailout should not happen. Cause: Armv6 not supported

Are you using Node 17? ARMv6 is no longer fully supported. Please go back to Node 16 in this case.

ebaauw · 2021-12-31T07:00:11Z

Node v16.13.1. And sudo npm -g i homebridge@beta --no-optional still calls node-gyp.

adriancable · 2021-12-31T07:03:51Z

Node v16.13.1. And sudo npm -g i homebridge@beta --no-optional still calls node-gyp.

Try node --no-expose-wasm

Regarding calling node-gyp, which package is calling it? If it's es-abstract (you mentioned this earlier), that's a dependency of deep-equal which is a dependency of bonjour-hap, so not related to this PR.

ebaauw · 2021-12-31T07:08:34Z

Try node --no-expose-wasm

Running Homebridge or running npm?

Regarding calling node-gyp, which package is calling it? If it's es-abstract (you mentioned this earlier), that's a dependency of deep-equal which is a dependency of bonjour-hap, so not related to this PR.

Not sure. How can I force npm to keep a log on success? It’s a dependency that’s introduced in the new beta, as node-gyp isn’t invoked when installing the latest version.

Edit found the log, indeed it’s abstract-socket:

631 info run abstract-socket@2.1.1 install node_modules/homebridge/node_modules/abstract-socket node-gyp rebuild
632 info run abstract-socket@2.1.1 install { code: 0, signal: null }
633 timing build:run:install:node_modules/homebridge/node_modules/abstract-socket Completed in 210231ms

ebaauw · 2021-12-31T07:46:25Z

On a positive note, the avahi/dbus advertiser seems to work fine on my Pi 4B servers, running 32-bit buster or bullseye. I have the impression that Home on my iPad spends a little less time on updating the tiles than when using ciao.

Supereg · 2021-12-31T12:18:16Z

Looks good. I'm no where near a computer right not to actually test however.

Before this goes into a Homebridge releases I'd like to see auto-fallback to a valid advertiser if Avahi is not available on the system (assuming this is not already the case?), and the default to remain as Bonjour for existing installs for at least for the initial release, (I can make it so new installs have the advertiser set to Avahi if available).

@oznu as of now bonjour-hap is still the default and avahi has to be manually configured. I would give it a shot to implement the auto-default in hap-nodes next week. Not sure if this needs modifications then in the UI (apart from adding a new case) to properly display the default in the dropdown.

adriancable · 2021-12-31T16:44:10Z

Try node --no-expose-wasm

Running Homebridge or running npm?

Running whatever it was that gave the ARMv6 error.

adriancable · 2021-12-31T17:34:09Z

Edit found the log, indeed it’s abstract-socket:

From some Googling, it appears that npm --no-optional is ignored if the package-lock.json was made without --no-optional. So this is probably why you're still seeing node-gyp running. Can you try deleting package-lock.json and then running npm i --no-optional again?

Also npm i will still succeed even if an optionalDependency can't install because of lack of node-gyp or similar. So I still think the way things are right now is fine ... npm i will still work to install HAP-NodeJS even if abstract-socket isn't supported on the platform. And, if it's a platform where abstract-socket isn't supported, probably Avahi/D-Bus isn't supported either, so in practice no issue.

ebaauw · 2021-12-31T17:53:42Z

The issue is that the Homebridge beta crashes on a Pi Zero, before it’s even read config.json to find out which advertiser to use. I only use the Pi Zero for testing, so I couldn’t care less, but there are people out there actually running Homebridge on an arm6. This needs to be fixed before publishing a released version with the Avahi/D-Bus advertiser, or support for the arm6 needs to be dropped officially - again, I don’t care which; I’m just reporting what I encounter. This is a standard installation as per the Wiki, and and I installed the beta through the UI. I’m willing and able to try stuff on the command line, in order to analyse this issue, but this is a real issue, in practice.

adriancable · 2021-12-31T17:57:33Z

The issue is that the Homebridge beta crashes on a Pi Zero, before it’s even read config.json to find out which advertiser to use. I only use the Pi Zero for testing, so I couldn’t care less, but there are people out there actually running Homebridge on an arm6. This needs to be fixed before publishing a released version with the Avahi/D-Bus advertiser, or support for the arm6 needs to be dropped officially - again, I don’t care which; I’m just reporting what I encounter. This is a standard installation as per the Wiki, and and I installed the beta through the UI. I’m willing and able to try stuff on the command line, in order to analyse this issue, but this is a real issue, in practice.

I understand, but it's not clear how you came to the conclusion that the ARMv6 crash has something to do with the addition of the Avahi/D-Bus advertiser, and not some other dependency that's been updated in the HAP-NodeJS beta. In fact since your issue seems to come from using Homebridge, it isn't even clear that it's related to HAP-NodeJS at all. Can you share what info you have on this? Thanks!

bwp91 · 2021-12-31T17:59:18Z

support for the arm6 needs to be dropped officially

i don’t think this could happen until at least april 2023 when homebridge officially drops support for node 14. unless i am getting my versions of node that support arm6 mixed up!

Supereg · 2021-12-31T18:08:58Z

@ebaauw @adriancable Would be great if you could track it in a new, separate issue (tagged with beta).

ebaauw · 2021-12-31T18:09:36Z

but it's not clear how you came to the conclusion that the ARMv6 crash has something to do with the addition of the Avahi/D-Bus advertiser

I haven't. I'm just reacting here as Andi mentioned the Homebridge beta here. Given the time in between v1.3.9 and the beta, I guess there wouldn't be many other changes, but assumptions ...

Can you share what info you have on this?

I'm afaid, I already have. Frankly, I have no idea who issues the messages, not who causes the SIGTRAP. I'm not even sure if it's related to the invocation of node-gyp. This is just another discrepancy I noticed. Attached the npm log when installing the beta.
2021-12-31T07_15_18_070Z-debug-0.log

ebaauw · 2021-12-31T18:11:40Z

Would be great if you could track it in a new, separate issue (tagged with beta).

In HAP-NodeJS or in Homebridge?

Supereg · 2021-12-31T18:15:32Z

Would be great if you could track it in a new, separate issue (tagged with beta).

In HAP-NodeJS or in Homebridge?

HAP-NodeJS. Thanks!

ebaauw · 2021-12-31T18:41:41Z

Would be great if you could track it in a new, separate issue (tagged with beta).

@Supereg : #919

Try node --no-expose-wasm

@adriancable : That does the trick, see the issue. Does that mean the crash is related to dbus-native?

Again, on a positive note: I'm very happy with the PR, and it seems to run perfectly well on the Pi 4B. As I mentioned above, I have a feeling Home is spending less time Updating the tiles.

adriancable · 2021-12-31T19:33:48Z

Try node --no-expose-wasm

@adriancable : That does the trick, see the issue. Does that mean the crash is related to dbus-native?

No, unfortunately, I don't have any ARMv6 platform to test. But Googling around, I read that Node is deprecating ARMv6 from 16.1 onwards, and at least for now this command line switch seems to disable the bits which break.

adriancable · 2021-12-31T19:35:18Z

Again, on a positive note: I'm very happy with the PR, and it seems to run perfectly well on the Pi 4B. As I mentioned above, I have a feeling Home is spending less time Updating the tiles.

I also (independently) noticed the exact same thing. I am fairly sure the effect is real and not placebo. I wonder why this would be the case?

### Added - Support for new advertiser [AVAHI](homebridge/HAP-NodeJS#918) - Support for new advertiser [RESOLVED](homebridge/HAP-NodeJS#965) - Added `msg.hap.reachable` parameter to get device reachable state (related to NO_RESPONSE) ### Fixed - Accessory could not be recovered from NO_RESPONSE using single Characteristic - Make unsupported Characteristic error more descriptive [#456](#456) - FFmpeg No such file or directory [#495](#495) - allChars: properties have spaces in names [#496](#496) - Wait for host to return from unpublish/destroy before exiting, set published flag on destroy - Security system with characteristics, bad behaviour [#388](#388) ### Changed - Updated hap-nodejs to [0.9.7](https://github.com/homebridge/HAP-NodeJS/releases/tag/v0.9.7) (bug fixes) - Updated hap-nodejs to [0.9.8](https://github.com/homebridge/HAP-NodeJS/releases/tag/v0.9.8) (bug fixes) - Updated hap-nodejs to [0.10.0](https://github.com/homebridge/HAP-NodeJS/releases/tag/v0.10.0) (features) - Updated hap-nodejs to [0.10.1](https://github.com/homebridge/HAP-NodeJS/releases/tag/v0.10.1) (changes) - Updated hap-nodejs to [0.10.2](https://github.com/homebridge/HAP-NodeJS/releases/tag/v0.10.2) (bug fixes) - Updated hap-nodejs to [0.10.3](https://github.com/homebridge/HAP-NodeJS/releases/tag/v0.10.3) (bug fixes) - Updated hap-nodejs to [0.10.4](https://github.com/homebridge/HAP-NodeJS/releases/tag/v0.10.4) (bug fixes) - Updated hap-nodejs to [0.11.0](https://github.com/homebridge/HAP-NodeJS/releases/tag/v0.11.0) (features and bug fixes)

adriancable added 2 commits December 29, 2021 16:59

Add Avahi/D-Bus advertiser support

3b9738b

Add Avahi/D-Bus advertiser support (2)

a31d7a3

Supereg reviewed Dec 30, 2021

View reviewed changes

src/lib/Advertiser.ts Outdated Show resolved Hide resolved

src/lib/Accessory.ts Outdated Show resolved Hide resolved

src/lib/Advertiser.ts Show resolved Hide resolved

src/lib/Advertiser.ts Outdated Show resolved Hide resolved

Code clean-up per @Supereg's feedback.

278fa65

Supereg changed the base branch from master to beta-0.10.0 December 31, 2021 01:31

Supereg approved these changes Dec 31, 2021

View reviewed changes

Supereg merged commit ede2c66 into homebridge:beta-0.10.0 Dec 31, 2021

This was referenced Jan 3, 2022

Beta v1.4.0-beta.1 crashes on Pi Zero #919

Closed

Automatically use Avahi/DBus advertiser if available on platform #922

Merged

Supereg added the feature label Jan 16, 2022

This was referenced Jan 17, 2022

Rely on hap-nodejs to select a default for the advertiser homebridge/homebridge#3062

Merged

Use bonjour advertiser as default again, warn when Avahi is used on an unsupported platform. #927

Merged

A1bi mentioned this pull request Mar 7, 2022

Service announcements are not sent over IPv6 homebridge/ciao#10

Open

Shaquu mentioned this pull request Apr 12, 2022

Release Candidate 1.5.0 NRCHKB/node-red-contrib-homekit-bridged#481

Merged

koush pushed a commit to koush/HAP-NodeJS that referenced this pull request Jun 28, 2022

New advertiser: Avahi/D-Bus API (homebridge#918)

88b73e8

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

New advertiser: Avahi/D-Bus API #918

New advertiser: Avahi/D-Bus API #918

adriancable commented Dec 30, 2021

coveralls commented Dec 30, 2021 •

edited

Loading

Supereg left a comment

adriancable commented Dec 30, 2021

Supereg commented Dec 30, 2021

adriancable commented Dec 30, 2021

Supereg left a comment

Supereg commented Dec 31, 2021 •

edited

Loading

ebaauw commented Dec 31, 2021

ebaauw commented Dec 31, 2021

donavanbecker commented Dec 31, 2021

donavanbecker commented Dec 31, 2021

ebaauw commented Dec 31, 2021 •

edited

Loading

ebaauw commented Dec 31, 2021

donavanbecker commented Dec 31, 2021

adriancable commented Dec 31, 2021

oznu commented Dec 31, 2021

adriancable commented Dec 31, 2021

ebaauw commented Dec 31, 2021

adriancable commented Dec 31, 2021

ebaauw commented Dec 31, 2021 •

edited

Loading

ebaauw commented Dec 31, 2021

Supereg commented Dec 31, 2021

adriancable commented Dec 31, 2021

adriancable commented Dec 31, 2021

ebaauw commented Dec 31, 2021

adriancable commented Dec 31, 2021 •

edited

Loading

bwp91 commented Dec 31, 2021

Supereg commented Dec 31, 2021

ebaauw commented Dec 31, 2021 •

edited

Loading

ebaauw commented Dec 31, 2021

Supereg commented Dec 31, 2021

ebaauw commented Dec 31, 2021

adriancable commented Dec 31, 2021

adriancable commented Dec 31, 2021

New advertiser: Avahi/D-Bus API #918

New advertiser: Avahi/D-Bus API #918

Conversation

adriancable commented Dec 30, 2021

♻️ Current situation

💡 Proposed solution

Testing

coveralls commented Dec 30, 2021 • edited Loading

Pull Request Test Coverage Report for Build 1638863641

💛 - Coveralls

Supereg left a comment

Choose a reason for hiding this comment

adriancable commented Dec 30, 2021

Supereg commented Dec 30, 2021

adriancable commented Dec 30, 2021

Supereg left a comment

Choose a reason for hiding this comment

Supereg commented Dec 31, 2021 • edited Loading

ebaauw commented Dec 31, 2021

ebaauw commented Dec 31, 2021

donavanbecker commented Dec 31, 2021

donavanbecker commented Dec 31, 2021

ebaauw commented Dec 31, 2021 • edited Loading

ebaauw commented Dec 31, 2021

donavanbecker commented Dec 31, 2021

adriancable commented Dec 31, 2021

oznu commented Dec 31, 2021

adriancable commented Dec 31, 2021

ebaauw commented Dec 31, 2021

adriancable commented Dec 31, 2021

ebaauw commented Dec 31, 2021 • edited Loading

ebaauw commented Dec 31, 2021

Supereg commented Dec 31, 2021

adriancable commented Dec 31, 2021

adriancable commented Dec 31, 2021

ebaauw commented Dec 31, 2021

adriancable commented Dec 31, 2021 • edited Loading

bwp91 commented Dec 31, 2021

Supereg commented Dec 31, 2021

ebaauw commented Dec 31, 2021 • edited Loading

ebaauw commented Dec 31, 2021

Supereg commented Dec 31, 2021

ebaauw commented Dec 31, 2021

adriancable commented Dec 31, 2021

adriancable commented Dec 31, 2021

coveralls commented Dec 30, 2021 •

edited

Loading

Supereg commented Dec 31, 2021 •

edited

Loading

ebaauw commented Dec 31, 2021 •

edited

Loading

ebaauw commented Dec 31, 2021 •

edited

Loading

adriancable commented Dec 31, 2021 •

edited

Loading

ebaauw commented Dec 31, 2021 •

edited

Loading