[REQUEST] Zeroconf DNS TXT records parameters in ZigStarGW-FW to support Home Assistant ZHA network discovery #7 #3

Hedda · 2021-10-18T13:27:36Z

@xyzroe Please consider adding Zeroconf DNS TXT Reconds parameters in your firmware for automatic ZHA network discovery, see:

https://community.home-assistant.io/t/zha-automatic-network-discovery-of-zigbee-coordinator-bridges-gateways-ethernet-wifi-network-adapters-that-support-zeroconf/293300

This was previously requested for the original ZigStar gateway firmware here:

mercenaruss/zigstar_gateways#2

As you can see there, Home Assistant needs not only a service to be presented via Zeroconf but also parameters via Zeroconf DNS TXT records which include radio_type, baud_rate, and data_flow_control, which ZHA config flow use during initial configuration. Once your firmware provide those via Zeroconf then patches can be submitted to Home Assistant core which both makes its Zeroconf whitelist it for discovery as well as pass it along to the ZHA domain which uses the DNS TXT record parameters and puts them into the config flow to ZHA. What this achieved is that the user get a pop-up notification that a new device has been discovered and if they use they can go in to get your adapter automatically configured which makes the whole process very user-friendly to new users.

Since you based your new firmware on ZiGate-Ethernet you might also be interested in this same request posted to fairecasoimeme:

fairecasoimeme/ZiGate-Ethernet#7

PS: Please note I'm just a big fan Zigbee based hardware and of the ZHA integration in Home Assistant (as well as the zigpy project which it depends on) but do not actually own a ZigStar. My only wish with this request is to make it easier for users to get started.

xyzroe · 2021-10-18T13:39:29Z

@Hedda greetings to you!

My firmware already contains Zeroconf DNS TXT records since release 0.5.6

You can see the recordings themselves here

If you can submit a patch to Home Assistant core which will makes my Zeroconf DNS TXT Records in whitelist it for discovery I will be grateful!

xyzroe · 2021-10-18T13:41:49Z

@Hedda by the way, I took the idea in your request for the original ZigStar gateway. So thank you!

mercenaruss · 2021-10-18T16:57:18Z

@Hedda @xyzroe all what remains to make a PR to HA core,in order to be added in next updated.

Hedda · 2021-10-19T09:15:04Z

@Hedda greetings to you!

My firmware already contains Zeroconf DNS TXT records since release 0.5.6

You can see the recordings themselves here

Cool! It does look like missing a unique service name for the zigstar_zb_gw, baud_rate, and default tcp port is probably not good or?

I am not a C/C++ developer and at least not see in the code that you set baud_rate to a specific speed and looks like using the default service name and port? Looks like it uses strings so assume that it is set in configuration via web interface or something?

Note that while the service name does not need to contain type of Zigbee MCU (e.i. "cc2652p") however it should perhaps do so because the others have done so.

I believe that the important part is that both its unique hostname and unique service name starts with the word "zigstar". Looks like you set the service name now to "zig_star_gw" so wondering if that is on purpose when product is called ZigStar without a space in its name?

Anyway, in order to get it working Zeroconf and the ZHA integration in Home Assistant is going to first need to get these things:

hostname=zigstar_zb_gw
port=6638
service=zigstar_zb_gw_cc2652p
radio_type=znp
baud_rate=115200
data_flow_control=software

Can you maybe post the test results obtained with avahi-browse -a -r so that can see that txt format and info looks correct?

A compatible result from avahi-browse -a -r should look something like this:

_zigstar_zb_gw_cc2652p._tcp      local
  hostname = [zigstar_zb_gw.local]
  address = [192.168.0.174]
  port = [6638]
  txt = ["version=1.0" "radio_type=znp" "baud_rate=115200" "data_flow_control=software"]

If you can submit a patch to Home Assistant core which will makes my Zeroconf DNS TXT Records in whitelist it for discovery I will be grateful!

@Hedda @xyzroe all what remains to make a PR to HA core,in order to be added in next updated.

@xyzroe @mercenaruss Since I don't have a ZigStar Gateway I can't test it so maybe best would be if you test and submit together?

I believe only thing needed you is to change there is these two files (looking at how "tube*" was added for the "zha" domain):

https://github.com/home-assistant/core/blob/dev/homeassistant/generated/zeroconf.py

https://github.com/home-assistant/core/blob/dev/homeassistant/components/zha/manifest.json

Copy this and replace it with your service and name (plus remove "domain": "esphome" as your FW is not based on ESPHome):

https://github.com/home-assistant/core/blob/dev/homeassistant/generated/zeroconf.py#L79

   "_esphomelib._tcp.local.": [
       {
           "domain": "esphome"
       },
       {
           "domain": "zha",
           "name": "tube*"
       }
   ],

So think should your unique zigstar service discovery for the zha domian should read something like:

   "_zigstar_zb_gw_cc2652p._tcp.local.": [
       {
           "domain": "zha",
           "name": "zigstar*"
       }
   ],

Then in the other file copy this and under it replace the matching type and name with your service and name:

https://github.com/home-assistant/core/blob/dev/homeassistant/components/zha/manifest.json#L26

   {
     "type": "_esphomelib._tcp.local.",
     "name": "tube*"
   }

So think should your unique zigstar service discovery in the zha integration manifest should read something like:

   {
     "type": "_zigstar_zb_gw_cc2652p._tcp.local.",
     "name": "zigstar*"
   }

xyzroe · 2021-10-19T11:06:28Z

@Hedda thanks for the detailed answer.

Parameters for transfer to Zeroconf are taken from the gateway settings.
Since there are no clear technical requirements for the name of the service, it will be called _zig_star_gw. (already used in my multi-tool)
Space and diphis are allowed in the name of the court at the address of the ZigStar website.
Current avahi-browse -a -r output is below:

= ens160 IPv4 ZigStarGW                                     _zig_star_gw._tcp    local
   hostname = [ZigStarGW.local]
   address = [10.0.10.145]
   port = [4444]
   txt = ["version=1.0" "radio_type=znp" "baud_rate=115200" "data_flow_control=software"]

You can test it on any ESP32 board, or ask me or @mercenaruss to help with tests. I would try to make a PR in Home Assistant, but I don't have time to figure it out yet. I would be grateful if you can do it.

Hedda · 2021-10-20T06:49:54Z

Since there are no clear technical requirements for the name of the service, it will be called _zig_star_gw. (already used in my multi-tool) . Space and diphis are allowed in the name of the court at the address of the ZigStar website.

@mercenaruss Since are its founder I am wondering what your opinion is in regards to how the ZigStar Gatewaty DNS name should be presented via local mDNS?

I do not mean to bike-shead this because believe it or not this is imporant for the future as the name you select for local mDNS you will be stuck with in order to keep backwards compatibility with Home Assistant Zeroconf discovery.

The problem there is that Home Assistant core should only need to use one name containing a single wildcard for Zeroconf disocvery based on its local mDNS name. As in detection in Home Assistant will use either "zigstar*", "zig-star*", or "zig_star*", but not more than one of them those.

With that, I mean that I do not think that we should ask Home Assistant to whitelist more than a single name for Zeroconf discovery of any ZigStar gateway. So the name you select now you will be stuck with forever. Please only pick one and stick with that.

So the question is if the local mDNS name for the zigstar gateway firmware should it be "zigstar" or "zig-star", or "zig_star"?

You can test it on any ESP32 board, or ask me or @mercenaruss to help with tests. I would try to make a PR in Home Assistant, but I don't have time to figure it out yet. I would be grateful if you can do it.

I can do a quick PR in the blind with a [WIP] (Work-In-Progress) tag to Home Assistant core if and when you are sure about hostname and service name but I currently do not have the possibility to test it myself, so you or another user (s) would have to test it and give a thumbs-up when it is OK to remove that [WIP] tag.

mercenaruss · 2021-10-20T10:27:59Z

Hi @Hedda , now really busy with work.
But i think we should use something like ""zigstar*.local"
For example, people will want to change hostname, what is actually also mDNS to zigstar-1.local and another device to zigstar-2.local,this scenario is possible if people use 2 Z2M instances and 2 LAN Coordinators.

But in case with ZHA only one coordinator is possible to be used at the moment, so not sure actually what local mDNS to use:

zigstar.local
zigstar*.local

Can you advice on this? With which one we should stick?

Btw i dont know what Tubezb used to 2 names for the same cc2652 setup.
Anyway we will stick to one only.

Hedda · 2021-10-21T09:46:52Z

I like to suggest that you either use "zig-star*" or "zigstar*" as the default hostname and the default mDNS/Zeroconf service name.

Nothing that it only the "zig-star*" or "zigstar*" prefix that you need to decide and nail down now, anything after that can change.

I only mean that "zig_star*" (with a space symbol in its name) does not make sense for consistency when the products are called "ZigStar" (without space or hyphen symbol in it) however your official website currently uses zig-star.com as its domian name. Just be aware but for consistency reason it really makes no sense to use a space symbol in the local mDNS address. Either of the other two makes more sense, regardless you only want to pick one and stick with just that one.

Personally think that I recommend you should use "zig-star" as zig-star.local makes sense when the product website is zig-star.com

Zeroconf and ZHA integrations in HA don't care about hostname, but need to whitelist both the mDNS prefix and the service prefix.

Keep in mind that you might make other models or of the gateway in the future so might want to add model and version too.

Example: "zig-star-gateway-cc2652p-v1.local" or "zigstar-gateway-cc2652p-v1.local"

If a user has multiple ZigStar gateways they want discoverable on the network then they could add a number or location to those.

Example:

"zig-star-gateway-cc2652p-v1-1.local"
"zig-star-gateway-cc2652p-v1-2.local"
"zig-star-gateway-cc2652p-v1-3.local"

or

"zig-star-gateway-cc2652p-v1-livingroom.local"
"zig-star-gateway-cc2652p-v1-basement.local"
"zig-star-gateway-cc2652p-v1-garage.local"

Note! While you are technically allowed to use space symbols in FQDN (Fully Qualified Domain Name) specification please know that Active Directory (AD) does not support space symbol in hostnames, so if your AD entries represent hostnames which you want to match the host public FQDN then understand that space is not a valid character.

PS: Yes I have CDO..., it is like OCD, but all the letters are in alphabetical order, as they should be! ;)

Hedda · 2021-10-22T13:31:38Z

Draft WIP PR posted here now -> home-assistant/core#58224

Can you test and verify? -> https://github.com/Hedda/home-assistant/tree/patch-7

Only two files updated compared to in the dev branch of Home Assistant core:

https://github.com/Hedda/home-assistant/blob/patch-7/homeassistant/generated/zeroconf.py#L79

    "_esphomelib._tcp.local.": [
        {
            "domain": "esphome"
        },
        {
            "domain": "zha",
            "name": "tube*"
        }
    ],
    "_zig_star_gw._tcp.local.": [
        {
            "domain": "zha",
            "name": "zigstar*"
        }
    ],

https://github.com/Hedda/home-assistant/blob/patch-7/homeassistant/components/zha/manifest.json#L25

"zeroconf": [
    {
      "type": "_esphomelib._tcp.local.",
      "name": "tube*"
    },
    {
      "type": "_zig_star_gw._tcp.local.",
      "name": "zigstar*"
    }
  ],

@mercenaruss The question remains if you wish to change to use different mDNS and service names such as ex. zig-star?

mercenaruss · 2021-10-22T13:36:43Z

@Hedda you are our savior, didn't have chance to figure out how build and test HA.
@xyzroe please can you check this PR.
I think we will keep mDNS/hostname: zigstar

Hedda · 2021-10-25T08:40:02Z

@xyzroe please can you check this PR. I think we will keep mDNS/hostname: zigstar

Then you likely want to change zig_star_gw to zigstar, zig-star, zigstar-gw or zig-star-gw as default for consistency:

https://github.com/xyzroe/ZigStarGW-FW/blob/main/src/main.cpp#L648-L652

ZigStarGW-FW/src/main.cpp

Lines 648 to 652 in fd41317

 MDNS.addService("zig_star_gw", "tcp", ConfigSettings.socketPort); 

 MDNS.addServiceTxt("zig_star_gw", "tcp", "version", "1.0"); 

 MDNS.addServiceTxt("zig_star_gw", "tcp", "radio_type", "znp"); 

 MDNS.addServiceTxt("zig_star_gw", "tcp", "baud_rate", String(ConfigSettings.serialSpeed)); 

 MDNS.addServiceTxt("zig_star_gw", "tcp", "data_flow_control", "software");

Again, personally, I think it currently would make the most sense to use zig-star to mDNS will become zigstar.local as it since it as it stands today you are currently the owner of zig-star.com as the public domain (with a hyphen symbol instead of a space symbol).

However, if you also buy the zigstar.com domain name and migrate to it as your primary public domain then it would make more sense to me to use zigstar since the name of the product is "ZigStar" spelt without a hyphen (and without space symbol).

xyzroe · 2021-10-27T10:17:05Z

@Hedda
I can leave the name of the service as it is, I can also change it to zigstar_gw - it's not a problem.

I will do it as it will be in the PR in the home-assistant/core that you do, but your draft PR does not pass some checks, could you look and fix it? home-assistant/core/pull/58224

You can leave the service name as it is, that's okay.

Personally, I see no reason to make any changes to the name of the service.

Hedda · 2022-02-03T09:06:37Z

FYI, Zero-configuration networking (zeroconf) looks to have been changed and improved in the Home Assistant 2022.02 release:

home-assistant/core#62133

https://www.home-assistant.io/integrations/zeroconf/

This change is only a concern for custom integration developers.

Currently zeroconf matching only allows matching the macaddress, model, and manufacturer properties along with the name from the ZeroconfServiceInfo.

Since properties are arbitrarily defined by the zeroconf service, the list of named properties has grown over time.

Matching now allows for any arbitrarily defined property. All property matches must be lowercase, wildcards are supported

The top level keys model, manufacturer, and macaddress are now deprecated from components manifest.json file and should be moved into a properties dict.

For example:

-    {"type":"_airplay._tcp.local.","model":"appletv*"}
+    {"type":"_airplay._tcp.local.","properties":{"model":"appletv*"}}

Hedda · 2022-02-06T06:16:54Z

By the way, Home Assistant founders/leads now explicitly said this year they will prioritize focus on features like these types:

https://www.home-assistant.io/blog/2022/01/19/streamlining-experiences/

Check out this video at around 11 min 55 sec in where talk about that focus point about making it easier to get started, etc.

https://www.youtube.com/watch?v=t_2D_KoFIfU&t=710s

Hopefully, it should be a greater chance of getting more help from other Home Assistant developers with this if ask them.

Hedda · 2022-02-06T09:33:51Z

FYI, PR home-assistant/core#58224 for Home Assistant core has been closed since cannot test it myself to confirm working or not.

It would be more than welcomed if anyone (who owns a ZigStar LAN Gateway) could take over to get zeroconf discovery in ZHA.

https://www.home-assistant.io/integrations/zha#discovery-via-usb-or-zeroconf

Again, other than Tube's Zigbee Gateways you can also checkout syssi's and thegroove's implementations as proof-of-concept:

https://github.com/syssi/esphome-zb-gw03

https://github.com/thegroove/esphome-zeroconf

https://github.com/thegroove/esphome-zha-ezsp-zeroconf

Hedda · 2022-03-30T12:20:35Z

FYI, for reference, noticed that fairecasoimeme from ZiGate is working on PR for Home Assistant to add ZiGate discovery support:

See -> home-assistant/core#68577 ("Add ZiGate device on automatic integration USB and ZEROCONF")

It will add automatic ZeroConf (MDNS) discovery of ZiGate Ethernet Gateway (not WiFi Gateway), as well as ZiGate USB adapters.

https://www.home-assistant.io/integrations/zha#discovery-via-usb-or-zeroconf

As you can see in PR, ZHA and Home Assistant developers are assisting fairecasoimeme from ZiGate once PR was submitted.

Hopefully, it should be a greater chance of getting more help from other Home Assistant developers with this if you submit a PR.

xyzroe · 2022-04-15T00:55:35Z

@Hedda unfortunately I don't have the opportunity to do this at the moment.
If you yourself or @mercenaruss can do this, it will be great.

Hedda · 2022-04-19T13:17:40Z

Don't have a ZigStar to test myself. Hopefully @mercenaruss can look at this after home-assistant/core#68577 been merged.

mercenaruss · 2022-04-20T13:17:08Z

@Hedda @xyzroe Will be back from vacation by beginining of May and will solve this once and forever.

xyzroe · 2022-04-21T12:44:10Z

@mercenaruss Do you know when the war will end?
While there is a war going on in my country, there is definitely no time for a project.

Other than that, I can't figure out what is required of me? Firmware has long supported Zeroconf DNS TXT records. This mechanism is used to search for ZigStars in the Multitool. If we need to correct some values, please let us know which ones.

Hedda · 2022-05-10T06:42:49Z

Don't have a ZigStar to test myself. Hopefully @mercenaruss can look at this after home-assistant/core#68577 been merged.

FYI, PR for ZiGate's Zeroconf has been merged in Home Assistant core making it an example for adding additional more readable:

home-assistant/core#68577

https://github.com/home-assistant/core/blob/dev/homeassistant/components/zha/manifest.json#L85-L88

https://github.com/home-assistant/core/blob/dev/homeassistant/components/zha/config_flow.py#L198-L201

https://github.com/home-assistant/core/blob/dev/tests/components/zha/test_config_flow.py#L80-L107

https://github.com/home-assistant/core/blob/dev/homeassistant/generated/zeroconf.py#L384-L389

xyzroe · 2022-07-13T10:11:43Z

@mercenaruss any news about ZigStar's Zeroconf merging in Home Assistant ?

mercenaruss · 2022-07-18T20:32:38Z

@xyzroe yes,i make the test,give me some days.

Hedda · 2022-08-03T16:04:42Z

@mercenaruss any news about ZigStar's Zeroconf merging in Home Assistant ?

@xyzroe yes,i make the test,give me some days.

Nice! Looking forward to ZHA supporting. Again you can look at home-assistant/core#58224 as an untested example. Note to not manually edit the homeassistant/generated/zeroconf.py file as I did, instead use python -m script.hassfest to update/re-generate it. Also remember to submit matching pull reuest for ZHA docs on home-assistant.io for their 'next' branch:

https://github.com/home-assistant/home-assistant.io/blob/next/source/_integrations/zha.markdown

https://www.home-assistant.io/integrations/zha#discovery-via-usb-or-zeroconf

mercenaruss · 2022-09-11T22:13:35Z

@Hedda Integration of zeroconf is done.
PR submitted.
issues closed

Hedda · 2022-09-12T10:05:18Z

@mercenaruss Nice, see you submitted home-assistant/core#78237 as pull request to Home Assistant core (for ZHA integration).

Note that they also want a matching ZHA docs PR to the next branch of home-assistant.io with cross-linked to parent/child PR:

https://github.com/home-assistant/home-assistant.io/blob/next/source/_integrations/zha.markdown

https://github.com/home-assistant/home-assistant.io/edit/next/source/_integrations/zha.markdown#L182

| [ZigStar Stick (CC2652 + CH340B variant)](https://zig-star.com/projects/zigbee-stick-v4/) | USB | 1A86:7523 |
| [ZigStar LAN Gateway - Zigbee to Ethernet Serial Coordinator)](https://www.tubeszb.com/) | Zeroconf | zigstar_gw._tcp.local. |
| [Tube’s EFR32 Pro Ethernet/Serial Coordinator](https://www.tubeszb.com/) | USB| 10C4:EA60 |
| [Tube's CC2652P2 USB-powered Zigbee to Ethernet Serial Coordinator)](https://www.tubeszb.com/) | Zeroconf | tube_zb_gw_cc2652p2.local. |

mercenaruss · 2022-09-12T11:39:03Z

@Hedda Done: home-assistant/home-assistant.io#24105

Hedda mentioned this issue Oct 22, 2021

Add Zeroconf discovery for ZigStar Zigbee coordinator to ZHA home-assistant/core#58224

Closed

22 tasks

xyzroe added the enhancement New feature or request label Oct 27, 2021

xyzroe closed this as completed Sep 12, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[REQUEST] Zeroconf DNS TXT records parameters in ZigStarGW-FW to support Home Assistant ZHA network discovery #7 #3

[REQUEST] Zeroconf DNS TXT records parameters in ZigStarGW-FW to support Home Assistant ZHA network discovery #7 #3

Hedda commented Oct 18, 2021

xyzroe commented Oct 18, 2021 •

edited

Loading

xyzroe commented Oct 18, 2021

mercenaruss commented Oct 18, 2021

Hedda commented Oct 19, 2021 •

edited

Loading

xyzroe commented Oct 19, 2021

Hedda commented Oct 20, 2021

mercenaruss commented Oct 20, 2021

Hedda commented Oct 21, 2021 •

edited

Loading

Hedda commented Oct 22, 2021 •

edited

Loading

mercenaruss commented Oct 22, 2021 •

edited

Loading

Hedda commented Oct 25, 2021 •

edited

Loading

xyzroe commented Oct 27, 2021

Hedda commented Feb 3, 2022

Hedda commented Feb 6, 2022

Hedda commented Feb 6, 2022 •

edited

Loading

Hedda commented Mar 30, 2022 •

edited

Loading

xyzroe commented Apr 15, 2022

Hedda commented Apr 19, 2022 •

edited

Loading

mercenaruss commented Apr 20, 2022

xyzroe commented Apr 21, 2022

Hedda commented May 10, 2022 •

edited

Loading

xyzroe commented Jul 13, 2022

mercenaruss commented Jul 18, 2022

Hedda commented Aug 3, 2022 •

edited

Loading

mercenaruss commented Sep 11, 2022

Hedda commented Sep 12, 2022

mercenaruss commented Sep 12, 2022

[REQUEST] Zeroconf DNS TXT records parameters in ZigStarGW-FW to support Home Assistant ZHA network discovery #7 #3

[REQUEST] Zeroconf DNS TXT records parameters in ZigStarGW-FW to support Home Assistant ZHA network discovery #7 #3

Comments

Hedda commented Oct 18, 2021

xyzroe commented Oct 18, 2021 • edited Loading

xyzroe commented Oct 18, 2021

mercenaruss commented Oct 18, 2021

Hedda commented Oct 19, 2021 • edited Loading

xyzroe commented Oct 19, 2021

Hedda commented Oct 20, 2021

mercenaruss commented Oct 20, 2021

Hedda commented Oct 21, 2021 • edited Loading

Hedda commented Oct 22, 2021 • edited Loading

mercenaruss commented Oct 22, 2021 • edited Loading

Hedda commented Oct 25, 2021 • edited Loading

xyzroe commented Oct 27, 2021

Hedda commented Feb 3, 2022

Hedda commented Feb 6, 2022

Hedda commented Feb 6, 2022 • edited Loading

Hedda commented Mar 30, 2022 • edited Loading

xyzroe commented Apr 15, 2022

Hedda commented Apr 19, 2022 • edited Loading

mercenaruss commented Apr 20, 2022

xyzroe commented Apr 21, 2022

Hedda commented May 10, 2022 • edited Loading

xyzroe commented Jul 13, 2022

mercenaruss commented Jul 18, 2022

Hedda commented Aug 3, 2022 • edited Loading

mercenaruss commented Sep 11, 2022

Hedda commented Sep 12, 2022

mercenaruss commented Sep 12, 2022

xyzroe commented Oct 18, 2021 •

edited

Loading

Hedda commented Oct 19, 2021 •

edited

Loading

Hedda commented Oct 21, 2021 •

edited

Loading

Hedda commented Oct 22, 2021 •

edited

Loading

mercenaruss commented Oct 22, 2021 •

edited

Loading

Hedda commented Oct 25, 2021 •

edited

Loading

Hedda commented Feb 6, 2022 •

edited

Loading

Hedda commented Mar 30, 2022 •

edited

Loading

Hedda commented Apr 19, 2022 •

edited

Loading

Hedda commented May 10, 2022 •

edited

Loading

Hedda commented Aug 3, 2022 •

edited

Loading