auto create parameter.json

[zivillian]

After reversing the android app, ism7mqtt no longer requires the smartset pc app and database to create a valid parameter.json. Instead there is an ism7config binary which directly connects to the ism, reads the available devices and parameters and writes them to the paramter.json. This is currently still in beta, because I haven't found anyone to test the new implementation.

Do you have any idea how this might be implemented for this addon? I'm still not really using HA, so I don't know how this is done by other integrations, but maybe you have an idea.

[b3nn0]

Hmm, I think that should be fairly easy to implement.
Instead of copy-pasting the parameter.json to the web-interface-configuration of this Addon, I could just remove this setting, and instead, on startup, check if parameters.json already exists, and if not, run ism7config first and store the generated parameters.json.

Is there an ism7config binary somewhere, so I can do a quick test?
Will it be included in your docker image, so I can actually run it first?

[b3nn0]

Never mind, found everything I need. Worked well for me.
I'll implement this into the addon once it is released on your side (v0.0.12?).

I'll leave the option to override it from the addon-settings, in case people only want to query specific values.

EDIT: Are you interested in the log-output for generating the parameters.json for my CGW-2-20 for an additional data point?

[zivillian]

Thanks for your fast response. That sounds really good and I'll let you know here, when it's released.

I would suggest to only generate the file if it's missing and store it in a permanent location, so anyone can remove unwanted parameters (I did this for at least 5 duplicates) or manually adjust the config.

PS:
Regarding your log file: if it worked then I don't think I need it. Have you used the binaries from zivillian/ism7mqtt#43 for testing? If so, I'd say it's tested and also merge after zivillian/ism7mqtt#57.

[b3nn0]

I used this one I think:
https://github.com/zivillian/ism7mqtt/actions/runs/4909017244

On creating, there was quite elaborate logging, and it did print some stuff that looked like errors, not sure if normal.
However, it did generate a parameters.json that looked valid. Didn't test it though (should I?).

Stuff that looked suspicious:

2023-05-11 11:08:36.4328|WARN|GW-1|<GW:1,-1> - Response state:ER !!! entry:Resp -> Seq:A;30, Ba:0x35, TelegrNr:9007, State:ER, set value to null
2023-05-11 11:08:36.4376|WARN|GW-1|<GW:1,-1> - Response state:ER !!! entry:Resp -> Seq:A;29, Ba:0x35, TelegrNr:9024, State:ER, set value to null
2023-05-11 11:08:36.4376|WARN|GW-1|<GW:1,-1> - Response state:ER !!! entry:Resp -> Seq:A;31, Ba:0x35, TelegrNr:9028, State:ER, set value to null
2023-05-11 11:08:36.4408|ERROR|GW-1|<GW:1,-1> - Missing device answer - ParameterId:34005000000, state:Error
2023-05-11 11:08:36.4408|ERROR|GW-1|<GW:1,-1> - Missing device answer - ParameterId:34004500000, state:Error
2023-05-11 11:08:36.4408|ERROR|GW-1|<GW:1,-1> - Missing device answer - ParameterId:34005500000, state:Error

Can I assume that the ism7config binary will be part of your docker image? That would certainly simplify things for me a lot, because the Add-on builds on that one.

And yes, the parameters.json will be stored in a user-accessible location to be modified, and only auto-generated if it doesn't exist.

[zivillian]

I've enabled trace logging for the smartset binaries in the ism7config tool, just in case something goes wrong. There may be some errors like yours, but as far as I can tell, this is normal since it's trying to detect the installed components and some may not be present.

I will update the docker image to also contain the ism7config binary.

It would be great if you can also test, but I think it's much easier after I've updated the docker image.

[zivillian]

I've updated the docker image (currently only the master tag to not break someones setup) to also contain the ism7config tool.

[b3nn0]

Feel free to push it as a version/latest. The current add-on version explicitly references v0.0.12, so it shouldn't break anything for this add-on.

Regarding the parameter file creation:
Seems to have worked out, but I had to export DOTNET_SYSTEM_GLOBALIZATION_INVARIANT=false explicitly before running it, not sure why.

I prepared everything to switch to v0.0.13 once it's on dockerhub. It will create the parameters.json in shared storage if it doesn't exist already, so the user can edit it if desired.

[b3nn0]

Oops, something is strange with that version though.
All my sensors have duplicated, because their ID has changed.
E.g.
old: sensor.wolf_bm2_0x30_22023_antilegionellenfunktion
new: sensor.wolf_bm2_0x30_220023_antilegionellenfunktion

Seems to be the same schema for all sensors.
was that intentional?

[zivillian]

was that intentional?

sadly yes - the ids changed during the migration from the desktop to the mobile app - that's why v0.0.13 is also a breaking change in ism7mqtt

[zivillian]

I've found the following migration in the Wolf binaries, but I don't know if there is any way to migrate HA entities:

[zivillian]

Seems to have worked out, but I had to export DOTNET_SYSTEM_GLOBALIZATION_INVARIANT=false

What happens if you omit this?

[b3nn0]

Unfortunately, migration is not really an option with the way mqtt discovery is implemented.
It's not that big of an issue, but users will have to manually delete all the old discovery topics with something like MQTTExplorer. I will note that in the CHANGES that are displayed before installing the update.

[b3nn0]

What happens if you omit this?

It errors out with some XML parsing error stack trace.. didn't save it unfortunately, I can send it tomorrow if desired

[b3nn0]

This is what I get if I run it without the mentioned environment variable:

++ /app/ism7config -t /config/ism7-parameters.json
Unhandled exception. System.InvalidOperationException: There is an error in XML document (120, 6).
 ---> System.TypeInitializationException: The type initializer for 'Wolf.SocketServer.ISMInterface.DateTimeConverterTemplate' threw an exception.
 ---> System.Globalization.CultureNotFoundException: Only the invariant culture is supported in globalization-invariant mode. See https://aka.ms/GlobalizationInvariantMode for more information. (Parameter 'name')
de-DE is an invalid culture identifier.
   at System.Globalization.CultureInfo..ctor(String name, Boolean useUserOverride)
   at System.Globalization.CultureInfo..ctor(String name)
   at Wolf.SocketServer.ISMInterface.DateTimeConverterTemplate..cctor()
   --- End of inner exception stack trace ---
   at Microsoft.Xml.Serialization.GeneratedAssembly.XmlSerializationReaderConverterTemplateConfig.Read15_DateTimeConverterTemplate(Boolean isNullable, Boolean checkType)
   at Microsoft.Xml.Serialization.GeneratedAssembly.XmlSerializationReaderConverterTemplateConfig.Read34_ConverterTemplateConfig(Boolean isNullable, Boolean checkType)
   at Microsoft.Xml.Serialization.GeneratedAssembly.XmlSerializationReaderConverterTemplateConfig.Read35_ConverterTemplateConfig()
   --- End of inner exception stack trace ---
   at System.Xml.Serialization.XmlSerializer.Deserialize(XmlReader xmlReader, String encodingStyle, XmlDeserializationEvents events)
   at System.Xml.Serialization.XmlSerializer.Deserialize(XmlReader xmlReader)
   at lucon.common.secret.XmlSerialization.DeserializeXmlFile[DeserializeType](XmlSerializer serializer, String path, Func`2 getStream, Boolean useEncryption)
   at Wolf.SocketServer.ISMInterface.ConverterTemplateServiceImplISM..ctor(FileSystemSettings settings, Func`2 getStream)
   at ism7config.Program.Main(String[] args) in /app/ism7config/Program.cs:line 77
   at ism7config.Program.<Main>(String[] args)
/run.sh: line 29:   129 Aborted                 (core dumped) /app/ism7config -t /config/ism7-parameters.json

[zivillian]

I'll take care of the XML parsing - maybe it's already fixed in another branch...

[zivillian]

I've pushed a new docker image, which already contains the env variable

[b3nn0]

perfect. Let me know once a consistent tag is on dockerhub, then I'll release the new add-on version.

[sniesen]

Hi! I'm using the new version @zivillian provided yesterday to create the parameter.json - in HA I get the following error:

Starting ism7mqtt --hass-id=WolfLinkPro --interval=60 -t /parameter.json -d
++ ISM_ARGS='--hass-id=WolfLinkPro --interval=60 -t /parameter.json'
++ [[ true == \t\r\u\e ]]
++ ISM_ARGS+=' -d'
++ cd /app
++ '[' true ']'
++ echo 'Starting ism7mqtt --hass-id=WolfLinkPro --interval=60 -t /parameter.json -d'
++ /app/ism7mqtt --hass-id=WolfLinkPro --interval=60 -t /parameter.json -d

XXXXXXXX

< WolfLinkXXXXXXXX2023-06-27T18:40:57300.10.1.300.300.13.0WOLFLink

< 
System.InvalidOperationException: Sequence contains no matching element
at System.Linq.ThrowHelper.ThrowNoMatchException()
at ism7mqtt.Ism7Config.AddDevice(String ip, String ba)
at ism7mqtt.Ism7Client.LoadInitialValuesAsync(CancellationToken cancellationToken) in /app/ISM7/Ism7Client.cs:line 281
at ism7mqtt.ResponseDispatcher.DispatchAsync(IResponse response, CancellationToken cancellationToken) in /app/ISM7/ResponseDispatcher.cs:line 32
at ism7mqtt.Ism7Client.ReadPipeAsync(PipeReader source, CancellationToken cancellationToken) in /app/ISM7/Ism7Client.cs:line 213
++ sleep 10

Any ideas?

[b3nn0]

The current version in the addon is still the previous version v0.0.12, with a different parameters file format. That's what we're working on right now to geht the new version in the add-on.
The parameters.json will then be auto-generated directly by the add-on. No manual steps needed.

[sniesen]

Let me know if I can help in any way. I got a rare setup with a CHA-10 supported by a TGB-2.

[zivillian]

@b3nn0 I've pushed the v0.0.13 tag to docker

[b3nn0]

Alright, bumped the addon version with v0.0.13, including automatic parameter.json generation. Let's see how it works out.

[zivillian]

@sniesen it would be great if you could test and confirm, that it actually works for your setup

[zivillian]

@b3nn0 I have a few comments and didn't know where to leave them.

This shouldn't be needed anymore:

hassio-addon-ism7mqtt/ism7mqtt/run.sh

Line 14 in ef57467

export DOTNET_SYSTEM_GLOBALIZATION_INVARIANT=false

Feel free to link directly to https://github.com/zivillian/ism7mqtt/issues/new:

hassio-addon-ism7mqtt/ism7mqtt/run.sh

Line 26 in ef57467

echo "Parameter file creation seems to have failed. Please report to the ISM7MQTT Author"

[b3nn0]

Ah, makes sense, yes. I wasn't sure if the v0.0.13 tag on dockerhub references yesterdays v0.0.13 tag on github, and would therefore still have the issue or not, so thought "better safe than sorry". But I just noticed that you re-wrote the tag, so I guess that should be fine.

[sniesen]

Looks great so far! Thanks a lot to the two of you!

[zivillian]

@sniesen That's good news - thanks for your quick feedback. Can you please check if you are able to write values like 1x Warmwasser and if not report at zivillian/ism7mqtt#57?