From 679dd901d195b04629973d6307a422e263c71fcf Mon Sep 17 00:00:00 2001 From: Travis Howse Date: Wed, 30 Dec 2020 22:30:50 +1000 Subject: [PATCH 01/11] Reformat readme --- README.md | 58 +++++++++++++++++++++++++++---------------------------- 1 file changed, 28 insertions(+), 30 deletions(-) diff --git a/README.md b/README.md index c5f1ade..85a4b36 100644 --- a/README.md +++ b/README.md @@ -48,19 +48,16 @@ address_offset: 0 variant: sungrow scan_batching: 100 ``` - -`ip` (Required) The IP address of the modbus device to be polled. Presently only modbus TCP/IP is supported. - -`port` (Optional: default 502) The port on the modbus device to connect to. - -`update_rate` (Optional: default 5) The number of seconds between polls of the modbus device. - -`address_offset` (Optional: default 0) This offset is applied to every register address to accommodate different Modbus addressing systems. In many Modbus devices the first register is enumerated as 1, other times 0. See section 4.4 of the Modbus spec. - -`variant` (Optional) Allows variants of the ModbusTcpClient library to be used. Setting this to 'sungrow' enables support of SungrowModbusTcpClient. This library transparently decrypts the modbus comms with sungrow SH inverters running newer firmware versions. - -`scan_batching` (Optional: default 100) Must be between 1 and 100 inclusive. Modbus read operations are more efficient in bigger batches of contiguous registers, but different devices have different limits on the size of the batched reads. This setting can also be helpful when building a modbus register map for an uncharted device. In some modbus devices a single invalid register in a read range will fail the entire read operation. By setting `scan_batching` to `1` each register will be scanned individually. This will be very inefficient and should not be used in production as it will saturate the link with many read operations. - +| Field name | Required | Default | Description | +| ---------- | -------- | ------- | ----------- | +| ip | Required | N/A | The IP address of the modbus device to be polled. Presently only modbus TCP/IP is supported. | +| port | Optional | 502 | The port on the modbus device to connect to. | +| update_rate | Optional | 5 | The number of seconds between polls of the modbus device. | +| address_offset | Optional | 0 | This offset is applied to every register address to accommodate different Modbus addressing systems. In many Modbus devices the first register is enumerated as 1, other times 0. See section 4.4 of the Modbus spec. | +| variant | Optional | N/A | Allows variants of the ModbusTcpClient library to be used. Setting this to 'sungrow' enables support of SungrowModbusTcpClient. This library transparently decrypts the modbus comms with sungrow SH inverters running newer firmware versions. | +| scan_batching | Optional | 100 | Must be between 1 and 100 inclusive. Modbus read operations are more efficient in bigger batches of contiguous registers, but different devices have different limits on the size of the batched reads. This setting can also be helpful when building a modbus register map for an uncharted device. In some modbus devices a single invalid register in a read range will fail the entire read operation. By setting `scan_batching` to `1` each register will be scanned individually. This will be very inefficient and should not be used in production as it will saturate the link with many read operations. | + +### Register settings ```yaml registers: - pub_topic: "forced_charge/mode" @@ -83,24 +80,25 @@ registers: - pub_topic: "first_bit_of_second_byte" address: 13001 mask: 0x0010 + - pub_topic: "load_control/optimized/end_time" + address: 13013 + json_key: hours + - pub_topic: "load_control/optimized/end_time" + address: 13014 + json_key: minutes ``` This section of the YAML lists all the modbus registers that you consider interesting. -`address` (Required) The decimal address of the register to read from the device, starting at 0. Many modbus devices enumerate registers beginning at 1, so beware. - -`pub_topic` (Optional) This is the topic to which the value of this register will be published. - -`set_topic` (Optional) Values published to this topic will be written to the Modbus device. - -`retain` (Optional: default false) Controls whether the value of this register will be published with the retain bit set. - -`pub_only_on_change` (Optional: default true) Controls whether this register will only be published if its value changed from the previous poll. - -`table` (Optional: default 'holding') The Modbus table to read from the device. Must be 'holding' or 'input'. - -`value_map` (Optional) A series of human-readable and raw values for the setting. This will be used to translate between human-readable values via MQTT to raw values via Modbus. If a value_map is set for a register the interface will reject raw values sent via MQTT. If value_map is not set the interface will try to set the Modbus register to that value. Note that the scale is applied after the value is read from Modbus and before it is written to Modbus. - -`scale` (Optional: default 1) After reading a value from the Modbus register it will be multiplied by this scalar before being published to MQTT. Values published on this register's `set_topic` will be divided by this scalar before being written to Modbus. - -`mask` (Optional: default 0xFFFF) This is a 16-bit number that can be used to select a part of a Modbus register to be referenced by this register. For example a mask of `0xFF00` will map to the most significant byte of the 16-bit Modbus register at `address`. A mask of `0x0001` will reference only the least significant bit of this register. +| Field name | Required | Default | Description | +| ---------- | -------- | ------- | ----------- | +| address | Required | N/A | The decimal address of the register to read from the device, starting at 0. Many modbus devices enumerate registers beginning at 1, so beware. | +| pub_topic | Optional | N/A | This is the topic to which the value of this register will be published. | +| set_topic | Optional | N/A | Values published to this topic will be written to the Modbus device. | +| retain | Optional | false | Controls whether the value of this register will be published with the retain bit set. | +| pub_only_on_change | Optional | true | Controls whether this register will only be published if its value changed from the previous poll. | +| table | Optional | holding | The Modbus table to read from the device. Must be 'holding' or 'input'. | +| value_map | Optional | N/A | A series of human-readable and raw values for the setting. This will be used to translate between human-readable values via MQTT to raw values via Modbus. If a value_map is set for a register the interface will reject raw values sent via MQTT. If value_map is not set the interface will try to set the Modbus register to that value. Note that the scale is applied after the value is read from Modbus and before it is written to Modbus. | +| scale | Optional | 1 | After reading a value from the Modbus register it will be multiplied by this scalar before being published to MQTT. Values published on this register's `set_topic` will be divided by this scalar before being written to Modbus. | +| mask | Optional | 0xFFFF | This is a 16-bit number that can be used to select a part of a Modbus register to be referenced by this register. For example a mask of `0xFF00` will map to the most significant byte of the 16-bit Modbus register at `address`. A mask of `0x0001` will reference only the least significant bit of this register. | +| json_key | Optional | N/A | The value of this register will be published to its pub_topic in JSON format. E.G. `{ key: value }` Registers with a json_key specified can share a pub_topic. All registers with shared pub_topics must have a json_key specified. Multiple registers can be published to the same topic in the same JSON message. | From bd2038de98768e0f817fe4795918392e7c14091b Mon Sep 17 00:00:00 2001 From: Travis Howse Date: Thu, 31 Dec 2020 14:55:17 +1000 Subject: [PATCH 02/11] Working on input validation for json_key --- README.md | 2 +- modbus4mqtt/modbus4mqtt.py | 29 ++++++++++++++++++++++++++++- 2 files changed, 29 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index 85a4b36..39c2818 100644 --- a/README.md +++ b/README.md @@ -101,4 +101,4 @@ This section of the YAML lists all the modbus registers that you consider intere | value_map | Optional | N/A | A series of human-readable and raw values for the setting. This will be used to translate between human-readable values via MQTT to raw values via Modbus. If a value_map is set for a register the interface will reject raw values sent via MQTT. If value_map is not set the interface will try to set the Modbus register to that value. Note that the scale is applied after the value is read from Modbus and before it is written to Modbus. | | scale | Optional | 1 | After reading a value from the Modbus register it will be multiplied by this scalar before being published to MQTT. Values published on this register's `set_topic` will be divided by this scalar before being written to Modbus. | | mask | Optional | 0xFFFF | This is a 16-bit number that can be used to select a part of a Modbus register to be referenced by this register. For example a mask of `0xFF00` will map to the most significant byte of the 16-bit Modbus register at `address`. A mask of `0x0001` will reference only the least significant bit of this register. | -| json_key | Optional | N/A | The value of this register will be published to its pub_topic in JSON format. E.G. `{ key: value }` Registers with a json_key specified can share a pub_topic. All registers with shared pub_topics must have a json_key specified. Multiple registers can be published to the same topic in the same JSON message. | +| json_key | Optional | N/A | The value of this register will be published to its pub_topic in JSON format. E.G. `{ key: value }` Registers with a json_key specified can share a pub_topic. All registers with shared pub_topics must have a json_key specified. In this way, multiple registers can be published to the same topic in a single JSON message. | diff --git a/modbus4mqtt/modbus4mqtt.py b/modbus4mqtt/modbus4mqtt.py index 5e1726f..107c600 100755 --- a/modbus4mqtt/modbus4mqtt.py +++ b/modbus4mqtt/modbus4mqtt.py @@ -147,9 +147,36 @@ def _on_message(self, client, userdata, msg): continue self._mb.set_value(register.get('table', 'holding'), register['address'], int(value), register.get('mask', 0xFFFF)) + # This throws ValueError exceptions if the imported registers are invalid + @staticmethod + def _validate_registers(registers): + all_pub_topics = set() + duplicate_pub_topics = set() + # Key: shared pub_topics, value: list of json_keys + duplicate_json_keys = {} + + # Look for duplicate pub_topics + for register in registers: + if register['pub_topic'] in all_pub_topics: + duplicate_pub_topics.add(register['pub_topic']) + duplicate_json_keys[register['pub_topic']] = [] + all_pub_topics.add(register['pub_topic']) + + # Check that all registers with duplicate pub topics have json_keys + for register in registers: + if register['pub_topic'] in duplicate_pub_topics: + if 'json_key' not in register: + raise ValueError("Bad YAML configuration. pub_topic '{}' duplicated across registers without json_key field. Registers that share a pub_topic must also have a unique json_key.".format(register['pub_topic'])) + if register['json_key'] in duplicate_json_keys[register['pub_topic']]: + raise ValueError("Bad YAML configuration. pub_topic '{}' duplicated across registers with a duplicated json_key field. Registers that share a pub_topic must also have a unique json_key.".format(register['pub_topic'])) + duplicate_json_keys[register['pub_topic']] += [register['json_key']] + def _load_modbus_config(self, path): yaml=YAML(typ='safe') - return yaml.load(open(path,'r').read()) + result = yaml.load(open(path,'r').read()) + registers = [register for register in result['registers'] if 'pub_topic' in register] + mqtt_interface._validate_registers(registers) + return result def loop_forever(self): while True: From 72c0f3364999c29e7b583e43b89876b09593d226 Mon Sep 17 00:00:00 2001 From: Travis Howse Date: Thu, 31 Dec 2020 22:01:38 +1000 Subject: [PATCH 03/11] Add unit test for yaml validation. --- tests/test_mqtt.py | 37 +++++++++++++++++++++++++++++++++++++ 1 file changed, 37 insertions(+) diff --git a/tests/test_mqtt.py b/tests/test_mqtt.py index 0a63e57..ed962ea 100644 --- a/tests/test_mqtt.py +++ b/tests/test_mqtt.py @@ -366,5 +366,42 @@ def test_address_offset(self): mock_mqtt().publish.assert_any_call(MQTT_TOPIC_PREFIX+'/publish', 2, retain=False) + def test_register_validation(self): + valids = [[ # Different json_keys for same topic + {'address': 13049, 'json_key': 'a', 'pub_topic': 'ems/EMS_MODE'}, + {'address': 13050, 'json_key': 'A', 'pub_topic': 'ems/EMS_MODE'}, + {'address': 13050, 'json_key': 'b', 'pub_topic': 'ems/EMS_MODE'} + ], + [ # Different topics, duplicate json_key + {'address': 13050, 'json_key': 'A', 'pub_topic': 'ems/EMS_MODEA'}, + {'address': 13050, 'json_key': 'A', 'pub_topic': 'ems/EMS_MODEB'} + ], + [ # Different topic, no json_key + {'address': 13050, 'pub_topic': 'ems/EMS_MODEA'}, + {'address': 13050, 'json_key': 'A', 'pub_topic': 'ems/EMS_MODEB'} + ]] + invalids = [[ # Duplicate json_key for a topic + {'address': 13050, 'json_key': 'A', 'pub_topic': 'ems/EMS_MODE'}, + {'address': 13050, 'json_key': 'A', 'pub_topic': 'ems/EMS_MODE'} + ], + [ # Missing json_key for a register with a duplicated pub_topic + {'address': 13049, 'pub_topic': 'ems/EMS_MODE'}, + {'address': 13050, 'json_key': 'A', 'pub_topic': 'ems/EMS_MODE'} + ]] + for valid in valids: + try: + modbus4mqtt.mqtt_interface._validate_registers(valid) + except: + self.fail("Threw an exception checking a valid register configuration") + for invalid in invalids: + fail = False + try: + modbus4mqtt.mqtt_interface._validate_registers(invalid) + except: + fail = True + if not fail: + self.fail("Didn't throw an exception checking an invalid register configuration") + + if __name__ == "__main__": unittest.main() \ No newline at end of file From 9a16aa116636d82965a087c27005e4c2dcafe9a3 Mon Sep 17 00:00:00 2001 From: Travis Howse Date: Thu, 31 Dec 2020 23:01:19 +1000 Subject: [PATCH 04/11] Add sending values in json. Unit tests to follow. --- README.md | 2 +- modbus4mqtt/modbus4mqtt.py | 31 +++++++++++++++++++++++++++++-- tests/test_mqtt.py | 8 ++++++++ 3 files changed, 38 insertions(+), 3 deletions(-) diff --git a/README.md b/README.md index 39c2818..d5e0de4 100644 --- a/README.md +++ b/README.md @@ -101,4 +101,4 @@ This section of the YAML lists all the modbus registers that you consider intere | value_map | Optional | N/A | A series of human-readable and raw values for the setting. This will be used to translate between human-readable values via MQTT to raw values via Modbus. If a value_map is set for a register the interface will reject raw values sent via MQTT. If value_map is not set the interface will try to set the Modbus register to that value. Note that the scale is applied after the value is read from Modbus and before it is written to Modbus. | | scale | Optional | 1 | After reading a value from the Modbus register it will be multiplied by this scalar before being published to MQTT. Values published on this register's `set_topic` will be divided by this scalar before being written to Modbus. | | mask | Optional | 0xFFFF | This is a 16-bit number that can be used to select a part of a Modbus register to be referenced by this register. For example a mask of `0xFF00` will map to the most significant byte of the 16-bit Modbus register at `address`. A mask of `0x0001` will reference only the least significant bit of this register. | -| json_key | Optional | N/A | The value of this register will be published to its pub_topic in JSON format. E.G. `{ key: value }` Registers with a json_key specified can share a pub_topic. All registers with shared pub_topics must have a json_key specified. In this way, multiple registers can be published to the same topic in a single JSON message. | +| json_key | Optional | N/A | The value of this register will be published to its pub_topic in JSON format. E.G. `{ key: value }` Registers with a json_key specified can share a pub_topic. All registers with shared pub_topics must have a json_key specified. In this way, multiple registers can be published to the same topic in a single JSON message. If any of the registers that share a pub_topic have the retain field set that will affect the published JSON message. Conflicting retain settings are invalid. | diff --git a/modbus4mqtt/modbus4mqtt.py b/modbus4mqtt/modbus4mqtt.py index 107c600..8eb5d80 100755 --- a/modbus4mqtt/modbus4mqtt.py +++ b/modbus4mqtt/modbus4mqtt.py @@ -1,6 +1,7 @@ #!/usr/bin/python3 from time import sleep +import json import logging from ruamel.yaml import YAML import click @@ -77,6 +78,10 @@ def poll(self): self.connect_modbus() return + # This is used to store values that are published as JSON messages rather than individual values + json_messages = {} + json_messages_retain = {} + for register in self._get_registers_with('pub_topic'): try: value = self._mb.get_value(register.get('table', 'holding'), register['address']) @@ -98,8 +103,21 @@ def poll(self): if value in register['value_map'].values(): # This is a bit weird... value = [human for human, raw in register['value_map'].items() if raw == value][0] - retain = register.get('retain', False) - self._mqtt_client.publish(self.prefix+register['pub_topic'], value, retain=retain) + if register.get('json_key', False): + # This value won't get published to MQTT immediately. It gets stored and sent at the end of the poll. + if register['pub_topic'] not in json_messages: + json_messages[register['pub_topic']] = {} + json_messages_retain[register['pub_topic']] = False + json_messages[register['pub_topic']][register['json_key']] = value + if 'retain' in register: + json_messages_retain[register['pub_topic']] = register['retain'] + else: + retain = register.get('retain', False) + self._mqtt_client.publish(self.prefix+register['pub_topic'], value, retain=retain) + + for topic, message in json_messages.items(): + m = json.dumps(message) + self._mqtt_client.publish(self.prefix+topic, m, retain=json_messages_retain[topic]) def _on_connect(self, client, userdata, flags, rc): if rc == 0: @@ -154,12 +172,15 @@ def _validate_registers(registers): duplicate_pub_topics = set() # Key: shared pub_topics, value: list of json_keys duplicate_json_keys = {} + # Key: shared pub_topics, value: set of retain values (true/false) + retain_setting = {} # Look for duplicate pub_topics for register in registers: if register['pub_topic'] in all_pub_topics: duplicate_pub_topics.add(register['pub_topic']) duplicate_json_keys[register['pub_topic']] = [] + retain_setting[register['pub_topic']] = set() all_pub_topics.add(register['pub_topic']) # Check that all registers with duplicate pub topics have json_keys @@ -170,6 +191,12 @@ def _validate_registers(registers): if register['json_key'] in duplicate_json_keys[register['pub_topic']]: raise ValueError("Bad YAML configuration. pub_topic '{}' duplicated across registers with a duplicated json_key field. Registers that share a pub_topic must also have a unique json_key.".format(register['pub_topic'])) duplicate_json_keys[register['pub_topic']] += [register['json_key']] + if 'retain' in register: + retain_setting[register['pub_topic']].add(register['retain']) + # Check that there are no disagreements as to whether this pub_topic should be retained or not. + for topic, retain_set in retain_setting.items(): + if len(retain_set) > 1: + raise ValueError("Bad YAML configuration. pub_topic '{}' has conflicting retain settings.".format(topic)) def _load_modbus_config(self, path): yaml=YAML(typ='safe') diff --git a/tests/test_mqtt.py b/tests/test_mqtt.py index ed962ea..8e91649 100644 --- a/tests/test_mqtt.py +++ b/tests/test_mqtt.py @@ -379,6 +379,10 @@ def test_register_validation(self): [ # Different topic, no json_key {'address': 13050, 'pub_topic': 'ems/EMS_MODEA'}, {'address': 13050, 'json_key': 'A', 'pub_topic': 'ems/EMS_MODEB'} + ], + [ # Retain specified twice and consistent + {'address': 13050, 'json_key': 'A', 'pub_topic': 'ems/EMS_MODE', 'retain': True}, + {'address': 13050, 'json_key': 'B', 'pub_topic': 'ems/EMS_MODE', 'retain': True} ]] invalids = [[ # Duplicate json_key for a topic {'address': 13050, 'json_key': 'A', 'pub_topic': 'ems/EMS_MODE'}, @@ -387,6 +391,10 @@ def test_register_validation(self): [ # Missing json_key for a register with a duplicated pub_topic {'address': 13049, 'pub_topic': 'ems/EMS_MODE'}, {'address': 13050, 'json_key': 'A', 'pub_topic': 'ems/EMS_MODE'} + ], + [ # Retain specified twice and inconsistent + {'address': 13050, 'json_key': 'A', 'pub_topic': 'ems/EMS_MODE', 'retain': True}, + {'address': 13050, 'json_key': 'B', 'pub_topic': 'ems/EMS_MODE', 'retain': False} ]] for valid in valids: try: From 5f34c97c025ed32be1f340883a344c13e2b4c6d0 Mon Sep 17 00:00:00 2001 From: Travis Howse Date: Thu, 31 Dec 2020 23:19:26 +1000 Subject: [PATCH 05/11] Add test for json_key --- README.md | 2 +- modbus4mqtt/modbus4mqtt.py | 2 +- tests/test_json_key.yaml | 15 +++++++++++++++ tests/test_mqtt.py | 20 ++++++++++++++++++++ 4 files changed, 37 insertions(+), 2 deletions(-) create mode 100644 tests/test_json_key.yaml diff --git a/README.md b/README.md index d5e0de4..30675b6 100644 --- a/README.md +++ b/README.md @@ -101,4 +101,4 @@ This section of the YAML lists all the modbus registers that you consider intere | value_map | Optional | N/A | A series of human-readable and raw values for the setting. This will be used to translate between human-readable values via MQTT to raw values via Modbus. If a value_map is set for a register the interface will reject raw values sent via MQTT. If value_map is not set the interface will try to set the Modbus register to that value. Note that the scale is applied after the value is read from Modbus and before it is written to Modbus. | | scale | Optional | 1 | After reading a value from the Modbus register it will be multiplied by this scalar before being published to MQTT. Values published on this register's `set_topic` will be divided by this scalar before being written to Modbus. | | mask | Optional | 0xFFFF | This is a 16-bit number that can be used to select a part of a Modbus register to be referenced by this register. For example a mask of `0xFF00` will map to the most significant byte of the 16-bit Modbus register at `address`. A mask of `0x0001` will reference only the least significant bit of this register. | -| json_key | Optional | N/A | The value of this register will be published to its pub_topic in JSON format. E.G. `{ key: value }` Registers with a json_key specified can share a pub_topic. All registers with shared pub_topics must have a json_key specified. In this way, multiple registers can be published to the same topic in a single JSON message. If any of the registers that share a pub_topic have the retain field set that will affect the published JSON message. Conflicting retain settings are invalid. | +| json_key | Optional | N/A | The value of this register will be published to its pub_topic in JSON format. E.G. `{ key: value }` Registers with a json_key specified can share a pub_topic. All registers with shared pub_topics must have a json_key specified. In this way, multiple registers can be published to the same topic in a single JSON message. If any of the registers that share a pub_topic have the retain field set that will affect the published JSON message. Conflicting retain settings are invalid. The keys will be alphabetically sorted. | diff --git a/modbus4mqtt/modbus4mqtt.py b/modbus4mqtt/modbus4mqtt.py index 8eb5d80..2bf2495 100755 --- a/modbus4mqtt/modbus4mqtt.py +++ b/modbus4mqtt/modbus4mqtt.py @@ -116,7 +116,7 @@ def poll(self): self._mqtt_client.publish(self.prefix+register['pub_topic'], value, retain=retain) for topic, message in json_messages.items(): - m = json.dumps(message) + m = json.dumps(message, sort_keys=True) self._mqtt_client.publish(self.prefix+topic, m, retain=json_messages_retain[topic]) def _on_connect(self, client, userdata, flags, rc): diff --git a/tests/test_json_key.yaml b/tests/test_json_key.yaml new file mode 100644 index 0000000..ca9d324 --- /dev/null +++ b/tests/test_json_key.yaml @@ -0,0 +1,15 @@ +ip: 192.168.1.90 +registers: + - pub_topic: "publish" + json_key: "A" + address: 1 + retain: true + - pub_topic: "publish" + address: 2 + json_key: "B" + value_map: + on: 1 + off: 2 + - pub_topic: "publish2" + address: 3 + json_key: "A" \ No newline at end of file diff --git a/tests/test_mqtt.py b/tests/test_mqtt.py index 8e91649..da8d666 100644 --- a/tests/test_mqtt.py +++ b/tests/test_mqtt.py @@ -366,6 +366,26 @@ def test_address_offset(self): mock_mqtt().publish.assert_any_call(MQTT_TOPIC_PREFIX+'/publish', 2, retain=False) + def test_json_key(self): + # Validating the various json_key rules is among the responsibilities of test_register_validation() below. + with patch('paho.mqtt.client.Client') as mock_mqtt: + with patch('modbus4mqtt.modbus_interface.modbus_interface') as mock_modbus: + mock_modbus().connect.side_effect = self.connect_success + mock_modbus().get_value.side_effect = self.read_modbus_register + + m = modbus4mqtt.mqtt_interface('kroopit', 1885, 'brengis', 'pranto', './tests/test_json_key.yaml', MQTT_TOPIC_PREFIX) + m.connect() + + self.modbus_tables['holding'][0] = 0 + self.modbus_tables['holding'][1] = 1 + self.modbus_tables['holding'][2] = 2 + self.modbus_tables['holding'][3] = 3 + m.poll() + print(mock_mqtt.mock_calls) + + mock_mqtt().publish.assert_any_call(MQTT_TOPIC_PREFIX+'/publish2', '{"A": 3}', retain=False) + mock_mqtt().publish.assert_any_call(MQTT_TOPIC_PREFIX+'/publish', '{"A": 1, "B": "off"}', retain=True) + def test_register_validation(self): valids = [[ # Different json_keys for same topic {'address': 13049, 'json_key': 'a', 'pub_topic': 'ems/EMS_MODE'}, From cf2f45eafbdad5a64069b4a55863c5b55ee70c70 Mon Sep 17 00:00:00 2001 From: Travis Howse Date: Thu, 31 Dec 2020 23:19:51 +1000 Subject: [PATCH 06/11] Remove debug print. --- tests/test_mqtt.py | 1 - 1 file changed, 1 deletion(-) diff --git a/tests/test_mqtt.py b/tests/test_mqtt.py index da8d666..b17f412 100644 --- a/tests/test_mqtt.py +++ b/tests/test_mqtt.py @@ -381,7 +381,6 @@ def test_json_key(self): self.modbus_tables['holding'][2] = 2 self.modbus_tables['holding'][3] = 3 m.poll() - print(mock_mqtt.mock_calls) mock_mqtt().publish.assert_any_call(MQTT_TOPIC_PREFIX+'/publish2', '{"A": 3}', retain=False) mock_mqtt().publish.assert_any_call(MQTT_TOPIC_PREFIX+'/publish', '{"A": 1, "B": "off"}', retain=True) From 2f4bca46797f7bcd8f7f15cdce27b670567f83c5 Mon Sep 17 00:00:00 2001 From: Travis Howse Date: Thu, 31 Dec 2020 23:26:12 +1000 Subject: [PATCH 07/11] Add comment --- modbus4mqtt/modbus4mqtt.py | 1 + 1 file changed, 1 insertion(+) diff --git a/modbus4mqtt/modbus4mqtt.py b/modbus4mqtt/modbus4mqtt.py index 2bf2495..34c653f 100755 --- a/modbus4mqtt/modbus4mqtt.py +++ b/modbus4mqtt/modbus4mqtt.py @@ -115,6 +115,7 @@ def poll(self): retain = register.get('retain', False) self._mqtt_client.publish(self.prefix+register['pub_topic'], value, retain=retain) + # Transmit the queued JSON messages. for topic, message in json_messages.items(): m = json.dumps(message, sort_keys=True) self._mqtt_client.publish(self.prefix+topic, m, retain=json_messages_retain[topic]) From 002f59a2bdffd2581dd3073dabcf3278eaa42c1c Mon Sep 17 00:00:00 2001 From: Travis Howse Date: Fri, 1 Jan 2021 00:11:21 +1000 Subject: [PATCH 08/11] Add todo --- modbus4mqtt/modbus4mqtt.py | 1 + 1 file changed, 1 insertion(+) diff --git a/modbus4mqtt/modbus4mqtt.py b/modbus4mqtt/modbus4mqtt.py index 34c653f..33b66a3 100755 --- a/modbus4mqtt/modbus4mqtt.py +++ b/modbus4mqtt/modbus4mqtt.py @@ -140,6 +140,7 @@ def _on_subscribe(self, client, userdata, mid, granted_qos): def _on_message(self, client, userdata, msg): # print("got a message: {}: {}".format(msg.topic, msg.payload)) + # TODO Handle json_key writes. topic = msg.topic[len(self.prefix):] for register in [register for register in self.registers if 'set_topic' in register]: if topic != register['set_topic']: From 02d163d50c98efb8d22239b1ce31ff8440678dc8 Mon Sep 17 00:00:00 2001 From: Travis Howse Date: Fri, 1 Jan 2021 14:22:22 +1000 Subject: [PATCH 09/11] Add a check for an invalid combination of set_topic and json_key. --- modbus4mqtt/modbus4mqtt.py | 2 ++ tests/test_mqtt.py | 4 ++++ 2 files changed, 6 insertions(+) diff --git a/modbus4mqtt/modbus4mqtt.py b/modbus4mqtt/modbus4mqtt.py index 33b66a3..4d8030c 100755 --- a/modbus4mqtt/modbus4mqtt.py +++ b/modbus4mqtt/modbus4mqtt.py @@ -183,6 +183,8 @@ def _validate_registers(registers): duplicate_pub_topics.add(register['pub_topic']) duplicate_json_keys[register['pub_topic']] = [] retain_setting[register['pub_topic']] = set() + if 'json_key' in register and 'set_topic' in register: + raise ValueError("Bad YAML configuration. Register with set_topic '{}' has a json_key specified. This is invalid. See https://github.com/tjhowse/modbus4mqtt/issues/23 for details.".format(register['set_topic'])) all_pub_topics.add(register['pub_topic']) # Check that all registers with duplicate pub topics have json_keys diff --git a/tests/test_mqtt.py b/tests/test_mqtt.py index b17f412..05bed8a 100644 --- a/tests/test_mqtt.py +++ b/tests/test_mqtt.py @@ -414,6 +414,10 @@ def test_register_validation(self): [ # Retain specified twice and inconsistent {'address': 13050, 'json_key': 'A', 'pub_topic': 'ems/EMS_MODE', 'retain': True}, {'address': 13050, 'json_key': 'B', 'pub_topic': 'ems/EMS_MODE', 'retain': False} + ], + [ # set_topic and json_key both specified + {'address': 13050, 'json_key': 'A', 'pub_topic': 'ems/EMS_MODE', 'set_topic': 'ems/EMS_MODE/set', 'retain': True}, + {'address': 13050, 'json_key': 'B', 'pub_topic': 'ems/EMS_MODE', 'retain': False} ]] for valid in valids: try: From db624da310735a7722d61ceefb659121af218c90 Mon Sep 17 00:00:00 2001 From: Travis Howse Date: Fri, 1 Jan 2021 14:23:53 +1000 Subject: [PATCH 10/11] Add detail to readme. --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 30675b6..f2dd274 100644 --- a/README.md +++ b/README.md @@ -94,7 +94,7 @@ This section of the YAML lists all the modbus registers that you consider intere | ---------- | -------- | ------- | ----------- | | address | Required | N/A | The decimal address of the register to read from the device, starting at 0. Many modbus devices enumerate registers beginning at 1, so beware. | | pub_topic | Optional | N/A | This is the topic to which the value of this register will be published. | -| set_topic | Optional | N/A | Values published to this topic will be written to the Modbus device. | +| set_topic | Optional | N/A | Values published to this topic will be written to the Modbus device. Cannot yet be combined with json_key. See https://github.com/tjhowse/modbus4mqtt/issues/23 for details. | | retain | Optional | false | Controls whether the value of this register will be published with the retain bit set. | | pub_only_on_change | Optional | true | Controls whether this register will only be published if its value changed from the previous poll. | | table | Optional | holding | The Modbus table to read from the device. Must be 'holding' or 'input'. | From 012ab887ecfddd03badddb9c2cea743c0e035774 Mon Sep 17 00:00:00 2001 From: Travis Howse Date: Fri, 1 Jan 2021 14:27:18 +1000 Subject: [PATCH 11/11] Add comment link and bump version number. --- modbus4mqtt/modbus4mqtt.py | 2 +- modbus4mqtt/version.py | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/modbus4mqtt/modbus4mqtt.py b/modbus4mqtt/modbus4mqtt.py index 4d8030c..90bd423 100755 --- a/modbus4mqtt/modbus4mqtt.py +++ b/modbus4mqtt/modbus4mqtt.py @@ -140,7 +140,7 @@ def _on_subscribe(self, client, userdata, mid, granted_qos): def _on_message(self, client, userdata, msg): # print("got a message: {}: {}".format(msg.topic, msg.payload)) - # TODO Handle json_key writes. + # TODO Handle json_key writes. https://github.com/tjhowse/modbus4mqtt/issues/23 topic = msg.topic[len(self.prefix):] for register in [register for register in self.registers if 'set_topic' in register]: if topic != register['set_topic']: diff --git a/modbus4mqtt/version.py b/modbus4mqtt/version.py index 0c1df37..4d6a819 100644 --- a/modbus4mqtt/version.py +++ b/modbus4mqtt/version.py @@ -1 +1 @@ -version="0.3.5" +version="0.4.0"