MIHO004 / ENER034 without Miihome gateway

[megatron-uk]

Since I now own 13 (yes, not a typo) ENER010 power strips which will probably be running from just a handful of sockets I figured it would probably be a good idea to monitor what they're actually pulling from the wall (though I don't expect to have loads of things turned on at the same time).

How do I use (probably 2x) MIHO004 without a miihome gateway; is that even a supported configuration? Do I need to pull some sort of unique broadcast device id from them the same as we have to do with the older legacy remotes?

[whaleygeek]

Hi John,

The MIHO004 is a transmit only device, but it uses an FSK device that transmits data on a regular period using the OpenThings payload format. Inside that payload format is a unique deviceid.

To use without a MiHome gateway, run setup_tool.py and start the "mihome discovery mode". This will then repeatedly listen in FSK mode, and when your MIHO004 transmits an energy report (it chooses when to send this, I forget how often, perhaps every 30 seconds or 1 minute), a message will pop up on the console screen with the device details.

setup_tool.py when in discovery modem is configured to use the 'ask' joining behaviour, inside the code. This means you then get asked 'Do you want to register to device: %s?' where %s will be the deviceid unique to that device. Answer with a y and it will store details about that device inside the registry.kvs file, so that all the other examples programs and the other options on the setup_tool.py menu will recognise the device.

You can monitor readings now from setup_tool.py using the 'watch devices' or 'logging' options, to gain some confidence that the device is working.

Once you have each of your devices learnt to the registry.kvs file and showing correct readings, you may then want to use one of the example Python programs in the root of the src folder to build your application from. There are a number of different design pattern examples there, depending on what you wish to achieve.

But it should be possible to monitor a large number of devices.

Note that due to there being no advanced support in the Energenie protocols for collision detection, if two devices transmit a report simultaneously, you may get corrupted data. However if the monitors power up at different times, report times will be staggered.

There is a potential use-case I identified (power cut), where if your house consumer supply goes off and on due to a house-wide power cut, all plugs are likely to power up at the same time and report at the same interval from power on.

It is normal for device manufacturers to insert a random delay on power-up to prevent this situation, but I haven't yet been able to verify with the engineers at Energenie HQ whether these devices have a random power-up delay in their firmware or not.

Please do report back here as to how you get on with this.

Hope this helps.

[whaleygeek]

Also note, the drivers are written to allow interoperability between legacy devices and MiHome devices. The radio will be left in receive FSK (OpenThings) mode most of the time so that you don't miss energy reports from transmit-only devices. If you request a legacy device to switch on or off, the radio auto-reconfigures into OOK transmit mode, bursts a run of 10 messages to that switch to change it's state, then immediately reconfigures back to FSK receive mode.

So, it should be possible to build a system with both legacy and MiHome devices, and I did indeed have a large number of both devices present and working when I originally tested this code.

So, you have all the building blocks to build a simple system or a system of comparable complexity and flexibility of the MiHome gateway yourself in Python.

[megatron-uk]

Cheers David. That's pretty much exactly what I needed to know. My existing application (https://github.com/megatron-uk/sdlRFController) is designed to run macro actions when pressing a touchscreen button, so you can press, for example "Sega Megadrive", and the configuration of devices to buttons knows that to do that we have to do the following power-on actions:

• Monitor
• Scart switch
• Upscaler
• Mixer and amplifier
• Finally, turn on Megadrive

Here's what the actual definition of one of my touchscreen buttons looks like, in this case for Roland MT-32 MIDI synth module:

'text'	: "MT-32",
'image'	: "btn_mt32.bmp",
'poweron' : [
	{'remote' : 0xABCDE, 'socket' : 1, 'action' : "ON"},	# Turn this remote socket on
	{'tags' : ["speakers"], 'action' : "ON"},	# Turn on any device with a tag of 'speakers'
],
'poweroff' : [
	{'remote' : 0xABCDE, 'socket' : 1, 'action' : "OFF"},	# Turn this remote socket off
],
'remote'	: 0xABCDE,
'socket'	: 1,
'tags'	: ["midi"],

A mixture of how the button renders on-screen, and also the actions to take when pressed 'on', or 'off'. In 'on' mode it sends a power-on signal to remote/device id 0xABCDE, socket 1 (which is the MIDI module itself), but then also sends an 'on' signal to every other device tagged as 'speakers'... which includes an amp module, a mixer and a midi controller. All which have their own 'remote' and 'socket' identifiers, of course. The power on/off code unwinds to the individual device signals a tag is assigned to via the function setButtonPower()

It's a little bit like a Logitech Harmony remote, but for the dumb Energenie legacy remotes :)

What I want to be able to do is monitor the power use of all the ENER010 devices as I'm powering things on, so that I know what sort of headroom I've got with the mains supply. I think with what you've provided I can do that relatively straight forward and add a seperate Python thread listening for those broadcast messages on a periodic basis, logging them and graphing the results on the touchscreen.

[Achronite]

Or you could try my new node-red nodes node-red-contrib-energenie-ener314rt. I've just made the github repo accessible for beta. It allows for control & monitoring using node-red, and it should deal with your use case.

[megatron-uk]

Hi, after finally getting my radio module working I've been able to look in to this.

First thing I found was that there's an exception thrown if you're using Python 3 -

Traceback (most recent call last):
  File "setup_tool.py", line 345, in <module>
    setup_tool()
  File "setup_tool.py", line 338, in setup_tool
    handle_choice(MAIN_MENU, choice)
  File "setup_tool.py", line 310, in handle_choice
    menu[choice-1][1]()
  File "setup_tool.py", line 139, in do_mihome_discovery
    energenie.loop() # Allow receive processing
  File "/home/pi/pyenergenie/src/energenie/__init__.py", line 83, in loop
    registry.fsk_router.incoming_message(address, msg)
  File "/home/pi/pyenergenie/src/energenie/Registry.py", line 206, in incoming_message
    self.handle_unknown(address, message)
  File "/home/pi/pyenergenie/src/energenie/Registry.py", line 218, in handle_unknown
    self.unknown_cb(address, message)
  File "/home/pi/pyenergenie/src/energenie/Registry.py", line 281, in unknown_device
    y = self.ask_fn(address, message)
  File "/home/pi/pyenergenie/src/energenie/__init__.py", line 129, in ask
    y = raw_input(MSG)
NameError: name 'raw_input' is not defined

... so it looks like we may need to change raw_input() to input() (plus associated key value to character mapping) to make it Python 3 safe. This is in __init__.py used by the setup tool.

Using Python 2 it runs, and eventually after 30 seconds or so I see a response from my MIHO004:

No route to an object, for device:(4, 1, 3966)
{'header': {'sensorid': 3966, 'productid': 1, 'encryptPIP': 26315, 'mfrid': 4}, 'type': 'OK', 'rxtimestamp': 1556264709.344994, 'recs': [{'paramunit': 'W', 'typeid': 128, 'valuebytes': [0, 0], 'value': 0, 'length': 2, 'wr': False, 'paramname': 'REAL_POWER', 'paramid': 112}, {'paramunit': 'VAR', 'typeid': 128, 'valuebytes': [255, 249], 'value': -7, 'length': 2, 'wr': False, 'paramname': 'REACTIVE_POWER', 'paramid': 113}, {'paramunit': 'V', 'typeid': 0, 'valuebytes': [244], 'value': 244, 'length': 1, 'wr': False, 'paramname': 'VOLTAGE', 'paramid': 118}, {'paramunit': 'Hz', 'typeid': 32, 'valuebytes': [49, 243], 'value': 49.94921875, 'length': 2, 'wr': False, 'paramname': 'FREQUENCY', 'paramid': 102}]}
Do you want to register to device: (4, 1, 3966)?

Selecting y added the following entry to registry.kvs:

ADD auto_0x1_0xf7e
type=MIHO004
device_id=3966

In my test code I am trying:

energenie.init()
mon1 = energenie.Devices.MIHO004(device_id = 3966)

while True:
    energenie.loop()
    time.sleep(1)
    print("")
    print("==========================")
    r = mon1.get_readings()
    print(r.voltage)
    print("")

This eventually, after a few loops of listening, shows the following text:

No route to an object, for device:(4, 1, 3966)
Unknown address: (4, 1, 3966)

If I run mihome_energy_monitor.py and use the stored device entry I see the following after a few seconds of listening:

Incoming from (4, 1, 3966)
No route to an object, for device:(4, 1, 3966)
Unknown address: (4, 1, 3966)

[whaleygeek]

Regarding raw_input yes that is a bug, we should fix it for sure.

Regarding no-route, there was a PR I merged a few weeks ago that uses a different way of calculating the path to the registry.kvs file, it's possible that depending where your cwd is when you invoke the script does or does not have a bearing on which file is actually used.

Follow through the code here to see if it is loading your real registry when init() is called, I have a hunch it might for some reason now be not finding your registry and creating an in memory default blank registry, and thus when the message arrives it doesn't know which object (via fsk_router) to pass it to.

One of the perils of me not having a live setup is that I currently rely on PR authors to test their changes, which is why I am often slow to merge PR's.

Let me know what you discover. There are some print() statements in that referenced code, do they generate output in your case?

[megatron-uk]

Ok, so it seems to try and load registry.kvs from the current working directory, which, in my case, is not the pyenergenie/src folder (I have the energenie subfolder symlinked in to my own project folder).

If I symlink registry.kvs in to my project folder I can see the values being parsed on energenie.init():

loaded registry from file
REGISTERED DEVICES:
  auto_0x1_0xf7e -> MIHO004(0xf7e)
ROUTES:
MIHO004(0xf7e)

The problem with the error No route to an object, for device:(4, 1, 3966) seems to stem from Registry.routes being empty. If I insert debug code in Registry.incoming_message():

def incoming_message(self, address, message):
    print(self.routes)
    print(address)
    print(message)
    ....

I see the following output:

{}
(4, 1, 3966)
message: {'type': 'OK', 'recs': [{'paramid': 112, 'value': 0, 'length': 2, 'paramname': 'REAL_POWER', 'typeid': 128, 'paramunit': 'W', 'wr': False, 'valuebytes': [0, 0]}, {'paramid': 113, 'value': -7, 'length': 2, 'paramname': 'REACTIVE_POWER', 'typeid': 128, 'paramunit': 'VAR', 'wr': False, 'valuebytes': [255, 249]}, {'paramid': 118, 'value': 241, 'length': 1, 'paramname': 'VOLTAGE', 'typeid': 0, 'paramunit': 'V', 'wr': False, 'valuebytes': [241]}, {'paramid': 102, 'value': 49.94921875, 'length': 2, 'paramname': 'FREQUENCY', 'typeid': 32, 'paramunit': 'Hz', 'wr': False, 'valuebytes': [49, 243]}], 'header': {'sensorid': 3966, 'productid': 1, 'mfrid': 4, 'encryptPIP': 3711}, 'rxtimestamp': 1556283460.776451}

I would have thought that the init() call should have added an entry to the registry for 3966, but it appears empty.

[megatron-uk]

So I realised that the code in mihome_energy_monitor.py doesn't seem to work as-is anymore, looking back at #75 I now see that to get power readings from the transmit-only devices, you need to specifically 'get' the device from the registry:

registry.kvs

ADD mon1
device_id=3966
type=MIHO004

test.py

#!/usr/bin/env python3

import time
import energenie

APP_DELAY = 1

energenie.init()
mon1 = energenie.registry.get('mon1')

print("======")

while True:
    energenie.loop()
    time.sleep(1)
    print("")
    print("==========================")
    r = mon1.get_readings()
    print(r.voltage)

For some reason this gives me different behaviour of the registry routes data structure, here's the output now:

loaded registry from file
REGISTERED DEVICES:
  mon1 -> MIHO004(0xf7e)
ROUTES:
Adding rx route for transmit enabled device MIHO004(0xf7e)

And I see the mains line voltage being printed!

[whaleygeek]

Ok, looks like I need to re-test mihome_energy_monitor.py and fix or update appropriately, thanks.