Energy monitoring not refreshing

[AdmiralStipe]

Energy monitoring sometimes refreshes the very second the switch toggles, but sometimes it doesn't refresh at all. Heater in the smartplug is running, but the current and current consumption show 0, or vice-versa (heater is off, but the consumption shows 10A / 2300W). Restarting HA may or may not help resolving it (cca 50% success rate).

Additional issue for me: status changes (from physical buttons) are not instant, instead they take from 10 to 30 seconds (as before the last release).

System
Operating system
Raspbian GNU/Linux 10 (buster) on Raspberry PI 4B
Docker version
19.03.13

Supervisor version: 247
Host system version: Home Assistant 0.116.3, running on Raspberry Pi 4B in Docker (Supervised install)

[postlund]

Make sure you are running version 3.0.0 of this component. All versions prior to that polls the device every 30s (I think it's every 60s for 1.2.0 and earlier). That makes it quite unreliable.

[AdmiralStipe]

I AM using the latest version, but it doesn't solve it... I was hoping this one would solve it, as it didn't work well before :)

[postlund]

Can you enable some logs and provide us?

[AdmiralStipe]

Which logs do you have in mind, HA general logs or plugin specific? If plugin specific, how do I enable them?

[postlund]

The ones for localtuya I would say. Should be something like:

logger:
  default: warning
  logs:
    custom_components.localtuya: debug

[AdmiralStipe]

Done... I kept it running for a few minutes and tried to switch on and off some of the plugs.
Without changing anything, the detection of switch now occured almost instantly (regardless of software switch or physical button), but the consumption still does not work (at least most of cases). What I have found out furthermore is, that the consumption status changes the very second I open the Consumption page on my phone in original TuyaSmart app - looks like the app somehow forces or pushes the consumption status change.
I'm not sure if you will be able to see that in logs, but one plug also changes the consumption status with the change of On/Off state, and the other two plugs did not (well, at least until I opened the TuyaSmart app on my phone).
Just to add, all of my plugs are the same model (named simply Robaxo or Robas??? but using TuyaSmart and/or Smartlife android app).

Shall I post the log here (privacy concerns because of IP number, MAC adresses, ID's,...) or shall I send it to some email address?

[postlund]

You can send the logs to me by email, probably better from privacy perspective 👍

[AdmiralStipe]

Just to make sure... did you receive my mail with the log and is the log long enough? I sent it to the mail address from your profile...

[postlund]

@AdmiralStipe Ah, yes, sorry for not givning any feedback. I have received it and will have a look at it ASAP.

[AdmiralStipe]

I sent you an e-mail with more detailed description, as I'm afraid, I didn't explain my issue properly.
Let me continue with my thinking here:
If I understand the log file correctly, the Tuya device reports status change by itself and localtuya only interpretes it. So, I see, that all my Tuya devices report change of state (On/Off) with the change of dps 1 (True/False) correctly and on time. However, when the dps 1 changes, I expect, that also dps 18 and dps 19 change (the current starts/stops flowing and the consumption of energy starts/stops). But in the device's status update there is only dps 1 and the time of change, but the dps 18 and 19 are missing! They are sometimes (but rarely) reported a few seconds later (this time only dps 18, 19 and 20, there is no dps1, as it has been reported before).
However, dps 18 and 19 are reported the very same moment as I open the TuyaSmart app on my phone and go to Consumption page. From that I understand, that the Tuya app knows how to demand or force that report.
So, would it be possible to implement a toggleable setting (obviously only my plugs need it :) ), which would request status of dps 18 and 19 (and possibly 20) from the device immediately whenever the dps 1 changes?

[AdmiralStipe]

So, If you don't mind I would like to try to fiddle with the code myself a little bit (I'm no programmer, but I can do some light customisation stuff). I would like to try to force my smartplugs to send full status report (all available dps) whenever the switch status (dps 1) changes.
Can you give me two hints, so I know where to start:

1. which part of plugin (which file and roughly which line) takes care of inital status query (I understood that at startup the plugin requests full status report for all dps from the smartplugs)?
2. which part of plugin (again, which file and which lines) interpretes the received dps data?
   Thx in advance.

[nqkdev]

I got the same issue. Any progress on this? :)

[postlund]

I will try to get back on this horse soon, but I have a few other matters I need to attend first. @AdmiralStipe I will try to give a short explanation once I'm done with what I'm working on.

[MarlevPy]

I got the same issue as well.
While I wait for the pros I can appreciate the good work you're doing with this integration!
Big thanks and good job!!

[michel130]

I have exactly the same issue! Hope this gets resolved soon. Keep up the good work :)

[postlund]

If anyone has any idea on how to move forward, feel free to try them out. I don't have any clues at the moment😕

[postlund]

@AdmiralStipe OK, so, fiddle away and see what happens 🙂

The initial status update (where an entity requests the state of all datapoints) is done here:

localtuya/custom_components/localtuya/common.py

Line 150 in f3cfde0

status = await self._interface.status()

Whenever a status update is received from the device (by pytuya), this callback method is called:

localtuya/custom_components/localtuya/common.py

Line 186 in f3cfde0

def status_updated(self, status):

The status variable passed to the method contains the value of a known datapoints so far. So you will not be able to know if "1" changed at this point as the value of individual datapoints are not exposed like that.

It's also worth knowing that all this logic is handled by an instance of TuyaDevice. Each physical device has one instance of that class used to communicate with the device. Each entity has access to it in case it needs to send something and it is also responsible for telling each entity when datapoints changes to they can reflect upon that. The status_updated method is called in each entity class associated with a TuyaDevice whenever one or more datapoint change.

localtuya/custom_components/localtuya/common.py

Line 296 in f3cfde0

def status_updated(self):

It's of course possible to hook directly into pytuya and fiddle around, but the external interface is what recommend that you use. My last two cents here are that I don't think it's gonna work to request all datapoints at this point as I believe that it will just respond with whatever has already been reported. But who knows.

[AdmiralStipe]

Oh, I see... I thought I will be able to detect whether only dps1 or also other datapoints are included in a variable....
Well, I'm quite stubborn so I'll try to figure out something anyway. I will very likely fail (as I said, I'm no programmer), but I don't wanna give up without a fight :D

[postlund]

It's possible to do that from within pytuya, but I changed it to keep a cache of every DP and just pass that along to the listener because of some problems we had. Can't remember which issue number it was related to now. But that's how it works currently.

[makkie2002]

I think the issue is that the device does not automatically report changes to any of the “measurements” like voltage, current and power. This is logic as these values change very often and would otherwise flood the communication channel. Probably we need to actively poll these values from the device (e.g. every 10s). No idea how to do it though. I however know that other devices in Home Assistant allow to define a polling interval in their configuration.

[AdmiralStipe]

I tried a few different things and I failed, as expected :).
This level of coding is a total overkill for my level and all I managed was to completely kill the code a couple of times (all Tuya plugs unavailable and the localtuya could not be reloaded :) ).
So I guess @makkie2002 is right - to make this work , the code should actively poll the selected additional values 18, 19 and 20 (of course only if the plug is On) on a predefined interval, but at the same time rely on async for immediate status change reporting (for dps1, it's working great and really instant since async implementation) .
Unfortunately I'm not able to write this in a code.
@postlund, is ti even possible to have both ways (async and selective polling) active at the same time?

[postlund]

@AdmiralStipe It's possible, I made a prototype in #117. When adding a sensor, you can specify a poll interval in seconds. At every interval, the current value of the datapoint used by the sensor is requested by setting it to None. This is a small quirk in the firmware, as nothing will happen with the datapoint but it will trigger a status update containing only the value of the specified datapoint.

Feel free to try this out, maybe it can help us getting further (it's not a complete solution, just a PoC). You will need to remove and re-add and previous devices having a sensor entity for this to work.

[AdmiralStipe]

Tried it with great hope and expectations :), but unfortunately my report is not so cheer.
Everything behaved completely the same as before. I followed your instructions and completely removed previous devices, added them once again (through config, not yaml), added sensors, added poll time of 10 seconds, restarted and started testing. Sometimes the current and power changes have been reported, sometimes (mostly) not. Switch status is again always reported correctly and on time. I'm starting to think that I might just have crappy plugs which behave badly. I use these.
Attached you can check the latest log (timed simulation). From privacy reasons I changed all seven plugs id's from their original to 11111111111111111111 through 77777777777777777777.
Maybe just a few additional words of comment to it.
I started the simulation (switched the plug on) at 00:50:00. The heater started heating immediately, but as you can see, the current and power reported remain 0 through the following 10 sec polls. At 00:51:00 I opened TuyaSmart app on my phone for this particular plug and that forced report of the correct current and power (because of the app, not the poll or self report).
At 00:52:00 I switched the plug off (so also current and power should be 0), but the current and the power remained on previous values through following polls. At 00:52:00 I again opened TuyaSmart app and again the changes of current and power (both 0) have been forced reported.
So, it seems to me, that my plugs are only obeying the commands from original Android app and not localtuya :(.
I'm getting more and more attracted to flashing Tasmota...

log1.txt

[postlund]

I was kinda expecting that result. Found this issue which sounds exactly what you are experiencing:

codetheweb/tuyapi#272

However, it was closed with the "solution" to kill the app and never open it again. Not sure that makes any difference... Try to put some additional effort into sniffing the app traffic with be a good step. Another thing that I would personally recommend trying is to set up a developer account at tuya and re-pair the device that way (that's how I do it). This will prevent you from using smartlife though, but I wouldn't be surprised if it worked.

[AdmiralStipe]

I agree, sniffing traffic and finding out what exactly happens when the app demands status update would be good - unfortunately I don't know how to perform it effectively... I've heard of Wireshark and similar stuff, but I don't have an idea how exactly to set everything up and furthermore, what and how to capture then. If I only capture some encrypted garbage, it will be of no help for sure.
I think I will try your suggestion with developer account for now (maybe just for one plug for starters, since I'm not really that keen on losing app control over plugs despite controling them mostly through HA). Thx for the tip :).

[AdmiralStipe]

@postlund, by the way, if I open developer account and re-pair the devices (and you said I won't be able to use the app anymore) - how do I get the localkey then? Now I got it from the android app, so are there other ways to get it as "developer" - maybe directly in the develoepr profile somewhere?

[postlund]

@AdmiralStipe I followed this and used tuya-cli to register my devices:

https://github.com/codetheweb/tuyapi/blob/master/docs/SETUP.md

[AdmiralStipe]

OK, great, but I have another question - after following the guide and registering as developer, already in Step 1 I only have the possibility to use Cloud develompent as free trial (until July 2021). What happens after that, will everything continue to work or will I be forced to pay for Premium to use these functions?

[postlund]

I think that everything you need to register new devices will still work after that. Seems to work fine for me.

[andytuinman3]

Anyone tried the new tuya V2 yet? It should have local support real soon. But I don't know if this will fix it. Going to try it tonight to see if it works.

[AdmiralStipe]

Anyone tried the new tuya V2 yet? It should have local support real soon. But I don't know if this will fix it. Going to try it tonight to see if it works.

Didn't try it yet, as I really don't want to edit all my automations and everything with new plug names, however I'm really interested in knowing, if energy monitoring works for my plugs (Robaxo) and how big is status-change delay when switching the plugs. In any case, I will remain with localtuya for sure until official Tuya supports local mode. Then it depends... if energy reporting on demand still won't be supported here (it now seems that it won't), I will most likely switch.
If you will try it, please report your findings.

[rospogrigio]

What is this Tuya V2 you are talking about?

[JasperS2307]

@rospogrigio

What is this Tuya V2 you are talking about?

https://github.com/tuya/tuya-home-assistant

[andytuinman3]

Too bad. Doesn't work. It's even worse now. Power measurements don't change even if I have the smart life app open. And if I turn it off via smart life it doesn't change on home assistant. Already pushed out an Issue for this.

[andytuinman3]

Anyone tried the new tuya V2 yet? It should have local support real soon. But I don't know if this will fix it. Going to try it tonight to see if it works.

Didn't try it yet, as I really don't want to edit all my automations and everything with new plug names, however I'm really interested in knowing, if energy monitoring works for my plugs (Robaxo) and how big is status-change delay when switching the plugs. In any case, I will remain with localtuya for sure until official Tuya supports local mode. Then it depends... if energy reporting on demand still won't be supported here (it now seems that it won't), I will most likely switch.
If you will try it, please report your findings.

Ok so I got a tip to make it work. And for me the Tuya V2 works like a charm. Every 30 seconds the power consumption is refreshed. No way to edit it but 30 seconds is fine for me. Now we just have to wait until they make local Tuya available

[AdmiralStipe]

Ok so I got a tip to make it work. And for me the Tuya V2 works like a charm. Every 30 seconds the power consumption is refreshed. No way to edit it but 30 seconds is fine for me. Now we just have to wait until they make local Tuya available

So, would you mind sharing this tip you got? I'm keen on using it, as 30secs refresh is enough for me too, no need to have it more frequently, but unfortunately all my devices become unavailable when using Tuya v2...

[andytuinman3]

First of all. You can use local tuya and tuya v2 next to each other. So testing should be easy.
Second make sure that the API "Device status notification" is enabled. That made it work for me.
Go to iot.tuya.com go to "cloud" and click on your project. Then select API at the top. Select "go to authorize" and select "Device status notification" from the list. I had to scroll through it since searching didnt work.

Now restart HA and the tuya devices should refresh every 30 seconds.

[AdmiralStipe]

First of all. You can use local tuya and tuya v2 next to each other. So testing should be easy.
Second make sure that the API "Device status notification" is enabled. That made it work for me.
Go to iot.tuya.com go to "cloud" and click on your project. Then select API at the top. Select "go to authorize" and select "Device status notification" from the list. I had to scroll through it since searching didnt work.

Now restart HA and the tuya devices should refresh every 30 seconds.

Great, thx a lot, will try it as soon as I come home (on vacation atm).
One more question, if you switch the plug with a button on it (not through HA), does the state in HA change immediately or at least in a timely manner, or does it still take up to several minutes to update the state?

[andytuinman3]

I tested it and switched off the TV. When I went to integrations/tuyav2/TV plug it was off. So not more then 10 seconds

[AdmiralStipe]

I tested it and switched off the TV. When I went to integrations/tuyav2/TV plug it was off. So not more then 10 seconds

Perfect, can't wait to try it... if this will work, then we only have to wait for them to enable local mode and then we have the native improved Localtuya 😁

[solideus]

First of all. You can use local tuya and tuya v2 next to each other. So testing should be easy.
Second make sure that the API "Device status notification" is enabled. That made it work for me.
Go to iot.tuya.com go to "cloud" and click on your project. Then select API at the top. Select "go to authorize" and select "Device status notification" from the list. I had to scroll through it since searching didnt work.

Now restart HA and the tuya devices should refresh every 30 seconds.

Works!!!

Thank you very much for the help. Tested for 24 hours with a washing machine and home assistant records the variations without the need for the application from the mobile. Tested with LocalTuya.

However, I do not have functions such as daily consumption, while with tasmota I do. So I'm waiting for my ESP8266s to arrive to change the WB2S chip that it brings.

[michaelzoellner]

It worked... unfortunately, only for a couple of hours and only for one of two devices (of the same maker and type). The other device still updated at the 10 min frequency. I disconnected and reconnected it from the Smart Life app, no luck. Since a little while, the first device went back to the known behavior as well. Updates only every 10 mins (or if the Smart Life app is open). Any suggestions?

[AdmiralStipe]

It worked... unfortunately, only for a couple of hours and only for one of two devices (of the same maker and type). The other device still updated at the 10 min frequency. I disconnected and reconnected it from the Smart Life app, no luck. Since a little while, the first device went back to the known behavior as well. Updates only every 10 mins (or if the Smart Life app is open). Any suggestions?

Same here.... only it didn't last couple of hours, but minutes instead :(.
So no luck with offical integration and now even this repo seems to be more or less dormant (last PR one month ago)... :(
@jasonacox, any hope on waiting for realisation of the function from your last post?

Or probably better to toss these shitty Chinese plugs to garbage and get some proper (not Tuya) ones...

[lishan89uc]

@AdmiralStipe Sorry to hear you are having so much trouble but i want to see if I summarize your situation. You

• added the plugs to iot.tuya.com manually without the app
• Never disabled the internet of the plugs
• later on you installed tuya 2.0 to try to get it to work(monitoring power)
• So far the best you have been able to do is get it working for a few minutes

please correct me i am just trying to make the situation clearer to diagnose the issue. I am not a programmer either. I however got 2 plugs working very reliably.Bn-link plugs
After adding the plugs to localtuya the power monitoring was horrible. After a few localtuya updates it started to work.
Things that I have notice:

• All non power monitoring plugs works better with the internet blocked. (I use a ASUS GT-ac5300 so blocking them was a flip of a switch.)
• The power monitoring dp is unreliable. So i use the current one and setup a sensor template. P=I V
• Turning off the internet of the power monitoring switches will still allow them to turn on and off however they will no longer update the current after some time.
• opening the app on my iPhone while HA is restarting causes me a lot of trouble reconnecting to the devices.(disabling the internet of the devices help just re-enable the monitoring plugs after it connects)
• disabling the internet of the plugs eliminates the unavailable status in the logs. (very few at least)

I have 3 brands of "tuya" devices Feit dimmer, BNLink power monitoring plugs, Feit plugs(most unstable maybe wifi issue with aimesh). As of 2021.7.3 all of them are working very reliably. With exception of the monitoring plugs I disable all their internet.
If you are going to buy non tuya switches I highly recommend you stay away from WEMO ones. I have had good experience with shelly relay switches but they are not really plugs... I wish you the best of luck. Feel free to ask if you have any questions for me I want to help as much as I can.

Edit I remember a few extra things:

• If you or any family member open the tuya app even after the switches are connected via localtuya. they can still disconnect or become unresponsive.
• if the HA integration is active at the same time as localtuya it sometimes cause the same issue as above. I haven't tried tuya 2.0 so not sure if that would cause the same issue
• if you have a dimmer or some kind of 3 way wall switch. The only way to get state update when turning on and off with the non-smart switch is with localtuya. not sure why it doesn't register with tuya server even the app doesn't update.

[AdmiralStipe]

@AdmiralStipe Sorry to hear you are having so much trouble but i want to see if I summarize your situation. You

• added the plugs to iot.tuya.com manually without the app
• Never disabled the internet of the plugs
• later on you installed tuya 2.0 to try to get it to work(monitoring power)
• So far the best you have been able to do is get it working for a few minutes

please correct me i am just trying to make the situation clearer to diagnose the issue.

The answer to all questions is Yes :). As already stated, obviously my plugs are shitty and I'm getting sick of trying, experimenting and waiting for partial solutions and it it will be better (at least for my mind peace :) ) to just replace them with something better.
I will keep this issue opened however, as maybe someone else will benefit from it.

[zeminator]

Sad to notice that at least my own smartplugs with energy consuption will not update at all without Tuya App. This is not only LocalTuya issue. I did remove one of my smartplugs from LocalTuya and followed the 18,19,20 values with tuya-cli and noticed that values will not update at all (load or without) in the smartplug.

Only when smartplug was connected to internet and I was using Tuya App, those values was changing in DPS @ smartplug.

[jeremysherriff]

I've recently purchased some Kogan SmarterHome Plugs with Energy Monitoring (same as a few in this thread). Is there any need for diagnostics like Wireshark, or do we know what the commands being sent are and are waiting for coding and PRs?

My devices are NOT local-only, and my Android device is not rooted so my ability to get Wireshark captures may be limited, but I'm an IT engineer (infrastructure, not coding) so I'm not inexperienced.

[joe-sydney]

I have the Kogan SmarterHome Smart Plug With Energy Meter & 5V 2.4A USB Ports and have the energy monitoring data working with both localtuya and Tuya v2. With localtuya, there is the patch to apply UpdateDPS at each heartbeat interval and with Tuya v2, I used the tinytuya script that sends the UpdateDPS command at a configurable interval (set in the script).

[jeremysherriff]

PR #491 does not appear to fix my issue for getting power/energy data with the same "Kogan SmarterHome Smart Plug With Energy Meter & 5V 2.4A USB Ports" devices. DP 19 current_consumption remains unchanged after 20 minutes. No idea why it would work for some people but not others.

[joe-sydney]

I used PR#549. There are 2 files involved.

[jeremysherriff]

Working perfectly, thanks @joe-sydney

[pimw1]

I used PR#549. There are 2 files involved.

This is working awesome! I've put the details here: #549

[AdmiralStipe]

Solved with #549 :).