Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DHT sensors protocol revision and refactoring #7468

Closed

Conversation

KrzysztofPrzygoda
Copy link

@KrzysztofPrzygoda KrzysztofPrzygoda commented Jan 9, 2020

Description:

Related issue (if applicable): fixes #5619
IMO, the root-cause of the problem were both: 1) DHT22 bad timings and 2) noInterrupts() that was switched on too late (should be on at the very beginning of the protocol). My AM2302 (with SONOFF Basic R2 on GPIO3 Serial In) has been working stable for 3 days now. Previously it hung (null readings) every few hours with power cycle as a remedy. I did not any hardware adjustments like better coords, connections etc. Just code changes and hanging is gone.

Changes:

  1. Only DHT22 protocol timings have been changed regarding data sheet.
  2. noInterrupts() command moved at the very begining of the protocol to ensure reading stability.
  3. One switch statement created to embrace protocols for all sensors in one place for better code readability (protocols revision gathered below and placed in code comments).

Revision based on:

DHT21, DHT22, AM2301, AM2302
Specs:
https://cdn-shop.adafruit.com/datasheets/Digital+humidity+and+temperature+sensor+AM2302.pdf

Protocol:

  1. MCU PULLS LOW data bus for 1 to 10ms to activate sensor
  2. MCU PULLS UP data bus for 20-40us to ask sensor for response
  3. SENSOR PULLS LOW data bus for 80us as a response
  4. SENSOR PULLS UP data bus for 80us for data sending preparation
  5. SENSOR starts sending data (LOW 50us then HIGH 26-28us for "0" or 70us for "1")

DHT11
Specs:
https://www.mouser.com/datasheet/2/758/DHT11-Technical-Data-Sheet-Translated-Version-1143054.pdf

Protocol:

  1. MCU PULLS LOW data bus for at least 18ms to activate sensor
  2. MCU PULLS UP data bus for 20-40us to ask sensor for response
  3. SENSOR PULLS LOW data bus for 80us as a response
  4. SENSOR PULLS UP data bus for 80us for data sending preparation
  5. SENSOR starts sending data (LOW 50us then HIGH 26-28us for "0" or 70 us for "1")

SI7021
Specs:
https://www.silabs.com/documents/public/data-sheets/Si7021-A20.pdf

Protocol:
Reverse-engineered on #735 (comment):

  1. MCU PULLS LOW data bus for at 500us to activate sensor
  2. MCU PULLS UP data bus for ~40us to ask sensor for response
  3. SENSOR starts sending data (LOW 40us then HIGH ~25us for "0" or ~75us for "1")
  4. SENSOR sends "1" start bit as a response
  5. SENSOR sends 16 bits (2 bytes) of a humidity with one decimal (i.e. 35.6% is sent as 356)
  6. SENSOR sends 16 bits (2 bytes) of a temperature with one decimal (i.e. 23.4C is sent as 234)
  7. SENSOR sends 8 bits (1 byte) checksum of 4 data bytes

Checklist:

  • The pull request is done against the latest dev branch
  • Only relevant files were touched
  • Only one feature/fix was added per PR.
  • The code change is tested and works on core 2.6.1
  • The code change pass travis tests. Your PR cannot be merged unless tests pass
  • I accept the CLA.

1. Only DHT22 protocol has been changed according to the issue arendst#5619.
2. noInterrupts() command moved to the begining of the reading protocol.
3. One switch created to embrace protocols for all sensors in one place for better code readability (protocols revision gathered below and placed in code comments).
4. Tested on AM2302. Usually issue arendst#5619 occured after several hours and needed power cycle reboot. Currently works without a problem for over 2 days.

DHT21, DHT22, AM2301, AM2302
Specs:
https://cdn-shop.adafruit.com/datasheets/Digital+humidity+and+temperature+sensor+AM2302.pdf

Protocol:
1. MCU PULLS LOW data bus for 1 to 10ms to activate sensor
2. MCU PULLS UP data bus for 20-40us to ask sensor for response
3. SENSOR PULLS LOW data bus for 80us as a response
4. SENSOR PULLS UP data bus for 80us for data sending preparation
5. SENSOR starts sending data (LOW 50us then HIGH 26-28us for "0" or 70us for "1")

DHT11
Specs:
https://www.mouser.com/datasheet/2/758/DHT11-Technical-Data-Sheet-Translated-Version-1143054.pdf

Protocol:
1. MCU PULLS LOW data bus for at least 18ms to activate sensor
2. MCU PULLS UP data bus for 20-40us to ask sensor for response
3. SENSOR PULLS LOW data bus for 80us as a response
4. SENSOR PULLS UP data bus for 80us for data sending preparation
5. SENSOR starts sending data (LOW 50us then HIGH 26-28us for "0" or 70 us for "1")

SI7021
Specs:
https://www.silabs.com/documents/public/data-sheets/Si7021-A20.pdf

Protocol:
Reverse-engineered on arendst#735 (comment):
1. MCU PULLS LOW data bus for at 500us to activate sensor
2. MCU PULLS UP data bus for ~40us to ask sensor for response
3. SENSOR starts sending data (LOW 40us then HIGH ~25us for "0" or ~75us for "1")
4. SENSOR sends "1" start bit as a response
5. SENSOR sends 16 bits (2 bytes) of a humidity with one decimal (i.e. 35.6% is sent as 356)
6. SENSOR sends 16 bits (2 bytes) of a temperature with one decimal (i.e. 23.4C is sent as 234)
7. SENSOR sends 8 bits (1 byte) checksum of 4 data bytes
@KrzysztofPrzygoda
Copy link
Author

Closing to restart Travis.

@ildena
Copy link

ildena commented Jan 10, 2020

An offset for humidity could be useful too... :-)

@KrzysztofPrzygoda
Copy link
Author

@ildena You mean like this in #2872?
This PR is focused on stable readings only.

@Jason2866
Copy link
Collaborator

Jason2866 commented Jan 11, 2020

@ildena why offset? If it measures wrong a offset does not help. Errors are not linear.
Only solution is to replace the sensor. If you need better accuracy take Bosch sensors.
BMP280 or BME280.

@ascillato2
Copy link
Collaborator

I would like to test further your changes due to I have several of these sensors working with Tasmota for more than 2 years and never have this issue. I'm not against this optimization, I'm in favour of it, I just want not to broke other people's setup since in reported issues the problem with these sensors were very sporadic and very low in last 2 years.

@arendst
Copy link
Owner

arendst commented Jan 11, 2020

An in addition, when do people finally start dropping these devices for way better ones.

@KrzysztofPrzygoda
Copy link
Author

KrzysztofPrzygoda commented Jan 11, 2020

Regarding code changes.
I did only rational changes like:

  1. Following strictly sensors specs in timings. Previous timings had no justification in the specifications.
  2. Turn off interruption to embrace entire communication with sensors. Previously sensors initiation wasn’t secured against interrupts.

So, there are no fancy changes than rational only.

Regarding variability in sensor performance.
@ascillato2 IMO, the differences in the same sensor type performance is due to production variability and you're lucky ;) There could be batches differences as well as “improvements” introduced later into production affecting newer batches. So, some batches could be more tolerant against specs than the others. Moreover, we are dealing with high frequencies, so there are many different factors that can lessen sensors tolerance (coords, connectors, soldering, noisy environment). The more code follow the specifications the less susceptible to variability it is. Any deviation only makes the situation worse.

@KrzysztofPrzygoda
Copy link
Author

KrzysztofPrzygoda commented Jan 11, 2020

An in addition, when do people finally start dropping these devices for way better ones.

@arendst They will definitely do that when you stop supporting bad ones ;)

@KrzysztofPrzygoda
Copy link
Author

KrzysztofPrzygoda commented Jan 11, 2020

@ildena As stated in #2872, you can calibrate sensor using Commands: Add, Sub, Mult and Scale in Rules. Probably, each sensor entity needs its own calibration and it cannot be permanently entered into the code. There is also pointless to build GUI for that while there are Rules available. More about accuracy: http://www.kandrsmith.org/RJS/Misc/Hygrometers/calib_many.html

@KrzysztofPrzygoda
Copy link
Author

KrzysztofPrzygoda commented Jan 11, 2020

I added extra explanation to PR description:

I did not any hardware adjustments like: better coords, connections etc. Just code changes and hanging is gone.

PS. The 4th day of stable system operation without rebooting or power cycle passes.

@ildena
Copy link

ildena commented Jan 11, 2020

ok, I surrender... too many shooters :-) :-)
I used the DHT11 because, though it's not that accurate, it's small and very fast to sense the hum and temp changes that is why I need that. I know that there are other sensors more reliable but for my project DHT11 hit the goal.
I was wondering why in tasmota there is a tempoffset but not a similar command for humidity. I know the rules metod but I wanted to use the original tele of tasmota instead of writing another one... no problem!! :-)

@ascillato
Copy link
Contributor

There are some review comments in the code. Please, check them. I think that in the mobile version can not easily be seen. Thanks.

@KrzysztofPrzygoda
Copy link
Author

@ascillato If you refer to this PR, I can't see them on the computer either.

@KrzysztofPrzygoda
Copy link
Author

ok, I surrender... too many shooters :-) :-)

@ildena It's not about shooting anyone but facts and rationality. If you read these tests, you'll find that humidity reading error is a f(h,t) function, not a simple constant offset.

@ascillato
Copy link
Contributor

If you refer to this PR, I can't see them on the computer either

Yes, on this PR. Please, click on the upper tab called changes, and you will see them. Those are not obligatory. Just small suggestions.

@arendst arendst added the on hold by dev team Result - Feature request put on hold by member of development team label Jan 27, 2020
@stale
Copy link

stale bot commented Feb 6, 2020

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

@stale stale bot added the stale Action - Issue left behind - Used by the BOT to call for attention label Feb 6, 2020
@ascillato
Copy link
Contributor

Testing....

@stale stale bot removed the stale Action - Issue left behind - Used by the BOT to call for attention label Feb 6, 2020
@arendst
Copy link
Owner

arendst commented Feb 10, 2020

@ascillato Adrian, did you find any issues with this driver or can I release it?

arendst added a commit that referenced this pull request Feb 10, 2020
Add new DHT driver. The old driver can still be used using define USE_DHT_OLD (#7468)
@arendst
Copy link
Owner

arendst commented Feb 10, 2020

Merged with additional Shelly code.

Thx.

@arendst arendst closed this Feb 10, 2020
@ascillato
Copy link
Contributor

ascillato commented Feb 10, 2020

@arendst

So far this new driver is working very good. @KrzysztofPrzygoda Good job!!! 👍

Sorry for the delay on testing.

@ascillato2 ascillato2 added enhancement Type - Enhancement that will be worked on fixed Result - The work on the issue has ended and removed on hold by dev team Result - Feature request put on hold by member of development team labels Feb 10, 2020
@ascillato2 ascillato2 self-requested a review February 10, 2020 17:40
@Jason2866
Copy link
Collaborator

@KrzysztofPrzygoda @arendst
My two AM2301 are working with this changed driver too.

@klaus-pittisch
Copy link

15/20 AM2301 working with ver 8.1.0.7.
5 / 20 report NULL after 2 days

:((

@ascillato
Copy link
Contributor

That was also reported at #7717

@ascillato
Copy link
Contributor

@KrzysztofPrzygoda any idea?

@pingme01
Copy link

Is it possible that this protocol change is affecting BME280 sensors? They are running fine on 8.1.0, but with the latest beta release and the final 8.2.0 my Sonoff Basic is not showing any data from this sensors. Have the same issue with another Sonoff Basic with BME280 sensor.

@effelle
Copy link
Contributor

effelle commented Mar 23, 2020

Let me guess, you are using Tasmota and not Tasmota-sensors...
Builds are changed.

@pingme01
Copy link

you are right...i've added the BME280 sensor like described on this page: https://github.com/arendst/Tasmota/wiki/BME280

@effelle
Copy link
Contributor

effelle commented Mar 23, 2020

For future reference: builds

@marlonpnz
Copy link

marlonpnz commented Jun 21, 2020

[SOLUTION FOUND HERE for null readings] - Tasmota version 8.3.1

Hi friends,

I am new here but want to contribute as I think it will help many folks.
I was having issues with this cheap sensors too (null readings after some hours in tasmota).
So I decided to investigate.
I used two DHT22(AM2301) sensors that were used in an old project, in which they worked flawlessly.
Then, analyzing the waveform of communications in oscilloscope I saw that the start pulse (sensor activation) had not a fix width. It had a minimum of 2ms, as the command "delay(2)" is supposed to do. However many times it took much more than 2ms, like 3, 5ms. I believe that it is because the delay() command calls an yield() behind the scenes, so sporadically it may be pulling the line too long (more than 10ms) and triggering a bug in the sensor, making it freeze.
Once the tasmota reach the null reading, the pulse still exists but the sensor do not respond anymore. Only power cycling the sensor (or all together) restore the readings, at least for some hours.
So, the incredible simple solution was replace the "delay(2)" command with "delayMicroseconds(1500)" in the line 80 of the file "xsns_06_dht.ino". Used 1500us because this was the value I had used in my old project, but 2000us should work equally well. To clarify, the delayMicroseconds() does not call yield(), then the delay time is fix.
After recompile and flash the esp's with the new bin file they worked fine. (for two days until now). I will report if they fail again. But, considering before that they worked for only a couple of hours, I hope the problem was solved.
May be someone else could test it also, to reproduce my results.
Anyway I think this change can be implemented right away, once it change nothing in the logic, just guarantees a fix pulse.
I hope this will help all the others brave DHT users too.

@eku
Copy link
Contributor

eku commented Jun 21, 2020

@marlonpnz as written in another issue, I even increased the delay from 2ms to 20ms to get more stability. Check out the Ethersex DHT driver written by me for reference.

@marlonpnz
Copy link

marlonpnz commented Jun 21, 2020

@marlonpnz as written in another issue, I even increased the delay from 2ms to 20ms to get more stability. Check out the Ethersex DHT driver written by me for reference.

Hi @eku, thanks you reply.
I actually could not shot the waveform exactly when sensor froze, because there is no way to trigger it with a simple oscilloscope. I am supposing at some point the inconsistent pull down delay may be triggering something in the sensor. It could even be some specific delay value window, intended (or not) to enable a configuration/calibration state in the sensor, for example. Of course, it would be a not published feature. As I made clear, the sensor stop working when we get the null readings, not ESP.
Your code may be more stable simply because a delay(20) is more consistent that a delay(2). The chance delay(20) produce a real delay greater than 20ms is much smaller than in the 2ms case (remember the yield() called inside delay(), but not in delayMicroseconds()). But, I think 20ms is way to much (unnecessary) if the minimum is 1ms, as per datasheet.
In my old project code, which I did myself (did no use library), the sensor never froze and worked for more than a year. This was the main reason I believed it could be fixed by code.
But anyway, against fact there is no argument. Simply try it out. If it do not work for you (and others) than the root cause is not the same.
I keep my suggestion to change that single line of code from "delay(2)" to "delayMicroseconds(2000)", which will CHANGE NOTHING in the logic! There is no harm at all.
I have made a lot of tests, including power source voltage levels, bypass capacitor, increasing the readings interval, etc. Only fixing that delay inconsistency worked.

arendst added a commit that referenced this pull request Jun 22, 2020
Change delay(2) to delayMicroseconds(2000) and see if it helps (#7468)
@pmknowles
Copy link

Sorry for re-opening. If you prefer I'll create a new issue.
Sollowing on from a query regarding the AM2302 giving null readings the OP sent me one of his sensors to test. It wouldn't work on an ESP8266. I connected the sensor to a Pi running node-red and could get sensor readings from the sensor.
I then tried flashing with this project https://github.com/projetsdiy/esp8266-dht22-mqtt-home-assistant/find/master
and got this result
image
without disconnecting the sensor I then flashed with the latest dev version and got this
image
Tasmota appears to be using the ESPEasy DHT library so I flashed and configured ESPEasy and got NaN.
There appears to be an issue with the ESPEasy DHT library as the arduino DHT.h library seems to work.
I don't know enough about the internals of Tasmota to see if replacing the ESPEasy library with the Arduino one would resolve the problem.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement Type - Enhancement that will be worked on fixed Result - The work on the issue has ended
Projects
None yet
Development

Successfully merging this pull request may close these issues.

DHT22 report null after a while