Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Secure MQTT getting exception 29 with two-way traffic #2256

Closed
mtnbrit opened this issue Jul 10, 2016 · 4 comments
Closed

Secure MQTT getting exception 29 with two-way traffic #2256

mtnbrit opened this issue Jul 10, 2016 · 4 comments
Labels
type: bug waiting for feedback Waiting on additional info. If it's not received, the issue may be closed.

Comments

@mtnbrit
Copy link

mtnbrit commented Jul 10, 2016

Basic Infos

Hardware

Hardware: ESP-12F
Core Version: 2.3.0

Description

running secure mqtt on port 8883, im using the lmroy PubSubClient mqtt lib connected to mosquitto 1.4.8 broker, esp is listening (subscribed) to several topics on broker, and also periodically publishing its own messages to different topics. I can trigger exception 29 by publishing something to one of the subscribed topics. I spent yesterday debugging the PubSubClient and MQTT libs to find that if the esp is publishing something while also receiving messages on topics its subscribed to, the received stream gets corrupted (random types and zero length or huge messages lengths) and crashes the PubSubClient new Publish() function called from client.loop(), _recv_message() and readPacket() .

It feels like the tls receive buffer is getting overwritten, I remember one of the memory optimizations igrr made was to use the same memory for both the rx and tx buffers, or something akin?

If I switch to non-tls mqtt on port 1883, it doesn't crash at all under the same conditions.

Maybe its not possible to send and receive at the same time over secure mqtt with the current state of axTLS buffers?

I haven tried the knolleary pubsubclient lib yet, its a lot of work to switch over, but I would if this is known to work.

Settings in IDE

Module: Generic ESP8266 Module
Flash Size: 4MB/1MB
CPU Frequency: 80Mhz
Flash Mode: dio
Flash Frequency: 40Mhz
Upload Using: OTA
Reset Method: ck

Sketch

I haven't had time to boil my sketch down to just the parts that crash, hopefully there is enough general info in the problem description to confirm or decode what may be going on.

...normal loop execution...
ESP free heap: 18240
(waiting between loops)
<---- I publish a message to one of the subscribed topics here
Exception (29):
epc1=0x4000df64 epc2=0x00000000 epc3=0x00000000 excvaddr=0x00000000 depc=0x00000000

ctx: cont
sp: 3fff2d60 end: 3fff3120 offset: 01a0

stack>>>
3fff2f00: 3fff6904 00000000 3fff69e4 4020f404
3fff2f10: 00000000 00000000 00000000 00003034
3fff2f20: 00000000 3fff6a08 3fff6904 401004d8
3fff2f30: 00000002 00000000 3fff6904 0000002d
3fff2f40: 3fff69e4 00000032 3fff6904 4020f944
3fff2f50: 00000003 00000003 3fff6909 402087a2
3fff2f60: 401059aa 190b8593 00000007 00000000
3fff2f70: 3fff1c94 00000001 3fff1c94 4020fa9c
3fff2f80: 40105c38 00000001 3fff1c94 4020ffc5
3fff2f90: 4022db76 3fff20e8 00000007 402087a2
3fff2fa0: 4022e9c3 00000000 00000000 402087e3
3fff2fb0: 3fff3b3c 3fff1c94 3fff17a8 40214db4
3fff2fc0: 3fff1c94 3fff1db0 3fff17a8 402061bc
3fff2fd0: 00000000 00000000 00000000 3a34333a
3fff2fe0: 353a6239 31383a62 5f706d2f 72776c66
3fff2ff0: 7277702d 2f38705f 5f727563 74617473
3fff3000: 65740065 3fffc600 00000001 3fff20e8
3fff3010: 402144d9 00000030 0000000a ffffffff
3fff3020: 402144c8 00000001 402144bc 00000000
3fff3030: 00000000 6e696874 312f7367 65663a38
3fff3040: 3a34333a 353a6239 31383a62 7373722f
3fff3050: 00000069 3fffdad0 3fff20f0 00000030
3fff3060: 00000000 3fffdad0 3fff20f0 00000030
3fff3070: 00000000 00004908 402105c4 40210d04
3fff3080: 00000000 00000000 3fff1ea4 3fff20f0
3fff3090: 00000000 3fff1e60 00000000 00000000
3fff30a0: 00000000 00000000 00000000 00000000
3fff30b0: 00303137 00000000 36331ea4 4000352e
3fff30c0: 322e3432 00000032 feefeffe feefeffe
3fff30d0: 41e68646 20000000 4038372f ec1dff33
3fff30e0: 3ffcda5b feefeffe 402146cc 402146b8
3fff30f0: 00000000 00000000 00000001 3fff20f0
3fff3100: 3fffdad0 00000000 3fff20e8 402144dc
3fff3110: feefeffe feefeffe 3fff2100 40100718
<<<stack<<<

ets Jan 8 2013,rst cause:1, boot mode:(3,7)

load 0x4010f000, len 1384, room 16
tail 8
chksum 0x2d
csum 0x2d
v3de0c112
~ld

Debug Messages

Decoding 24 results
0x4020f404: MQTT::Publish::Publish(unsigned char, unsigned char_, unsigned int) at /Users/jonm/Documents/Arduino/libraries/pubsubclient-master/src/MQTT.cpp:53
0x401004d8: malloc at /Users/jonm/Library/Arduino15/packages/esp8266/hardware/esp8266/2.3.0/cores/esp8266/umm_malloc/umm_malloc.c:1664
0x4020f944: MQTT::readPacket(Client&) at /Users/jonm/Documents/Arduino/libraries/pubsubclient-master/src/MQTT.cpp:53
0x402087a2: SSLContext::available() at /Users/jonm/Library/Arduino15/packages/esp8266/hardware/esp8266/2.3.0/libraries/ESP8266WiFi/src/WiFiClientSecure.cpp:517
0x401059aa: wdt_feed at ??:?
0x4020fa9c: PubSubClient::_recv_message() at /Users/jonm/Documents/Arduino/libraries/pubsubclient-master/src/PubSubClient.cpp:258
0x40105c38: ets_timer_arm_new at ??:?
0x4020ffc5: PubSubClient::loop() at /Users/jonm/Documents/Arduino/libraries/pubsubclient-master/src/PubSubClient.cpp:258
0x4022db76: wifi_get_opmode at ??:?
0x402087a2: SSLContext::available() at /Users/jonm/Library/Arduino15/packages/esp8266/hardware/esp8266/2.3.0/libraries/ESP8266WiFi/src/WiFiClientSecure.cpp:517
0x4022e9c3: wifi_station_get_connect_status at ??:?
0x402087e3: WiFiClientSecure::connected() at /Users/jonm/Library/Arduino15/packages/esp8266/hardware/esp8266/2.3.0/libraries/ESP8266WiFi/src/WiFiClientSecure.cpp:517
0x40214db4: PubSubClient::connected() at /Users/jonm/Documents/Arduino/libraries/pubsubclient-master/src/PubSubClient.cpp:336
0x402061bc: loop at /Users/jonm/Library/Mobile Documents/comappleCloudDocs/iGrowV2/esp8266 arduino code/air-mp-mq-rules-workingstable/air-mp-mq-rules-workingstable.ino/air-mp-mq-rules-workingstable.ino.ino:679
0x402144d9: loop_wrapper at /Users/jonm/Library/Arduino15/packages/esp8266/hardware/esp8266/2.3.0/cores/esp8266/core_esp8266_main.cpp:56
0x402144c8: loop_wrapper at /Users/jonm/Library/Arduino15/packages/esp8266/hardware/esp8266/2.3.0/cores/esp8266/core_esp8266_main.cpp:56
0x402144bc: loop_wrapper at /Users/jonm/Library/Arduino15/packages/esp8266/hardware/esp8266/2.3.0/cores/esp8266/core_esp8266_main.cpp:56
0x402105c4: _M_manager at /Users/jonm/Library/Arduino15/packages/esp8266/hardware/esp8266/2.3.0/libraries/ESP8266mDNS/ESP8266mDNS.cpp:384
0x40210d04: operator() at /Users/jonm/Library/Arduino15/packages/esp8266/hardware/esp8266/2.3.0/libraries/ESP8266mDNS/ESP8266mDNS.cpp:384
(inlined by) _M_invoke at /Users/jonm/Library/Arduino15/packages/esp8266/tools/xtensa-lx106-elf-gcc/1.20.0-26-gb404fb9-2/xtensa-lx106-elf/include/c++/4.8.2/functional:2071
0x4000352e: ?? ??:0
0x402146cc: std::_Function_base::Base_manager<void ()(MQTT::Publish const&)>::_M_manager(std::_Any_data&, std::_Any_data const&, std::_Manager_operation) at /Users/jonm/Library/Arduino15/packages/esp8266/tools/xtensa-lx106-elf-gcc/1.20.0-26-gb404fb9-2/xtensa-lx106-elf/include/c++/4.8.2/functional:1934
0x402146b8: std::_Function_handler<void (MQTT::Publish const&), void (*)(MQTT::Publish const&)>::_M_invoke(std::_Any_data const&, MQTT::Publish const&) at /Users/jonm/Library/Arduino15/packages/esp8266/tools/xtensa-lx106-elf-gcc/1.20.0-26-gb404fb9-2/xtensa-lx106-elf/include/c++/4.8.2/functional:2069
0x402144dc: loop_wrapper at /Users/jonm/Library/Arduino15/packages/esp8266/hardware/esp8266/2.3.0/cores/esp8266/core_esp8266_main.cpp:56
0x40100718: cont_norm at /Users/jonm/Library/Arduino15/packages/esp8266/hardware/esp8266/2.3.0/cores/esp8266/cont.S:109

@igrr igrr added the type: bug label May 6, 2017
@KillerKrapfen
Copy link

Hello,

I wanted to asked if there is some progress on this issue so far?

Thanks

@devyte
Copy link
Collaborator

devyte commented Oct 15, 2017

@mtnbrit I believe that some changes are in latest git related to tls. Please test with latest git, and report back here.

@devyte devyte added the waiting for feedback Waiting on additional info. If it's not received, the issue may be closed. label Oct 15, 2017
@igrr
Copy link
Member

igrr commented Oct 16, 2017

@devyte I don't recall changing anything which could have fixed this.

There is a pair of PRs which can resolve this:
igrr/axtls-8266#41
#3019

but I was planning a different implementation which would actually allow full-duplex data transfer as long as the fragment buffer size is sufficient to keep both, and return an error similar as this PR does only when there is no more space in the fragment buffer. However this is slightly lower on my todo list that implementing root certificate store, so haven't gotten to this yet.

@vshymanskyy
Copy link

Could be possibly related to #3002

igrr added a commit that referenced this issue Dec 25, 2017
When application requests to write data, check if there is any unread
decrypted data left. If there is, don't write immediately, but save the
data to be written. When all decrypted data has been consumed by the
application, send out the saved outgoing data.

Fixes #2256.
igrr added a commit that referenced this issue Dec 26, 2017
When application requests to write data, check if there is any unread
decrypted data left. If there is, don't write immediately, but save the
data to be written. When all decrypted data has been consumed by the
application, send out the saved outgoing data.

Fixes #2256.
devyte pushed a commit that referenced this issue Dec 26, 2017
* WiFiClientSecure: don't decrypt when testing for 'connected'

* WiFiClientSecure: don't trash unread decrypted data when writing

When application requests to write data, check if there is any unread
decrypted data left. If there is, don't write immediately, but save the
data to be written. When all decrypted data has been consumed by the
application, send out the saved outgoing data.

Fixes #2256.
@igrr igrr reopened this Dec 27, 2017
@igrr igrr closed this as completed Jan 2, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type: bug waiting for feedback Waiting on additional info. If it's not received, the issue may be closed.
Projects
None yet
Development

No branches or pull requests

5 participants