Possible SDK 2.2.1 HTTP Module Regression

[NicolSpies]

Expected behavior

I am trying to connect to the Telegram API, "api.telegram.org" as was successfully reported by @FrankX0 in #1707 after the SDK 2.2.0 update as reported in #2269.

The following behavior indicating a successful client handshake is expected:

HTTP client: hostname=api.telegram.org
HTTP client: port=443
HTTP client: method=GET
HTTP client: path=/bot446398269:AAF7jqBWZhj9oJlWT2f8mCS8_X5_LxJGLRA/sendMessage?chat_id=211698604&text=Hello!
HTTP client: DNS request
HTTP client: DNS pending
HTTP client: DNS found api.telegram.org 149.154.167.220
client handshake start.
client handshake ok!
HTTP client: Connected
HTTP client: Sending request header
HTTP client: All sent
Reason:[-0x7880]
HTTP client: Disconnected
http_status=200
strlen(full_response)=698
response={"ok":true,"result":{"message_id":91,"from":{"id":446398269,"is_bot":true,"first_name":"\u041c\u043e\u0438 \u0434\u0430\u0442\u0447\u0438\u043a\u0438","username":"smok_sensors_bot"},"chat":{"id":211698604,"first_name":"Alex","last_name":"Smok","username":"alexsmok","type":"private"},"date":1521057955,"text":"Hello!"}}<EOF>

Actual behavior

The actual behavior indicating a handshake failure is as follows:

HTTP client: hostname=api.telegram.org
HTTP client: port=443
HTTP client: method=GET
HTTP client: path=/bot446398269:AAF7jqBWZhj9oJlWT2f8mCS8_X5_LxJGLRA/sendMessage?chat_id=211698604&text=Hello!
HTTP client: DNS request
HTTP client: DNS pending
HTTP client: DNS found api.telegram.org 149.154.167.220
client handshake start.
pm open,type:2 0
client handshake failed!
Reason:[-0x7200]
HTTP client: Disconnected with error: -114
HTTP client: Connection timeout
HTTP client: Calling disconnect
HTTP client: manually Calling disconnect callback due to error -12
HTTP client: Disconnected
http_status=-1

Test code

http.get("https://api.telegram.org/bot446398269:AAF7jqBWZhj9oJlWT2f8mCS8_X5_LxJGLRA/sendMessage?chat_id=211698604&text=Hello!",nil,
      function(code, data)
          print(code, data)
      end)

NodeMCU version

NodeMCU custom build by frightanic.com
	branch: dev
	commit: 576747576658c41df7f9584e0e8be0232dadcfda
	SSL: true
	modules: cron,file,gpio,http,net,node,rtcmem,rtctime,sjson,sntp,tmr,uart,wifi,tls
 build created on 2018-10-31 09:18
 powered by Lua 5.1.4 on SDK 2.2.1(6ab97e9)

Hardware

NodeMCU devkit v2

[nwf]

0x7200 is MBEDTLS_ERR_SSL_INVALID_RECORD. SSL debugging information and/or a packet trace may shed more light.

[NicolSpies]

I can only build using frightanic.com , if someone can build a version for me where the tls.setDebug function is mapped to mbedtls_debug_set_threshold I can retest and supply more detail. Another possibility is to confirm the tests done in #1707 against a SDK 2.2.0 build.

[NicolSpies]

The test was repeated using the tls module using the following test code:

  srv = tls.createConnection()
  srv:connect(443,"149.154.167.220")

  srv:on("connection", function(sck, c)
    print("connected")
  end)

The debug information seems to indicate the same problem:

client handshake start.
client handshake failed!
Reason:[-0x7200]
pm open,type:2 0

[NicolSpies]

Could the returned message length be exceeding the maximum value as it appears to happen only when connecting to certain servers ?

[NicolSpies]

Ok, @marcelstoer, my time has come, I am giving your Docker NodeMCU build a try in order to get more SSL debugging data. I am struggling to find the tls.setDebug switch to enable the generation debugging information before building. Please help.

Hopefully this will shed more information on the MBEDTLS_ERR_SSL_INVALID_RECORD 0x7200 error

[marcelstoer]

Did you check app/include/user_mbedtls.h? There are a few "DEBUG" switches.

[NicolSpies]

Thanks, I will change #undef MBEDTLS_SSL_DEBUG_ALL to #define MBEDTLS_SSL_DEBUG_ALL in app/include/user_mbedtls.h to see if it works.

[NicolSpies]

@djphoenix @nwf I can now build image using docker, struggling to find the debug switches in mbedtls to provide SSL debuging info. Any suggestions I can try ?

[FrankX0]

If I recall correctly:

• Enable the generic firmware debug option
• Set the mbedTLS debug output level with tls.setDebug() from Lua (higher number is more output).

[NicolSpies]

The mbedTLS needs to be compiled with debug support. That is the switch I need to find. I have the following options in user_mbedtls.h:
#undef MBEDTLS_MEMORY_DEBUG or #undef MBEDTLS_SSL_DEBUG_ALL or #undef MBEDTLS_DEBUG_C

[NicolSpies]

Ok, it is solved.
Increasing the SSL_BUFFER_SIZE to 6000 fixed the 0x7200 handshake fail error.

[marcelstoer]

I'm glad there's an issue reference in the config file pointing at hints why we came up with that magic number. I think it's safe to assume that over time we'll only ever have to increase that buffer size as certificates and cipher suites evolve.

[NicolSpies]

@marcelstoer I will reduce the size further to find the minimum I need for telegram. Larger sizes cause low memory panic. Thanks for your docker facility, i am now proficient. Next is to try the LFS route.

[nwf]

Hopefully as ECC cryptography slowly trickles into pervasive support, we won't need such huge fragments, because we won't be carting around RSA public keys. But LFS does relieve some of the pressure against big SSL buffers.