Parsing String values crashes ESP8266

[JeffAshton]

Also noticing on the ESP8266, that parsing anything but a statically defined const char*, kills the parser.

#include <ArduinoJson.h>

void setup() {

  Serial.begin( 115200 );
  delay( 10 );
  Serial.println( "" );

  StaticJsonBuffer<200> jsonBuffer;

  String json = "{\"mqtt.host\":\"mqtt.test.com\",\"mqtt.port\":1883}";

  Serial.println( "Parsing json string" );
  JsonObject& root = jsonBuffer.parseObject( json );

  if( root.success() ) {
    Serial.println( "parsed" );
  } else {
    Serial.println( "failed" );
  }
}

void loop() {
}

Results in

Parsing json string

Exception (9):
epc1=0x402021cd epc2=0x00000000 epc3=0x00000000 excvaddr=0x3ffe9d67 depc=0x00000000

ctx: cont 
sp: 3ffe9b40 end: 3ffe9e30 offset: 01a0

>>>stack>>>
3ffe9ce0:  3fffdc20 00000013 3ffe8de4 402039a9  
3ffe9cf0:  3ffe8968 00000000 3ffe8de4 3ffe9e5c  
3ffe9d00:  3fffdc20 00000000 3ffe8de4 40202216  
3ffe9d10:  3ffe9d30 3ffe9d34 3ffe9d34 4020450a  
3ffe9d20:  3fffdc20 00000000 3ffe8de4 4020214e  
3ffe9d30:  3ffe8878 716d227b 682e7474 2274736f  
3ffe9d40:  716d223a 742e7474 2e747365 226d6f63  
3ffe9d50:  716d222c 702e7474 2274726f 3838313a  
3ffe9d60:  00007d33 00000000 00000000 00000000  
3ffe9d70:  00000000 00000000 00000000 00000000  
3ffe9d80:  00000000 00000000 00000000 00000000  
3ffe9d90:  00000000 00000000 00000000 00000000  
3ffe9da0:  00000000 00000000 00000000 00000000  
3ffe9db0:  00000000 00000000 00000000 00000000  
3ffe9dc0:  00000000 00000000 00000000 00000000  
3ffe9dd0:  00000000 00000000 00000000 00000000  
3ffe9de0:  00000000 00000000 00000000 00000000  
3ffe9df0:  00000000 00000000 00000000 00000037  
3ffe9e00:  3fff38a8 0000002e 0000002e 00000000  
3ffe9e10:  00000000 00000000 3ffe9e54 40201a07  
3ffe9e20:  00000000 00000000 3ffe8e10 40100378  
<<<stack<<<

[JeffAshton]

The issue occurs here
JsonBuffer.cpp#L22 when it tries to create the new JsonObject.

[bblanchon]

I'm not able to reproduce this bug.

Please dig deeper in the new operator, in JsonBufferAllocated.hpp, line 16.

The StaticJsonBuffer is probably too small to hold a copy of the JSON source and the object tree, but that's not a reason to crash like this.

Please also try to increase the size of the StaticJsonBuffer to see if it improves.

Also, please note that using String instead of char[] will have a huge negative impact on program performance and code size. The JSON input will be duplicated (you saw the call to strdup) and you'll pay the cost of String's internal allocation and duplication.

[bblanchon]

I fixed a bug which is triggered when the StaticJsonBuffer is too small to hold a copy of the JSON source.

It could be the same bug.

Please try branch issue104 and tell me if it works now.

[bblanchon]

I merge it into master.
Please give it a try.

[JeffAshton]

Still faulting with the latest

Parsing json string

Exception (9):
epc1=0x4020226d epc2=0x00000000 epc3=0x00000000 excvaddr=0x3ffe9727 depc=0x00000000

ctx: cont 
sp: 3ffe94f0 end: 3ffe97f0 offset: 01a0

>>>stack>>>
3ffe9690:  3ffe9848 00000012 3ffe96d0 40202a25  
3ffe96a0:  3fffdc20 00000013 3ffe9848 402031f5  
3ffe96b0:  3ffe8518 00000000 3ffe9848 3ffe981c  
3ffe96c0:  3fffdc20 3fff0a08 3ffe9848 402022c4  
3ffe96d0:  3ffe96f0 3ffe96f4 3ffe96f4 4020570a  
3ffe96e0:  3fffdc20 3fff0a08 3ffe9848 402021ee  
3ffe96f0:  3ffe8478 716d227b 682e7474 2274736f  
3ffe9700:  716d223a 742e7474 2e747365 226d6f63  
3ffe9710:  716d222c 702e7474 2274726f 3838313a  
3ffe9720:  00007d33 3fffdc20 3ffe981c 00000030  
3ffe9730:  00000000 00000000 00000000 00000000  
3ffe9740:  00000000 00000000 00000000 00000000  
3ffe9750:  00000000 00000000 00000000 00000000  
3ffe9760:  00000000 00000000 00000000 00000000  
3ffe9770:  00000000 00000000 00000000 00000000  
3ffe9780:  00000000 00000000 00000000 00000000  
3ffe9790:  00000000 00000000 00000000 00000000  
3ffe97a0:  00000000 00000000 00000000 00000000  
3ffe97b0:  00000000 00000000 00000000 00000037  
3ffe97c0:  3fff0a08 0000002e 0000002e 00000000  
3ffe97d0:  00000000 00000000 3ffe9814 40201a43  
3ffe97e0:  00000000 00000000 3ffe87d0 40100374  
<<<stack<<<

 ets Jan  8 2013,rst cause:2, boot mode:(3,6)

[bblanchon]

I ordered this and that
Average delivery 2-4 weeks, may take up to 6 weeks.

So I should eventually figure it out.

[asetyde]

Hi , i've also a Nodemcu 1.0 , esp8266 12e family , with my test nothing .. i use this distribution that i think it's quite good .

many thanks for your library @bblanchon , i tried to parse my struct to network but between c and java not so friendly game ... at now i start with your library . the best is take and parse a struct automatically in encode and also decode

[bblanchon]

@asetyde, thanks for your feedback. i'm glad to hear that it works on your side.

[asetyde]

sorry not , i do an error with example code , about this :

StaticJsonBuffer<200> jsonBuffer; 200 what ? byte bit kbyte ?

[bblanchon]

These are bytes, please have a look at the documentation.

You can try to increase the size of the buffer, but 200 bytes should be enough for the example above.

[asetyde]

now i've added , they can , i hope , help us .. i think

[asetyde]

thanks ok @bblanchon

[asetyde]

ok, wait Dynamic allocation is ok .

With static , how you determinate a static allocation in memory ... esp is not as arduino , has flash size distributed in this configuration :

[bblanchon]

If dynamic allocation is OK to you, you can replace StaticJsonBuffer by DynamicJsonBuffer.

[asetyde]

yes , i've done . also when i tried different buffer size , there are strange but different result .

[bblanchon]

Can you be more specific?

[asetyde]

So so :)

i don't have many time to try , but i think the problem is where you allocate your static object , because there are many different about memory organization and space from esp and avr ... family is different .

when i 've tried @JeffAshtonCT sketch with different size of buffer , reply on serial is many different :

also i think it's not good idea do this in setup function ...

i can do 3 fast example :

with 200 you see stack error but , with 10

with 50

with 99

[asetyde]

but is stupid test ..

[bblanchon]

Alexandro, it looks like you're using version 5.0.1. Can you checkout
master branch and try again?

Also, what happens with DynamicJsonBuffer?

I think that the stack may be too small. Do you know if you can change it?
This is usualy done in the compiler command line.
Le 31 août 2015 11:12, "Alexandro Todeschini" notifications@github.com a
écrit :

but is stupid test ..

—
Reply to this email directly or view it on GitHub
#104 (comment)
.

[igrr]

Hi all,
esp8266/Arduino guy here.

Exception is caused by an unaligned 32-bit store in ArduinoJson::JsonBuffer::createObject. The unaligned pointer comes from placement new operator in JsonBufferAllocated which leads to JsonBuffer::alloc. For this particular sample posted by @JeffAshtonCT, this calls StaticJsonBuffer::alloc. Apparently alloc happily returns unaligned pointers which is okay on AVR but leads to a CPU exception on this particular core. Padding all allocations to 4 bytes inside alloc fixes this issue.

virtual void* alloc(size_t bytes) {
#ifdef ESP8266
bytes = (bytes + 3) & (~3);
#endif
...

Something similar is also required in BlockJsonBuffer::allocInHead, but i haven't investigated further.

Edit:
some debug logs which show what happens in alloc:

Parsing json string
alloc: bytes=47, _buffer=3ffe97f4, _size=0
alloc: p=3ffe97f4
JsonParser::createObject this=3ffe97f0
alloc: bytes=8, _buffer=3ffe97f4, _size=47
alloc: p=3ffe9823
Exception (9):
epc1=0x402023b1 epc2=0x00000000 epc3=0x00000000 excvaddr=0x3ffe9823 depc=0x00000000

[bblanchon]

Thanks Ivan. I'll fix that ASAP.
Le 31 août 2015 11:47, "Ivan Grokhotkov" notifications@github.com a
écrit :

Hi all,
esp8266/Arduino guy here.

Exception is caused by an unaligned 32-bit store in
ArduinoJson::JsonBuffer::createObject. The unaligned pointer comes from placement
new operator in JsonBufferAllocated
https://github.com/bblanchon/ArduinoJson/blob/master/include/ArduinoJson/Internals/JsonBufferAllocated.hpp#L16
which leads to JsonBuffer::alloc. For this particular sample posted by
@JeffAshtonCT https://github.com/JeffAshtonCT, this calls
StaticJsonBuffer::alloc. Apparently alloc happily returns unaligned
pointers which is okay on AVR but leads to a CPU exception on this
particular core. Padding all allocations to 4 bytes inside alloc fixes
this issue.

virtual void* alloc(size_t bytes) {
#ifdef ESP8266
bytes = (bytes + 3) & (~3);
#endif
...

Something similar is also required in BlockJsonBuffer::allocInHead, but i
haven't investigated further.

—
Reply to this email directly or view it on GitHub
#104 (comment)
.

[asetyde]

Russian is the best :) 👍 thanks @igrr

@bblanchon when you are ok , i'll try

[asetyde]

Dynamicbuffer is ok , work

[bblanchon]

I fixed the alignment in branch alignment, it should work on the ESP8266.
I'll try to reduce the code size before merging it into master.
Please give it a try.

[asetyde]

but you 've merged or not ?

[bblanchon]

No I didn't.
I'm not satisfied with the code size.
Did you try the "alignment" branch?

[asetyde]

also dynamic has stack exception with a heavy use to transfer a struct (2k bytes)

[asetyde]

wait , now i'll try

[asetyde]

Pass ok ;

result :
100

---

200 (as example)

---

300

[bblanchon]

Thanks, I'll merge as soon as I find a way to get the code smaller.

[bblanchon]

I saw issue #110, but I don't understand what you want to do.
Please keep discussion separated, and comment in #110.

[asetyde]

ok

[bblanchon]

It's fixed in master and will soon be released as 5.0.2.

[asetyde]

@igrr Please , i not understand why memory space is too small , in BlockJsonBuffer , only parameter i see is :
static const size_t FIRST_BLOCK_CAPACITY = 32;

it can be has more memory ?
or in dynamic use max esp capacity

#pragma once

#include "../JsonBuffer.hpp"

#include <stdlib.h>

namespace ArduinoJson { namespace Internals { class DefaultAllocator { public: void* allocate(size_t size) { return malloc(size); } void deallocate(void* pointer) { free(pointer); } };

template <typename TAllocator> class BlockJsonBuffer : public JsonBuffer { struct Block; struct EmptyBlock { Block* next; size_t capacity; size_t size; }; struct Block : EmptyBlock { uint8_t data[1]; };

public:
BlockJsonBuffer() : _head(NULL) {}

~BlockJsonBuffer() {
Block* currentBlock = _head;

while (currentBlock != NULL) { Block* nextBlock = currentBlock->next; _allocator.deallocate(currentBlock); currentBlock = nextBlock;