Merge parser and generator API

[leventesen]

Hi again,

Is there an easy way to convert a Parser::JsonObject to Generator::JsonObject? I have a good reason to do this :)

[bblanchon]

Unfortunately not 😕

I've been scratching my head for a long time trying to find a common API for both the parser and the generator... but I'm not sure it will really happen.

Instead, I think that for v4.0 I'm going to change the API of the generator to something more simple.

My current thougth it to get rid of Generator::JsonObject and Generator::JsonArray to have something like that:

JsonWriter json(Serial);
json.beginObject();
json.add("sensor", "gps");
json.add("time", 1351824120);
json.beginArray("data");
json.add<6>(48.756080); 
json.add<6>(2.302038); 
json.endArray(); 
json.endObject();

Plain and simple.
I want to do that because it would solve/prevent issues #10, #13, #17 and #19.
It would be less fancy than the current API and a little more verbose, but it would be less error prone.

Anyway... my point is that in this new API, I could easy add helpers to write Parser::JsonArray and Parser::JsonObject, as well as containers of the STL like vector and map.

So that would solve your problem

[leventesen]

I want to help you to find a common API, so I will tell my thoughts..

• There should be only one JsonObject structure
• JsonParser will generate JsonObject
• User code will be able to create or modify JsonObject
• JsonSerializer will then serialize the JsonObject
• All these can be applied to JsonArray and the others.

[bblanchon]

You know what?
I was ready to say "no way this is possible!"
But I've been thinking to this for a while now and I think there is a way to do it.

The issue right now is that Parser::JsonObject to Generator::JsonObjectare really different.
One stores a pointer to a jsmn token, and the other one stores the actual values.

If we want to make only one JsonObject, we have to choose one of the two approachs.
Storing the values in the JsonObject would be impossible with the fixed allocation constraint.
Instead I think that JsonObject should store a pointer to token.
But not a jmsn token, a custom token that would represent a node in the object tree.
The actual tokens would be owned by the JsonParser or the JsonGenerator.

I'll implement something among those lines, let's see where it goes...

[bblanchon]

I made a lot of progress, there is still a lot to do but I think I can show you what the API will look like.
Please give me some feedback.

New API for the generator:

StaticJsonBuffer<16> json;

JsonArray array = json.createArray();
array.add(48.756080, 6); // 6 is the number of decimals to print
array.add(2.302038, 6);  // if not specified, 2 digits are printed

JsonObject root = json.createObject();
root["sensor"] = "gps";
root["time"] = 1351824120;
root["data"] = array;

As you can see, very few changes:

• The static memory allocation is handled by StaticJsonBuffer
• JsonArray and JsonArray are not template types anymore
• You need to call factory methods to build them

New API for the parser

char s[] = "{\"sensor\":\"gps\",\"time\":1351824120,\"data\":[48.756080,2.302038]}";

StaticJsonBuffer<16> json;

JsonObject root = json.parse(s);

char*  sensor    = root["sensor"];
long   time      = root["time"];
double latitude  = root["data"][0];
double longitude = root["data"][1];

Almost no change, JsonParser has been replaced by StaticJsonBuffer.

[leventesen]

I think this is a good starting point. Moving templates to StaticJsonBuffer is a nice solution. What about removing items? Is it possible with your allocation method?

[henrikekblad]

I'm really looking forward to the combined parser/generator. Would ease up things a lot.
Planning to use it for mysensors.org serial gateway if the footprint is acceptable.
The question is when it is possible to try it out? I totally understand if you can't give any ETAs.

[bblanchon]

Actually, it's almost finished 😄
It works well and it's stable enough to try it.

Still to be done:

• some more tests
• documentation
• optimizations

Please have a look at branch 4.0.

Be careful with the following points:

• I updated the library layout (requires Arduino IDE 1.0.6 or 1.5.x)
• The README file right now is wrong
• The API has changed significantly (see examples)

👉 Feedback is welcome. Please tell me what you think of it. 👈

[henrikekblad]

Thanks for your effort!
Will provide feedback.
One thing.. we sometimes carry binary data. And you seem to have dug deep into the json specs... Is the best solution to encode it in a string? Any standard we should follow regarding encoding? Base64 is the one coming into mind but it adds quite some some overhead regarding size.

[bblanchon]

I would personally try to avoid binary data in JSON, but if I really needed to then I would go for base64 encoding.

[bblanchon]

I tagged and released Arduino JSON 4.0 beta 1.

Although it's marked as "beta", it's been heavily tested and can safely be used in new projects.

[henrikekblad]

Great work!