ESP32 S3 crash on deserializeJson in version 7.2.0

[shm4r7]

Describe the bug
My application crashes when calling deserializeJson - it started to happen after upgrade to version 7.2.0, in version 7.1.0 this code worked as I expected.

Environment
Here is the environment that I used:

• Microcontroller: ESP32S3
• Core/runtime: Arduino v3.0.5
• IDE: VS Code + PlatformIO

Reproduction
Here is a small snippet that reproduces the issue.

#include <Arduino.h>
#include <ArduinoJSON.h>

String getSupportedShmartFeatures()
{
  JsonDocument doc;
  auto config = doc.to<JsonObject>();
  config["CAM"] = bool(SHMART_USE_CAMERA);

  String result;
  serializeJson(config, result);

  return result;
}

JsonDocument shmartMcuInfoDoc;
String shmartMcuInfo;
String prepareESPInfo()
{
  JsonObject obj = shmartMcuInfoDoc.to<JsonObject>();
  auto features = getSupportedShmartFeatures();
  auto featuresObject = obj["features"].to<JsonObject>();
  deserializeJson(featuresObject, features);

  serializeJson(shmartMcuInfoDoc, shmartMcuInfo);

  return shmartMcuInfo;
}

Compiler output
No errors during compilation

Program output
When it crash this is what is logged

assert failed: ArduinoJson::V720PB22::detail::ObjectData& ArduinoJson::V720PB22::detail::VariantData::toObject() VariantData.hpp:537 (type_ == VariantType::Null)

And here is a top stack of decoded esp32 exception:

> Backtrace: 0x403772c2:0x3fca8930 0x4037d2b5:0x3fca8950 0x403833f9:0x3fca8970 0x42003fab:0x3fca8aa0 0x42007232:0x3fca8ac0 0x420072d6:0x3fca8ae0 0x4200b746:0x3fca8b00 0x4200b7ad:0x3fca8ba0 0x4200b7cd:0x3fca8bd0 0x4200b8cd:0x3fca8bf0 0x4200c332:0x3fca8c60 0x42040095:0x3fca8ca0 0x4203f038:0x3fca8cd0 0x4203f6fa:0x3fca8d70 0x4203e3c8:0x3fca8d90 0x420c4386:0x3fca8db0 0x4203e573:0x3fca8dd0 0x4037fcb2:0x3fca8e60
>   #0  0x403772c2 in panic_abort at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/esp_system/panic.c:466
>   #1  0x4037d2b5 in esp_system_abort at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/esp_system/port/esp_system_chip.c:84
>   #2  0x403833f9 in __assert_func at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/newlib/assert.c:81
>   #3  0x42003fab in ArduinoJson::V720PB22::detail::VariantData::toObject() at .pio/libdeps/esp32s3cam/ArduinoJson/src/ArduinoJson/Variant/VariantData.hpp:537 (discriminator 1)
>   #4  0x42007232 in ArduinoJson::V720PB22::DeserializationError::Code ArduinoJson::V720PB22::detail::JsonDeserializer<ArduinoJson::V720PB22::detail::Reader<String, void> >::parseVariant<ArduinoJson::V720PB22::detail::AllowAllFilter>(ArduinoJson::V720PB22::detail::VariantData&, ArduinoJson::V720PB22::detail::AllowAllFilter, ArduinoJson::V720PB22::DeserializationOption::NestingLimit) at .pio/libdeps/esp32s3cam/ArduinoJson/src/ArduinoJson/Json/JsonDeserializer.hpp:80
>   #5  0x420072d6 in ArduinoJson::V720PB22::DeserializationError ArduinoJson::V720PB22::detail::JsonDeserializer<ArduinoJson::V720PB22::detail::Reader<String, void> >::parse<ArduinoJson::V720PB22::detail::AllowAllFilter>(ArduinoJson::V720PB22::detail::VariantData&, ArduinoJson::V720PB22::detail::AllowAllFilter, ArduinoJson::V720PB22::DeserializationOption::NestingLimit) at .pio/libdeps/esp32s3cam/ArduinoJson/src/ArduinoJson/Json/JsonDeserializer.hpp:35
>   #6  0x4200b746 in ArduinoJson::V720PB22::DeserializationError ArduinoJson::V720PB22::detail::doDeserialize<ArduinoJson::V720PB22::detail::JsonDeserializer, ArduinoJson::V720PB22::JsonObject&, ArduinoJson::V720PB22::detail::Reader<String, void>, ArduinoJson::V720PB22::detail::DeserializationOptions<ArduinoJson::V720PB22::detail::AllowAllFilter> >(ArduinoJson::V720PB22::JsonObject&, ArduinoJson::V720PB22::detail::Reader<String, void>, ArduinoJson::V720PB22::detail::DeserializationOptions<ArduinoJson::V720PB22::detail::AllowAllFilter>) at .pio/libdeps/esp32s3cam/ArduinoJson/src/ArduinoJson/Deserialization/deserialize.hpp:57
>   #7  0x4200b7ad in ArduinoJson::V720PB22::DeserializationError ArduinoJson::V720PB22::detail::deserialize<ArduinoJson::V720PB22::detail::JsonDeserializer, ArduinoJson::V720PB22::JsonObject&, String&, , void>(ArduinoJson::V720PB22::JsonObject&, String&) at .pio/libdeps/esp32s3cam/ArduinoJson/src/ArduinoJson/Deserialization/deserialize.hpp:68
>   #8  0x4200b7cd in ArduinoJson::V720PB22::detail::enable_if<ArduinoJson::V720PB22::detail::is_deserialize_destination<ArduinoJson::V720PB22::JsonObject&, void>::value, ArduinoJson::V720PB22::DeserializationError>::type ArduinoJson::V720PB22::deserializeJson<ArduinoJson::V720PB22::JsonObject&, String&>(ArduinoJson::V720PB22::JsonObject&, String&) at .pio/libdeps/esp32s3cam/ArduinoJson/src/ArduinoJson/Json/JsonDeserializer.hpp:705
>   #9  0x4200b8cd in prepareESPInfo() at lib/shmart/ShmartUtils.h:34

Expected output:

deserializeJson should work as it did in version 7.1.0 without throwing exception

[bblanchon]

Hi @shm4r7,

Thank you very much for reporting this bug ❤️

I didn't anticipate this use case, and I would need to think this through.

Allowing the caller to pass a JsonArray or a JsonObject as the first argument of deserializeJson() can lead to a mismatch with the actual input. For example, one could write:

deserializeJson(doc.to<JsonObject>(), "[1,2,3]");

In this case, the program expects an object but receives an array; it's not the situation you are in, but the current API allows it to happen.
ArduinoJson 7.1 was accidentally working because it was missing the required checks.

I see two ways to handle this situation:

1. forbid JsonArray and JsonObject as the first argument
2. return InvalidInput when the input type differs
3. add a new error code

I'm currently leaning toward option 1 as it's simpler to implement.
Besides, option 2 doesn't bring anything interesting and might be confusing.
Option 3 is excluded: I don't want to increase the library size for such a marginal use case.

Meanwhile, pass a JsonVariant (or a MemberProxy) instead:

  String prepareESPInfo()
  {
    JsonObject obj = shmartMcuInfoDoc.to<JsonObject>();
    auto features = getSupportedShmartFeatures();
-   auto featuresObject = obj["features"].to<JsonObject>();
-   deserializeJson(featuresObject, features);
+   deserializeJson(obj["features"], features);
 
    serializeJson(shmartMcuInfoDoc, shmartMcuInfo);

    return shmartMcuInfo;
  }

Best regards,
Benoit

[shm4r7]

Hi @bblanchon,

thank you for detailed explanation, It seems that I unnecesarily complicated thing calling .to<JsonObject>(). I agree option 1 sounds good, I would prefer to find it at compilation time rather than handle during runtime :)

Regards

[bblanchon]

The fix is available in ArduinoJson 7.2.1