How to deal with missing keys in from_json?

[dneise]

I am following this example: https://github.com/nlohmann/json#basic-usage

When there is a key missing in the json, inside from_json, I expected to get an exception, but I just get a core_dump.
I would like to avoid writing a lot of boilerplate to check for the existence of keys and type of values myself.
I am mostly a python guy and am used to the try..catch style of doing things. This awesome library looked like it supports this style in this case.

My own type looks like this, lives inside namespace Lid:

struct MotorReport
{
    int8_t motor_id;
    int32_t duration_in_ms;
    int8_t stop_reason;
    std::vector<int> current;
    std::vector<int> position;
};

void to_json(nlohmann::json& j, const MotorReport& r) {
    j = nlohmann::json{
        {"motor_id", r.motor_id},
        {"duration[ms]", r.duration_in_ms},
        {"motor_stop_reason", {
            {"motor_stop_id", r.stop_reason}
        }},
        {"current", r.current},
        {"position", r.position}
    };
}

void from_json(const nlohmann::json& j, MotorReport& r) {
    r.motor_id = j["motor_id"];
    r.duration_in_ms = j["duration[ms]"];
    r.stop_reason = j["motor_stop_reason"]["motor_stop_id"];
    r.current = j["current"].get<std::vector<int> >();
    r.position = j["position"].get<std::vector<int> >();
}

when the json is complete, from_json nicely works.

json j = R"(
     {
          "motor_id":1,
          "duration[ms]":4129,
          "motor_stop_reason":{
              "motor_stop_name":"User Interupt",
              "motor_stop_id":5
          },
          "current":[
              468,181,181,188,219,222,223,223,224,222,220,218,218,218,
              217,214,212,210,209,210,210,208,206,206,206
          ],
          "position":[
              37,37,37,37,36,37,37,37,37,37,36,37,37,
              37,37,37,37,37,37,37,37,36,37,37,37
          ]
      }
)"_json;

Lid::MotorReport mr = j;

When I leave out a necessary key, e.g. motor_id like this, I expected to get a std::domain_error or so.
So I tested this:

json j3 = R"(
     {
          "duration[ms]":4129,
          "motor_stop_reason":{
              "motor_stop_name":"User Interupt",
              "motor_stop_id":5
          },
          "current":[
              468,181,181,188,219,222,223,223,224,222,220,218,218,218,
              217,214,212,210,209,210,210,208,206,206,206
          ],
          "position":[
              37,37,37,37,36,37,37,37,37,37,36,37,37,
              37,37,37,37,37,37,37,37,36,37,37,37
          ]
      }
)"_json;

Lid::MotorReport mr3 = j3;

But I get a core dump instead:

test_json3: json.hpp:4010: const value_type& nlohmann::basic_json<ObjectType, ArrayType, StringType, BooleanType, NumberIntegerType, NumberUnsignedType, NumberFloatType, AllocatorType, JSONSerializer>::operator[](T*) const [with T = const char; ObjectType = std::map; ArrayType = std::vector; StringType = std::__cxx11::basic_string<char>; BooleanType = bool; NumberIntegerType = long int; NumberUnsignedType = long unsigned int; NumberFloatType = double; AllocatorType = std::allocator; JSONSerializer = nlohmann::adl_serializer; nlohmann::basic_json<ObjectType, ArrayType, StringType, BooleanType, NumberIntegerType, NumberUnsignedType, NumberFloatType, AllocatorType, JSONSerializer>::const_reference = const nlohmann::basic_json<>&; nlohmann::basic_json<ObjectType, ArrayType, StringType, BooleanType, NumberIntegerType, NumberUnsignedType, NumberFloatType, AllocatorType, JSONSerializer>::value_type = nlohmann::basic_json<>]: Assertion `m_value.object->find(key) != m_value.object->end()' failed.
Aborted (core dumped)

---

Just to be sure if exception work at all. When trying to access the missing key outside from_json I get a nice exception. So this:

try{
    int myint = j3["motor_id"];
}
catch(...){
    std::clog << boost::current_exception_diagnostic_information() << std::endl;
}

Leads to this:

Dynamic exception type: std::domain_error
std::exception::what: type must be number, but is null

---

I tried putting the same try{ }catch(...){} into from_json, but I still got a core dump and no nice exception I can catch.

So after this long text comes the question: What is your canonical way of dealing with missing keys inside from_json

Thanks a lot for the awesome work.

---

platform Ubuntu 16.04 LTS
gcc version 5.3.1 20160413 (Ubuntu 5.3.1-14ubuntu2.1)
Using the single header json.hpp v.2.1.1

[dneise]

Not sure if this makes any sense (as I'm not a C++ hacker at all), but for the time being I added a
#define NDEBUG, as explained here: http://www.cplusplus.com/reference/cassert/assert/.

This results in a std::domain_error, which I can nicely catch. Although the message of that exception is messed up a little bit then:

Dynamic exception type: std::domain_error
std::exception::what: type must be number, but is number

[theodelrieu]

Hi, I think that in your last example, the non-const operator[] is used, which creates a null object, and thus the conversion from null to int fails, and throws.

The const operator[] crashes, I personally use the at method for that purpose.

I believe there was an issue about those distinct behaviours, we could re-open it and see if we need to change that.

[nlohmann]

Indeed the const operator[] is the issue here. Please use at instead in function from_json. We may need to update the documentation.

[dneise]

Ah yes, thanks Niels. I didn't quite get Thèos reply ...

So now my from_json looks like this:

void from_json(const nlohmann::json& j, MotorReport& r) {
    r.motor_id = j.at("motor_id");
    r.duration_in_ms = j.at("duration[ms]");
    r.stop_reason = j.at("motor_stop_reason").at("motor_stop_id");
    r.current = j.at("current").get<std::vector<int> >();
    r.position = j.at("position").get<std::vector<int> >();
}

And I get a nice:

Dynamic exception type: std::out_of_range
std::exception::what: key 'motor_id' not found

That's perfect! Thanks a lot for your help.
(
If my opinion in this matter counts:
I would update the docu here:
https://github.com/nlohmann/json#basic-usage
)

Apart from that, this issue might be closed.

[nlohmann]

Thanks for the feedback. I shall edit the README accordingly.

[nlohmann]

d07596a added a comment to file README to use at() in from_json rather than operator[].