Global std::vector from json

[Xillez]

Hello!

I've noticed that this is a recurring issue and have been looking thought closed and open issues for any solutions, but for some reason getting a vector containing a custom type from json throws "[json.exception.type_error.302] type must be array, but is null" which is based on exception.what()'s output.

I have this JSON file:

[
   {
      "date":"20200116",
      "id":1589,
      "path":"D:\\_projects\\ChickenIdentifier\\P3-1589_L_20200116.jpg",
      "penName":"P3",
      "side":76
   },
   {
      "date":"20200116",
      "id":1589,
      "path":"D:\\_projects\\ChickenIdentifier\\P3-1589_R_20200116.jpg",
      "penName":"P3",
      "side":82
   },
   {
      "date":"20200116",
      "id":1592,
      "path":"D:\\_projects\\ChickenIdentifier\\P3-1592_L_20200116.jpg",
      "penName":"P3",
      "side":76
   },
   {
      "date":"20200116",
      "id":1592,
      "path":"D:\\_projects\\ChickenIdentifier\\P3-1592_R_20200116.jpg",
      "penName":"P3",
      "side":82
   }
]

What I've tried:

• auto vec = j.get<std::vector<PictureInfo>>(); <-- Produces the same error
• j.at("varName").get_to<std::vector<PictureInfo>>(c.required); <-- Dont think I can use since list doesnt have a name/key.

• auto pics = j.get>();  //<-- Produces the same error
  

std::vector pics = (std::vector) pics;

Im using Win10 and MS Visual Studio Community 2019

Lib version: "JSON for Modern C++ version 3.7.3"

Appreciate any help and please let me know if anything else is needed

[nlohmann]

Did you implement a from_json function for type PictureInfo?

[Xillez]

yes and i have validated the lib can find em as "to_json" is called.

void from_json(const nlohmann::json& j, PictureInfo& imgInfo)
{
    j.at("id").get_to(imgInfo.id);
    j.at("side").get_to(imgInfo.side);
    j.at("penName").get_to(imgInfo.penName);
    j.at("date").get_to(imgInfo.date);
    imgInfo.path.assign(j.at("path"));
    imgInfo.img = cv::imread(imgInfo.path.string(), cv::IMREAD_COLOR); // Read image file

    if (imgInfo.img.empty()) // Check for invalid input
    {
        std::cout << "Could not open or failed to be read" << std::endl;
    }
}

[nlohmann]

Does converting individual elements from the array work? For instance

PictureInfo pi = j[0];

[Xillez]

Haven't tested that, will try and see

[Xillez]

I just realized that a laps in my copy-paste has occurred, I was opening a file that doesn't exist but still get the same error only that it finds strings instead.

btw here's my current code for loading the file:

    std::ifstream in("filename");
    nlohmann::json j;
    if (in.is_open())
        in >> j;
    try
    {
        Public::files = j.get>();
    }
    catch (const std::exception & e)
    {
        std::cout << e.what() << std::endl;
    }

but I need to test later im afraid

[nlohmann]

Calling get without type seems wrong. It should be sufficient to say Public::files = j;.

Nonetheless, there should not be such an exception. Please try for a single element, and provide a stacktrace if possible.

[Xillez]

Yes, just markdown that thinks the template signs are tags and removes em as they're not complete. Supposed to be a std:vector with PictureInfo, anyways will indeed check once i get home.

[nlohmann]

Well in general, the conversion should work:

#include <iostream>
#include "json.hpp"

using json = nlohmann::json;

struct widget
{
    int i;
    std::string s;
};

void from_json(const nlohmann::json& j, widget& w)
{
    w.i = j["i"];
    w.s = j["s"];
}

int main()
{
    json j = R"( [ {"i": 1, "s": "one"}, {"i": 2, "s": "two"} ] )"_json;
    std::vector<widget> wv = j;
    
    for (auto& w : wv)
    {
        std::cout << w.i << " -> " << w.s << std::endl;
    }
}

Output:

1 -> one
2 -> two

[Xillez]

I get compile error: "more than one operator "=" matches these operands" and " 'operator =' is ambiguous"

With: Public::files = j;

[nlohmann]

What type is Public::files? What is the exact error message? Could you provide example code? It's hard to diagnose like this...

[Xillez]

Public::files is static inline member of "Public" class, and has type: std::vector<PictureInfo>

Error:
"more than one operator "=" matches these operands:
function "std::vector<_Ty, _Alloc>::operator=(std::vector<_Ty, _Alloc> &&_Right) [with _Ty=PictureInfo, _Alloc=std::allocator]"
function "std::vector<_Ty, _Alloc>::operator=(std::initializer_list<_Ty> _Ilist) [with _Ty=PictureInfo, _Alloc=std::allocator]"
operand types are: std::vector<PictureInfo, std::allocator> = nlohmann::json"

AND

" 'operator =' is ambiguous"

So there's multiple "operators=" that could be used in std::vector so it complains on which to use.

void Utils::loadData()   <--- NOTE! Simply called in main()
{
    std::ifstream in(Public::crestDataFilename);
    nlohmann::json j;
    if (in.is_open())
        in >> j;
    Public::files = j;   <-- NOTE! Error location
}

PictureInfo.hpp:

class PictureInfo
{
public:
    cv::Mat img;
    int id;
    char side;
    std::string penName;
    std::string date;
    sf::path path;

    PictureInfo();
    PictureInfo(sf::path path, std::string filename);
protected:
    //
private:
    //
};

void to_json(nlohmann::json& j, const PictureInfo& imgInfo);

void from_json(const nlohmann::json& j, PictureInfo& imgInfo);

PictureInfo.cpp:

PictureInfo::PictureInfo()
{}

PictureInfo::PictureInfo(sf::path path, std::string filename)
{
    ...
}

void to_json(nlohmann::json& j, const PictureInfo& imgInfo)
{
    j = nlohmann::json{
        { "id", imgInfo.id }, 
        { "side", imgInfo.side }, 
        { "penName", imgInfo.penName }, 
        { "date", imgInfo.date }, 
        { "path", imgInfo.path.string() }
    };
}

void from_json(const nlohmann::json& j, PictureInfo& imgInfo)
{
    j.at("id").get_to(imgInfo.id);
    j.at("side").get_to(imgInfo.side);
    j.at("penName").get_to(imgInfo.penName);
    j.at("date").get_to(imgInfo.date);
    imgInfo.path.assign(j.at("path"));
    imgInfo.img = cv::imread(imgInfo.path.string(), cv::IMREAD_COLOR); // Read image file

    if (imgInfo.img.empty()) // Check for invalid input
    {
        std::cout << "Could not open or failed to be read" << std::endl;
    }
}

[nlohmann]

Thanks. So does converting a single element work (see #2108 (comment))? And does anything change if you replace

Public::files = j;

by

std::vector<PictureInfo> foo = j;

[Xillez]

I get this either way: "[json.exception.type_error.302] type must be array, but is string"

std::vector temp = j;
Public::files = temp;

OR

Public::files.push_back(j[0]);

Maybe I have to make my own serializer?

[nlohmann]

Do you have a stacktrace for the exception? Maybe you just expect the wrong data type in the converter.

[Xillez]

ChickenIdentifier.exe!`Utils::loadData'::`1'::catch$16() Line 86	C++
[External Code]	
[Inline Frame] ChickenIdentifier.exe!nlohmann::detail::from_json(const nlohmann::basic_json &) Line 3116	C++
[Inline Frame] ChickenIdentifier.exe!nlohmann::detail::from_json_fn::operator()(const nlohmann::basic_json &) Line 3248	C++
[Inline Frame] ChickenIdentifier.exe!nlohmann::adl_serializer>,void>::from_json(const nlohmann::basic_json &) Line 3814	C++
[Inline Frame] ChickenIdentifier.exe!nlohmann::basic_json::get() Line 17186	C++
[Inline Frame] ChickenIdentifier.exe!nlohmann::basic_json::operator class std::vector>() Line 17479	C++
ChickenIdentifier.exe!Utils::loadData() Line 81	C++
ChickenIdentifier.exe!main(int argc, char * * argv) Line 18	C++
[External Code]	

[nlohmann]

What is the last line in your code?

[Xillez]

Not sure if I understood by "last lin in your code" but, exception comes from std::vector<PictureInfo> temp = j; which is in Utils::loadData().

void Utils::loadData()
{
    std::ifstream in(Public::crestDataFilename);
    nlohmann::json j;
    if (in.is_open())
        in >> j;

    try
    {
        std::vector temp = j;
        Public::files = temp;
    }
    catch (const std::exception & e)
    {
        std::cout << e.what() << std::endl;
    }
}

[nlohmann]

Yes, I meant that. It would be interesting to step into this. I do not know how to help here.

[Xillez]

I can give you a copy of the project if you wanna tinker with it. Whats your email?

[nlohmann]

mail@nlohmann.me

[Xillez]

Thanks for the help though and sorry if I confused you along the way!

[nlohmann]

Sorry, I'm running macOS. Can't build the sources. As last idea: please use the debugger and step through the code. Maybe you just expect the wrong data type in the converter.

[Xillez]

Will do indeed. You have a windows vm tho?

[nlohmann]

No, and I cannot spend more time on this issue. Please try to provide a small working example, but try the debugger first.

[Xillez]

yep will do, and thanks for your time!

[Xillez]

Found the error. Its failing in the construction of PictureInfo. It defaults to chararray instead of string which is what std::filesysem::path.assign() takes so it complains? Im not entirely sure, but a direct conversion from j.at("path") to std::string did the trick!

void from_json(const nlohmann::json& j, PictureInfo& imgInfo)
{
    j.at("id").get_to(imgInfo.id);
    j.at("side").get_to(imgInfo.side);
    j.at("penName").get_to(imgInfo.penName);
    j.at("date").get_to(imgInfo.date);
    imgInfo.path.assign((std::string) j.at("path"));   //<--- Convertion here from chararray to std::string
    imgInfo.img = cv::imread(imgInfo.path.string(), cv::IMREAD_COLOR); // Read image file

    if (imgInfo.img.empty()) // Check for invalid input
    {
        std::cout << "Could not open or failed to be read" << std::endl;
    }
}

[nlohmann]

Thanks for reporting back!