Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Serialize big data in json #1828

Closed
Roman-Koshelev opened this issue Nov 6, 2019 · 7 comments
Closed

Serialize big data in json #1828

Roman-Koshelev opened this issue Nov 6, 2019 · 7 comments
Labels
kind: question solution: proposed fix a fix for the issue has been proposed and waits for confirmation

Comments

@Roman-Koshelev
Copy link

For example, I want to add logged events to a JSON file and close it when I exit the program. I would also like to add them to a file as they become available.

What should I do to I make this work?

@ilqvya
Copy link
Contributor

ilqvya commented Nov 7, 2019

nop, json currently load ALL json into memory. It's a big feature to store part of json in memory depending on file pointers :)

@nlohmann
Copy link
Owner

nlohmann commented Nov 8, 2019

You can use the SAX interface to avoid parsing everything to memory. See https://nlohmann.github.io/json/classnlohmann_1_1basic__json_a8a3dd150c2d1f0df3502d937de0871db.html#a8a3dd150c2d1f0df3502d937de0871db for more information.

@pbialas7
Copy link

But can you use SAX to write not parse json to file?

@ilqvya
Copy link
Contributor

ilqvya commented Nov 16, 2019

@nlohmann can i use it with memory mapped file (if i have just pointer to data and its size) ?

@nlohmann
Copy link
Owner

But can you use SAX to write not parse json to file?

No this is not possible in the moment.

@nlohmann
Copy link
Owner

@nlohmann can i use it with memory mapped file (if i have just pointer to data and its size) ?

Yes, you can do this with

json j = json::parse({data, size});

@nlohmann nlohmann added the solution: proposed fix a fix for the issue has been proposed and waits for confirmation label Nov 22, 2019
@nlohmann
Copy link
Owner

Do you need further assistance with this issue?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind: question solution: proposed fix a fix for the issue has been proposed and waits for confirmation
Projects
None yet
Development

No branches or pull requests

4 participants