Partial deserialization with POJOs, but requires full reserialization

[fabienrenaud]

Hi,

I have a JSON blob I have no idea what its content that might be. It comes from some database and other scripts may change (delete, create, update) lots of keys/objects/arrays at any depth any time.

However, my java job still needs to read that huge and mysterious blob and update a fragment of it.
Problem is that i'd still like to do that with POJOs. But if I use POJOs, data gets loss while converting to the POJO.

Hereafter is a full executable example:

package test;

import com.fasterxml.jackson.annotation.JsonProperty;
import com.fasterxml.jackson.databind.DeserializationFeature;
import com.fasterxml.jackson.databind.JsonNode;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.fasterxml.jackson.databind.SerializationFeature;
import com.fasterxml.jackson.databind.node.ObjectNode;
import java.io.IOException;

public class JsonPojo {

    static final String JSON_INPUT = "{"
            + "\"a1\":{\"b1\":\"value_b1\",\"c1\":\"value_c2\"},"
            + "\"a2\":{\"b2\":\"value_b2\",\"c2\":\"value_c2\"},"
            + "\"a3\":{\"b3\":\"value_b3\",\"c3\":\"value_c3\"}"
            + "}";
    static final String JSON_OUTPUT = "{"
            + "\"a1\":{\"b1\":\"value_b1\",\"c1\":\"value_c2\"},"
            + "\"a2\":{\"b2\":\"value_b2\",\"c2\":\"hello\"},"
            + "\"a3\":{\"b3\":\"value_b3\",\"c3\":\"value_c3\"}"
            + "}";

    public static void main(String[] args) throws IOException {
        new JsonPojo().run();
    }

    void run() throws IOException {
        runWithTree();
        runWithPojo();
    }

    void runWithTree() throws IOException {
        ObjectMapper mapper = new ObjectMapper();
        mapper.enable(SerializationFeature.ORDER_MAP_ENTRIES_BY_KEYS);
        JsonNode node = mapper.readTree(JSON_INPUT);
        ((ObjectNode) node.get("a2")).put("c2", "hello");
        String output = mapper.writeValueAsString(node);
        if (JSON_OUTPUT.equals(output)) {
            System.out.println("runWithTree: OK");
        } else {
            System.out.println("runWithTree: BAD");
        }
    }

    void runWithPojo() throws IOException {
        ObjectMapper mapper = new ObjectMapper();
        mapper.enable(SerializationFeature.ORDER_MAP_ENTRIES_BY_KEYS);
        mapper.disable(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES); // have not choice but to use this to pass
        Pojo pojo = mapper.readValue(JSON_INPUT, Pojo.class);
        pojo.a.c = "hello";
        String output = mapper.writeValueAsString(pojo);
        if (JSON_OUTPUT.equals(output)) {
            System.out.println("runWithPojo: OK");
        } else if ("{\"a2\":{\"c2\":\"hello\"}}".equals(output)) {
            System.out.println("runWithPojo: BAD BUT EXPECTED");
        } else {
            System.out.println("runWithPojo: VERY BAD");
        }
    }

    static class Pojo {

        @JsonProperty("a2")
        private PojoA a;
    }

    static class PojoA {

        @JsonProperty("c2")
        private String c;
    }
}

Note: JSON_INPUT is extremely simplified and there are no patterns of any kind in the real blobs.

Questions:

1. Is that feature already supported by databind or some other module?
2. Could it be supported? (would consist in caching the root JsonNode and do a differential write (to note rewrite the properties that exist in the POJO).

I basically need the complementary feature of readerForUpdating, where i would update a bigger json blob with a smaller pojo.

[cowtowncoder]

I am not sure what exactly is asked here. Tests pass as expected; partial update is supported.

One thing that could help is use of "any-setters"/"any-getters". They are explained in http://www.cowtowncoder.com/blog/archives/2011/07/entry_458.html (for example), and allow you to keep track of additional extra properties.

But in most cases if you want to work with unknown content, JSON Tree (JsonNode, ObjectNode etc) is the way to go.

[fabienrenaud]

Well, no, in the second case, i want to have:
System.out.println("runWithPojo: OK");
not
System.out.println("runWithPojo: BAD BUT EXPECTED");

I don't see why i could not update a json (string) with a partial POJO representation of the full blob...

[cowtowncoder]

I do not understand how it would ever become 'OK' case here: Jackson never modifies JSON Strings, ever. It can construct POJOs from JSON input; and create JSON output from POJOs. In this case class Pojo has a single property, (re)named as a2. It has no other properties; there is nothing for other parts of JSON to go to. So it can not magically appear on output.

To support something similar, you would do well to read the link I pointed you to. It allows you to construct POJOs to return arbitrary set of additional values.

[fabienrenaud]

It could if I could write something like:

JsonNode node = mapper.readTree(fullJsonString);
mapper.writerForUpdating(node).write(pojo);

This is not a bug ticket. It's a "desired feature" ticket ;)

[cowtowncoder]

Ah. Ok, this makes sense. I think there may already be an issue filed for supporting what is basically a "merge". But it may be that it'd be easier to have code merging two JsonNode instances.

Idea of using a JsonNode for creating a JsonGenerator (to overwrite entries) is interesting however.
And if that was implemented, merging of two JsonNodes could just use that facility.

Thank you for explaining this; this sounds like an interesting feature to add.

[fabienrenaud]

And somehow (probably with a proxy), I'd like to have an annotation to deal nicely with this and never see a JsonNode...
Something like:

@JsonCacheTree
public class TinyPojo {  @JsonProperty("i") TinyPojoA a; }
public class TinyPojoA {  @JsonProperty("j") int j; }

void doIt(String hugeBlob) {
  TinyPojo obj = mapper.readValue(hugeBlob, TinyPojo.class);
  obj.a.i++;
  String newHugeBlob = mapper.writeValueAsString(obj);
  assert newHugeBlob.length() == hugeBlob.length() || newHugeBlob.length() == hugeBlob.length() + 1;
}

[cowtowncoder]

I don't think this is possible, since while annotations can indicate presence of extra storage, POJO type itself does not have any. So even if code knew it was to keep track of additional data as metadata, there's no place to stash it in.

[fabienrenaud]

Well, Jackson creates the POJO's instances, so when it meets that annotation, it could create a proxy instead and cache the tree in the invocation handler.

[cowtowncoder]

While I am not sure if it'd be technically doable (perhaps it is), I don't see working on adding such a complex feature in core databind. But if there is enough demand, and someone willing to work on making it happen, it could well be an add-on module or such.

In the meantime, I really think that the explicit way of using any-getter/any-setter is the way to go, if(f) you can modify original container type.

[cowtowncoder]

I think that #1399, ability to do more actual deep merging should work here.
So while I think it's an interesting idea to allow merge on write, I don't see a practical path forward.
May be re-filed with bit more complex suggestion, if necessary.