JSON Pointer

[miloyip]

Recently I have looked at several possible add-ons to RapidJSON (post v1.0), including those already proposed, and also some JSON related (pre-) standards. I found that the newest JSON schema requires JSON Pointer. While digging into JSON Pointer (RFC6901), I found that it should be simple and may be useful.

I tried to implement a draft version in json-pointer branch:

• pointer.h
• pointertest.cpp

I would like to seek comments on this.

Usage

There is only one template class GenericPointer. As often it is typedef as Pointer.

#include "rapidjson/pointer.h"

// ...
Document d;

// Create DOM by Set()
Pointer("/project").Set(d, "RapidJSON");
Pointer("/stars").Set(d, 10);
// { "project" : "RapidJSON", "stars" : 10 }

// Access DOM by Get()
if (Value* stars = Pointer("/stars").Get(d))  // return nullptr if the Pointer is not exist.
    stars->SetInt(stars->GetInt() + 1);
// { "project" : "RapidJSON", "stars" : 11 }

// Set() and Create() automatically generate parents if not exist.
Pointer("/a/b/0").Create(d);
// { "project" : "RapidJSON", "stars" : 11, "a" : { "b" : [ null ] } }

// GetWithDefault() returns reference instead of pointer. And it deep clones the default value.
Value& hello = Pointer("/hello").GetWithDefault(d, "world");
// { "project" : "RapidJSON", "stars" : 11, "a" : { "b" : [ null ] }, "hello" : "world" }

// Swap() is similar to Set()
Value x("C++");
Pointer("/hello").Swap(d, x);
// { "project" : "RapidJSON", "stars" : 11, "a" : { "b" : [ null ] }, "hello" : "C++" }
// x becomes "world"

Design

Pointer parses a JSON path into tokens. It involves escaping characters and creating the tokens array.
If a pointer is applied multiple times, it should be construct once, and then apply it to different DOMs or in different times. This is similar to regex.

However, if the path can be statically constructed in compile-time. I try to make this possible without dynamic allocations and parsing overhead with this usage:

// Construct a Pointer with static tokens, no dynamic allocation involved.
#define NAME(s) { s, sizeof(s) / sizeof(s[0]) - 1, kPointerInvalidIndex }
#define INDEX(i) { #i, sizeof(#i) - 1, i }

static const Pointer::Token kTokens[] = { NAME("foo"), INDEX(0) }; // equivalent to "/foo/0"

#undef NAME
#undef INDEX

Pointer p(kTokens, sizeof(kTokens) / sizeof(kTokens[0]));

StringBuffer s;
p.Stringify(s); // converts token back to JSON pointer
// s.GetString() becomes "/foo/0"

Helper functions

As the parameter order of function calls is a little bit strange: pointer.method(root, ...), I tried to add some helper functions to make the API more intuitive. The function signatures are methodValueByPointer(root, pointer, ...).

The pointer parameter can be either a literal string or Pointer instance.

The example at the beginning can be converted as:

Document d;

SetValueByPointer(d, "/project", "RapidJSON");
SetValueByPointer(d, "/stars", 10);
// { "project" : "RapidJSON", "stars" : 10 }

if (Value* stars = GetValueByPointer(d, "/stars"))  // return nullptr if the Pointer is not exist.
    stars->SetInt(stars->GetInt() + 1);
// { "project" : "RapidJSON", "stars" : 11 }

CreateValueByPointer(d, "/a/b/0");
// { "project" : "RapidJSON", "stars" : 11, "a" : { "b" : [ null ] } }

Value& hello = GetValueByPointerWithDefault(d, "/hello", "world");
// { "project" : "RapidJSON", "stars" : 11, "a" : { "b" : [ null ] }, "hello" : "world" }

Value x("C++");
SwapValueByPointer(d, "/hello", x);
// { "project" : "RapidJSON", "stars" : 11, "a" : { "b" : [ null ] }, "hello" : "C++" }
// x becomes "world"

Why Supporting JSON Pointer?

1. It can simplify some coding for manipulation with the DOM. Especially when there are multiple levels of objects/arrays, JSON pointer can do it in single calls, with single checking for existence. This may resolve issue Default Value? #151 and usage improvement #229.
2. it basically does not affect the current API. (Just added a GenericValue::ValueType typedef)
3. It is useful for accessing DOM by data-driven (not hard-code). For example, a JSON may contains JSON pointers which refers to the structure of itself or other JSONs. JSON Schema does this.
4. It is already a RFC, not a draft.
5. It is a simpler version of JSON Path. It can be a reference for implementing JSON Path, if we want to.

Improvements

(to be updated from discussions)

• Template type for Set() as in Value::AddMember() and Value::PushBack(). So setting primitive type can be simpler.
• Overloads for document root parameter. It uses document's allocator so it does not need an allocator parameter.
• Parsing error handling. Parse error is stored in Pointer.
• Resolving performance related to FindMember() as in Member query performance #102
• Handling of single - character for index, specified in RFC 6901 p.3
• URI fragment
• std::string overloads
• Documentation

Welcome for all suggestions.

[miloyip]

Hi @pah,

I faced a problem with StringRef in Pointer's API. Would like to have you advice.

At the beginning, I would like to differentiate a string literal and a string pointer, but fail to do so. For example,

ValueType& Set(ValueType& root, GenericStringRef<Ch> value, typename ValueType::AllocatorType& allocator) const {
    ValueType v(value);
    return Create(root, allocator) = v;
}

ValueType& Set(ValueType& root, const Ch* value, typename ValueType::AllocatorType& allocator) const {
    ValueType v(value, allocator);
    return Create(root, allocator) = v;
}

p.Set(d, "test", a) will resolve to the second API but not the first one. I checked Value's constructors uses allocator parameter to differentiate two APIs. Is there any way to solve the above problem with same number of parameters?

Currently I just remove the first one. That means, always clone a string with the allocator.

[pah]

Avoiding the second overload to be the better match would require adding the same set of functions as for the construction of GenericStringRef itself. This is probably not worth the complexity.

Performance-critical code could explicitly call

p.Set(d, StringRef("test"), a); 

instead to mark the string as constant.