Serialize and deserialize complex object graphs to JSON
Serializr is a utility library that helps converting json structures into complex object graphs and the other way around. For a quick overview, read the introduction blog post
Features:
- (De)serialize objects created with a constructor / class
- (De)serialize primitive values
- (De)serialize nested objects, maps and arrays
- Resolve references asynchronously (during deserialization)
- Supports inheritance
- Works on any ES5 environment (if ES3 is needed file a feature request)
- Convenience decorators for ESNext / Typescript
- Ships with typescript / flow typings
- Generic solution that works well with for example MobX out of the box
Non-features:
- Serializr is not an ORM or data management library. It doesn't manage object instances, provided api's like fetch, search etc. If you are building such a thing though, serializr might definitely take care of the serialization part for you :-).
- Serializr is not a MobX specific (de)serialization mechanism, it is generic and should fit work with any type of model objects
From npm: npm install serializr --save
From CDN: https://unpkg.com/serializr which declares the global serializr object.
import {
createModelSchema, primitive, reference, list, object, identifier, serialize, deserialize
} from "serializr";
// Example model classes
class User {
uuid = Math.floor(Math.random()*10000);
displayName = "John Doe";
}
class Message {
message = "Test";
author = null;
comments = [];
}
function fetchUserSomewhere(uuid) {
// Lets pretend to actually fetch a user; but not.
// In a real app this might be a database query
const user = new User();
user.uuid = uuid;
user.displayName = `John Doe ${uuid}`;
return user;
}
function findUserById(uuid, callback, context) {
// This is a lookup function
// identifier is the identifier being resolved
// callback is a node style calblack function to be invoked with the found object (as second arg) or an error (first arg)
// context is an object detailing the execution context of the serializer now
callback(null, fetchUserSomewhere(uuid))
}
// Create model schemas
createModelSchema(Message, {
message : primitive(),
author : reference(User, findUserById),
comments: list(object(Message))
});
createModelSchema(User, {
uuid : identifier(),
displayName: primitive()
});
// can now deserialize and serialize!
const message = deserialize(Message, {
message : "Hello world",
author : 17,
comments: [
{
message: "Welcome!",
author : 23
}
]
});
const json = serialize(message);
console.dir(message, {colors: true, depth: 10});With decorators (TypeScript or ESNext) building model schemas is even more trivial:
import {
createModelSchema, primitive, reference, list, object, identifier, serialize, deserialize, getDefaultModelSchema,
serializable
} from "serializr";
class User {
@serializable(identifier())
uuid = Math.random();
@serializable
displayName = "John Doe";
}
class Message {
@serializable
message = "Test";
@serializable(object(User))
author = null;
// Self referencing decorators work in Babel 5.x and Typescript. See below for more.
@serializable(list(object(Message)))
comments = [];
}
// You can now deserialize and serialize!
const message = deserialize(Message, {
message : "Hello world",
author : {uuid: 1, displayName: "Alice"},
comments: [
{
message: "Welcome!",
author : {uuid: 1, displayName: "Bob"}
}
]
});
console.dir(message, {colors: true, depth: 10});
// We can call serialize without the first argument here
//because the schema can be inferred from the decorated classes
const json = serialize(message);Decorator: Caveats
Babel 6.x does not allow decorators to self-reference during their creation, so the above code would not work for the Message class. Instead write:
class Message {
@serializable
message = "Test";
@serializable(object(User))
author = null;
comments = [];
constructor(){
getDefaultModelSchema(Message).props["comments"] = list(object(Message));
}
} TypeScript
Enable the compiler option experimentalDecorators in tsconfig.json or pass it as flag --experimentalDecorators to the compiler.
Babel 5.x
{
"stage": 1
}Babel 6.x:
Install support for decorators: npm i --save-dev babel-plugin-transform-decorators-legacy. And enable it in your .babelrc file:
{
"presets": ["es2015", "stage-1"],
"plugins": ["transform-decorators-legacy"]
}Probably you have more plugins and presets in your .babelrc already, note that the order is important and transform-decorators-legacy should come as first.
The two most important functions exposed by serializr are serialize(modelschema?, object) -> json tree and deserialize(modelschema, json tree) -> object graph.
What are those model schemas?
The driving concept behind (de)serialization is a ModelSchema. It describes how model object instances can be (de)serialize to json.
A simple model schema looks like this:
const todoSchema = {
factory: (context) => new Todo(),
extends: ModelSchema,
props: {
modelfield: PropSchema
}
}The factory tells how to construct new instances during deserialization.
The optional extends property denotes that this model schema inherits its props from another model schema.
The props section describes how individual model properties are to be (de)serialized. Their names match the model field names.
The combination fieldname: true is simply a shorthand for fieldname: primitive()
For convenience, model schemas can be stored on the constructor function of a class. This allows you to pass in a class reference wherever a model schema is required. See the examples below.
PropSchemas contain the strategy on how individual fields should be serialized. It denotes whether a field is a primitive, list, whether it needs to be aliased, refers to other model objects etc. PropSchemas are composable. See the API section below for the details, but these are the built-in property schemas:
primitive(): Serialize a field as primitive valueidentifier(): Serialize a field as primitive value, use it as identifier when serializing references (seereference)date(): Serializes dates (as epoch number)alias(name, propSchema): Serializes a field under a different namelist(propSchema): Serializes an array based collectionmap(propSchema): Serializes an Map or string key based collectionobject(modelSchema): Serializes an child model elementreference(modelSchema, lookupFunction?): Serializes a reference to another model elementcustom(serializeFunction, deserializeFunction): Create your own property serializer by providing two functions, one that converts modelValue to jsonValue, and one that does the inverse- There is a special prop schema:
"*": truethat serializes all enumerable, non mentioned values as primitive
It is possible to define your own prop schemas. You can define your own propSchema by creating a function that returns an object with the following signature:
{
serializer: (sourcePropertyValue: any) => jsonValue,
deserializer: (jsonValue: any, callback: (err, targetPropertyValue: any) => void, context?, currentPropertyValue?) => void
}
For inspiration, take a look at the source code of the existing ones on how they work, it is pretty straightforward.
The context object is an advanced feature and can be used to obtain additional context-related information about the deserialization process.
context is available as:
- first argument of factory functions
- third argument of the lookup callback of
refprop schema's (see below) - third argument of the
deserializerof a custom propSchema
When deserializing a model elememt / property, the following fields are available on the context object:
json: Returns the complete current json object that is being deserializedtarget: The object currently being deserialized. This is the object that is returned from the factory function.parentContext: Returns the parent context of the current context. For example if a child element is being deserialized, thecontext.targetrefers to the current model object, andcontext.parentContext.targetrefers to the parent model object that owns the current model object.args: If custom arguments were passed to thedeserialize/updatefunction, they are available ascontext.args.
JSDOC type defintions for usage w/o typescript.
Parameters
valueAnytargetClassget(Function | undefined)set(Function | undefined)configurablebooleanenumerablebooleansourcePropertyValueAnyjsonValueAnycallbackcpsCallbackcontextContextwriteablebooleanidAnytargetobjectcontextContextresultAnyerrorAnyidstringcallbackcpsCallbackfactorypropscurrentPropertyValueAny
Properties
serializerserializerFunctiondeserializerdeserializerFunctionidentifierboolean
Returns Any any - serialized object
Returns Any void
Returns Any void
Returns Any void
Creates a model schema that (de)serializes from / to plain javascript objects.
Its factory method is: () => ({})
Parameters
propsobject property mapping,
Examples
var todoSchema = createSimpleSchema({
title: true,
done: true
});
var json = serialize(todoSchema, { title: "Test", done: false })
var todo = deserialize(todoSchema, json)Returns object model schema
Creates a model schema that (de)serializes an object created by a constructor function (class).
The created model schema is associated by the targeted type as default model schema, see setDefaultModelSchema.
Its factory method is () => new clazz() (unless overriden, see third arg).
Parameters
clazz(constructor | class) class or constructor functionpropsobject property mappingfactoryfunction optional custom factory. Receives context as first arg
Examples
function Todo(title, done) {
this.title = title;
this.done = done;
}
createModelSchema(Todo, {
title: true,
done: true
})
var json = serialize(new Todo("Test", false))
var todo = deserialize(Todo, json)Returns object model schema
Decorator that defines a new property mapping on the default model schema for the class it is used in.
When using typescript, the decorator can also be used on fields declared as constructor arguments (using the private / protected / public keywords).
The default factory will then invoke the constructor with the correct arguments as well.
Parameters
arg1arg2arg3
Examples
class Todo {Returns PropertyDescriptor
Returns the standard model schema associated with a class / constructor function
Parameters
thingobject
Returns ModelSchema model schema
Sets the default model schema for class / constructor function.
Everywhere where a model schema is required as argument, this class / constructor function
can be passed in as well (for example when using object or ref.
When passing an instance of this class to serialize, it is not required to pass the model schema
as first argument anymore, because the default schema will be inferred from the instance type.
Parameters
clazz(constructor | class) class or constructor functionmodelSchemaModelSchema a model schema
Returns ModelSchema model schema
Serializes an object (graph) into json using the provided model schema. The model schema can be omitted if the object type has a default model schema associated with it. If a list of objects is provided, they should have an uniform type.
Parameters
arg1modelschema to use. Optionalarg2object(s) to serialize
Returns object serialized representation of the object
Deserializes a json structor into an object graph. This process might be asynchronous (for example if there are references with an asynchronous lookup function). The function returns an object (or array of objects), but the returned object might be incomplete until the callback has fired as well (which might happen immediately)
Parameters
schema(object | array) to use for deserializationjsonjson data to deserializecallbackfunction node style callback that is invoked once the deserializaiton has finished. First argument is the optional error, second argument is the deserialized object (same as the return value)customArgsAny custom arguments that are available ascontext.argsduring the deserialization process. This can be used as dependency injection mechanism to pass in, for example, stores.
Returns (object | array) deserialized object, possibly incomplete.
Similar to deserialize, but updates an existing object instance. Properties will always updated entirely, but properties not present in the json will be kept as is. Further this method behaves similar to deserialize.
Parameters
modelSchemaobject , optional if it can be inferred from the instance typetargetobject target instance to updatejsonobject the json to deserializecallbackfunction the callback to invoke once deserialization has completed.customArgsAny custom arguments that are available ascontext.argsduring the deserialization process. This can be used as dependency injection mechanism to pass in, for example, stores.
Indicates that this field contains a primitive value (or Date) which should be serialized literally to json.
Examples
createModelSchema(Todo, {
title: primitive()
})
console.dir(serialize(new Todo("test")))
// outputs: { title : "test" }Returns ModelSchema
Similar to primitive, but this field will be marked as the identifier for the given Model type.
This is used by for example reference() to serialize the reference
Identifier accepts an optional registerFn with the signature:
(id, target, context) => void
that can be used to register this object in some store. note that not all fields of this object might
have been deserialized yet.
Parameters
registerFnRegisterFunction optional function to register this object during creation.
Examples
var todos = {};
var s = _.createSimpleSchema({
id: _.identifier((id, object) => todos[id] = object),
title: true
})
_.deserialize(s, {
id: 1, title: "test0"
})
_.deserialize(s, [
{ id: 2, title: "test2" },
{ id: 1, title: "test1" }
])
t.deepEqual(todos, {
1: { id: 1, title: "test1" },
2: { id: 2, title: "test2" }
})Returns PropSchema
Similar to primitive, serializes instances of Date objects
Alias indicates that this model property should be named differently in the generated json. Alias should be the outermost propschema.
Parameters
namestring name of the json field to be used for this propertypropSchemaPropSchema propSchema to (de)serialize the contents of this field
Examples
createModelSchema(Todo, {
title: alias("task", primitive())
})
console.dir(serialize(new Todo("test")))
// { task : "test" }Returns PropSchema
Can be used to create simple custom propSchema.
Parameters
serializerfunction function that takes a model value and turns it into a json valuedeserializerfunction function that takes a json value and turns it into a model value
Examples
var schema = _.createSimpleSchema({
a: _.custom(
function(v) { return v + 2 },
function(v) { return v - 2 }
)
})
t.deepEqual(_.serialize(s, { a: 4 }), { a: 6 })
t.deepEqual(_.deserialize(s, { a: 6 }), { a: 4 })Returns PropSchema
object indicates that this property contains an object that needs to be (de)serialized
using its own model schema.
N.B. mind issues with circular dependencies when importing model schema's from other files! The module resolve algorithm might expose classes before createModelSchema is executed for the target class.
Parameters
modelSchemaModelSchema to be used to (de)serialize the object
Examples
class SubTask{}
class Todo{}
createModelSchema(SubTask, {
title: true
});
createModelSchema(Todo, {
title: true,
subTask: object(SubTask)
});
const todo = deserialize(Todo, {
title: "Task",
subTask: {
title: "Sub task"
}
});Returns PropSchema
reference can be used to (de)serialize references that point to other models.
The first parameter should be either a ModelSchema that has an identifier() property (see identifier)
or a string that represents which attribute in the target object represents the identifier of the object.
The second parameter is a lookup function that is invoked during deserialization to resolve an identifier to an object. Its signature should be as follows:
lookupFunction(identifier, callback, context) where:
identifieris the identifier being resolvedcallbackis a node style calblack function to be invoked with the found object (as second arg) or an error (first arg)contextsee context.
The lookupFunction is optional. If it is not provided, it will try to find an object of the expected type and required identifier within the same JSON document
N.B. mind issues with circular dependencies when importing model schemas from other files! The module resolve algorithm might expose classes before createModelSchema is executed for the target class.
Parameters
target: ModelSchema or stringlookupFnRefLookupFunction function
Examples
class User{}
class Post{}
createModelSchema(User, {
uuid: identifier(),
displayname: primitive()
})
createModelSchema(Post, {
author: reference(User, findUserById),
message: primitive()
})
function findUserById(uuid, callback) {
fetch("http://host/user/" + uuid)
.then((userData) => {
deserialize(User, userData, callback)
})
.catch(callback)
}
deserialize(
Post,
{
message: "Hello World",
author: 234
},
(err, post) => {
console.log(post)
}
)Returns PropSchema
List indicates that this property contains a list of things. Accepts a sub model schema to serialize the contents
Parameters
propSchemaPropSchema to be used to (de)serialize the contents of the array
Examples
class SubTask{}
class Task{}
class Todo{}
createModelSchema(SubTask, {
title: true
})
createModelSchema(Todo, {
title: true,
subTask: list(object(SubTask))
})
const todo = deserialize(Todo, {
title: "Task",
subTask: [{
title: "Sub task 1"
}]
})Returns PropSchema
Similar to list, but map represents a string keyed dynamic collection. This can be both plain objects (default) or ES6 Map like structures. This will be inferred from the initial value of the targetted attribute.
Parameters
propSchemaAny
const todoSchema = {
factory: () => {},
props: {
task: primitive(),
owner: reference("_userId", UserStore.findUserById) // attribute of the owner attribute of a todo + lookup function
subTasks: alias("children", list(object(todoSchema)))
}
}
const todo = deserialize(todoSchema,
{ task: "grab coffee", owner: 17, children: [] },
(err, todo) => { console.log("finished loading todos") }
);
const todoJson = serialize(todoSchema, todo)function Todo(parentTodo) {
this.parent = parentTodo; // available in subTasks
}
const todoSchema = {
factory: (context) => new Todo(context.parent),
props: {
task: primitive(),
owner: reference("_userId", UserStore.findUserById) // attribute of the owner attribute of a todo + lookup function
subTasks: alias("children", list(object(todoSchema)))
}
}
setDefaultModelSchema(Todo, todoSchema)
const todo = deserialize(Todo, // just pass the constructor name, schema will be picked up
{ task: "grab coffee", owner: 17, children: [] },
(err, todos) => { console.log("finished loading todos") }
);
const todoJson = serialize(todo) // no need to pass schema explicitlyfunction Todo() {
}
// creates a default factory, () => new Todo(), stores the schema as default model schema
createModelSchema(Todo, {
task: primitive()
})
const todo = deserialize(Todo, // just pass the constructor name, schema will be picked up
{ task: "grab coffee", owner: 17, children: [] },
(err, todos) => { console.log("finished loading todos") }
);
const todoJson = serialize(todo) // no need to pass schema explicitlyclass Todo {
@serializable(primitive())
task = "Grab coffee";
@serializable(reference("_userId", UserStore.findUserById))
owner = null;
@serializable(alias("children", list(object(todoSchema)))
subTasks = [];
}
// note that (de)serialize also accepts lists
const todos = deserialize(Todo,
[{
task: "grab coffee", owner: 17, children: []
}],
(err, todos) => { console.log("finished loading todos") }
);
const todoJson = serialize(todos)const someTodoStoreById = {}
getDefaultModelSchema(Todo).factory = (context, json) => {
if (someTodoStoreById[json.id])
return someTodoStoreById[json.id] // reuse instance
return someTodoStoreById[json.id] = new Todo()
};This pattern is useful to avoid singletons but allow to pass context specific data to constructors. This can be done by passing custom data to deserialize / update as last argument,
which will be available as context.args on all places where context is available:
class User {
constructor(someStore) {
// User needs access to someStore, for whatever reason
}
}
// create model schema with custom factory
createModelSchema(User, { username: true }, context => {
return new User(context.args.someStore)
})
// don't want singletons!
const someStore = new SomeStore()
// provide somestore through context of the deserialization process
const user = deserialize(
User,
someJson,
(err, user) => { console.log("done") },
{
someStore: someStore
}
)// models.js:
import {observable, computed} from 'mobx';
import {serializable, identifier} from 'serializr';
function randomId() {
return Math.floor(Math.random()*100000);
}
export class Box {
@serializable(identifier()) id = randomId();
@serializable @observable x = 0;
@serializable @observable y = 0;
@serializable @observable location = 0;
constructor(location, x, y){
this.location = location;
this.x = x;
this.y = y;
}
@serializable @computed get area() {
return this.x * this.y;
}
}
export class Arrow{
@serializable(identifier()) id = randomId();
@serializable(reference(Box)) from;
@serializable(reference(Box)) to;
}
// store.js:
import {observable, transaction} from 'mobx';
import {createSimpleSchema, identifier, list, serialize, deserialize, update} from 'serializr';
import {Box, Arrow} from './models';
// The store that holds our domain: boxes and arrows
const store = observable({
boxes: [],
arrows: [],
selection: null
});
// Model of the store itself
const storeModel = createSimpleSchema({
boxes: list(object(Box)),
arrows: list(object(Arrow)),
selection: reference(Box)
});
// Example Data
// You can push data in as a class
store.boxes.push(
new Box('Rotterdam', 100, 100),
new Box('Vienna', 650, 300)
);
// Or it can be an raw javascript object with the right properties
store.arrows.push({
id: randomId(),
from: store.boxes[0],
to: store.boxes[1]
});
// (de) serialize functions
function serializeState(store) {
return serialize(storeModel, store);
}
function deserializeState(store, json) {
transaction(() => {
update(storeModel, store, json);
})
}
// Print ... out for debugging
console.dir(serializeState(store), {depth: 10, colors: true});- If MobX, optimize by leveraging createTransformer and transactions
- Support async serialization (future)
- Support ImmutableJS out of the box
- Make
"*": truerespect extends clauses