This is the third chapter of the Kotlin Serialization Guide. In this chapter we'll take a look at serializers in more detail and see how custom serializers can be written.
Table of contents
- Introduction to serializers
- Custom serializers
- Primitive serializer
- Composite serializer via surrogate
- Hand-written composite serializer
- Sequential decoding protocol (experimental)
- Serializing 3rd party classes
- Passing a serializer manually
- Specifying serializer on a property
- Specifying serializers for a file
- Custom serializers for a generic type
- Format-specific serializers
- Contextual serialization
- Deriving external serializer for another Kotlin class (experimental)
Format, like JSON, controls the encoding of an object into specific output bytes, but how the object is decomposed
into its constituent properties is controlled by a serializer. So far, we've been using automatically-derived
serializers using the @Serializable annotation as explained in
the Serializable classes section or builtin serializers that were shown in
the Builtin classes section.
As a motivating example, let us take the following Color class with an integer value storing its rgb bytes.
@Serializable
class Color(val rgb: Int)
fun main() {
val green = Color(0x00ff00)
println(Json.encodeToString(green))
} You can get the full code here.
By default, this class serializes its rgb property into JSON.
{"rgb":65280}
Every class marked with the @Serializable annotation, like the Color class from the previous example,
gets an instance of the KSerializer interface automatically generated by the Kotlin serialization compiler plugin.
We can retrieve this instance using .serializer() function on the class's companion object.
We can examine its descriptor property that describes the structure of the serialized class. We'll learn more details about it in the upcoming sections.
fun main() {
val colorSerializer: KSerializer<Color> = Color.serializer()
println(colorSerializer.descriptor)
} You can get the full code here.
Color(rgb: kotlin.Int)
This serializer is automatically retrieved and used by the Kotlin serialization framework when the Color class
serialized on the top-level or used as a property of other classes.
You cannot define your own function
serializer()on a companion object of a serializable class.
For generic classes, like the Box class shown in the Generic classes section,
the automatically generated .serializer() function accepts as many parameters as there are type-parameters in the
corresponding class. These parameters are of type KSerializer, so the actual type argument's serializer has
to be provided when constructing an instance of a serializer for a generic class.
@Serializable
@SerialName("Box")
class Box<T>(val contents: T)
fun main() {
val boxedColorSerializer = Box.serializer(Color.serializer())
println(boxedColorSerializer.descriptor)
} You can get the full code here.
As we can see, a serializer was instantiated to serialize a concrete Box<Color>.
Box(contents: Color)
The serializers for the primitive builtin classes can be retrieved
using .serializer() extensions.
fun main() {
val intSerializer: KSerializer<Int> = Int.serializer()
println(intSerializer.descriptor)
}You can get the full code here.
Builtin collection serializers, when needed, must be explicitly constructed
using the corresponding functions ListSerializer(), SetSerializer(), MapSerializer(), etc.
These classes are generic, so to instantiate their serializer we must provide the serializers for the
corresponding number of their type parameters.
For example, we can produce a serializer for a List<String> in the following way.
fun main() {
val stringListSerializer: KSerializer<List<String>> = ListSerializer(String.serializer())
println(stringListSerializer.descriptor)
}You can get the full code here.
When in doubt, you can always use the top-level generic serializer<T>()
function to retrieve a serializer for an arbitrary Kotlin type in your source-code.
@Serializable
@SerialName("Color")
class Color(val rgb: Int)
fun main() {
val stringToColorMapSerializer: KSerializer<Map<String, Color>> = serializer()
println(stringToColorMapSerializer.descriptor)
}You can get the full code here.
A plugin-generated serializer is convenient, but it may not do what we want to see in JSON
for such a class as Color, so let's study alternatives.
We want to serialize the Color class as a hex string with green color represented as "00ff00".
To achieve this, we write an object that implements the KSerializer interface for the Color class.
object ColorAsStringSerializer : KSerializer<Color> {
override val descriptor: SerialDescriptor = PrimitiveSerialDescriptor("Color", PrimitiveKind.STRING)
override fun serialize(encoder: Encoder, value: Color) {
val string = value.rgb.toString(16).padStart(6, '0')
encoder.encodeString(string)
}
override fun deserialize(decoder: Decoder): Color {
val string = decoder.decodeString()
return Color(string.toInt(16))
}
}Serializer has three required pieces.
-
The serialize function implements SerializationStrategy. It receives an instance of Encoder and a value to serialize. It uses
encodeXxxfunctions ofEncoderto represent a value as a sequence of primitives. There is anencodeXxxfor each primitive type supported by serialization. In our example, encodeString is used. -
The deserialize function implements DeserializationStrategy. It receives an instance of Decoder and returns a deserialized value. It uses
decodeXxxfunctions ofDecoderthat mirror the corresponding functions ofEncoder. In our example, decodeString is used. -
The descriptor property must faithfully explain what exactly
encodeXxxanddecodeXxxfunctions do so that a format implementation knows in advance what encoding/decoding methods they call. Some formats might also use it to generate a schema of the serialized data. For primitive serialization, the PrimitiveSerialDescriptor function must be used with a unique name of the type that is being serialized. PrimitiveKind describes the specificencodeXxx/decodeXxxmethod that is being used in the implementation.
When the
descriptordoes not correspond to the encoding/decoding methods, then the behavior of the resulting code is unspecified and may arbitrary change in futures updates.
The next step is to bind serializer to a class. This is done with the @Serializableannotation by adding
the with property value.
@Serializable(with = ColorAsStringSerializer::class)
class Color(val rgb: Int)Now we can serialize Color class as we did before.
fun main() {
val green = Color(0x00ff00)
println(Json.encodeToString(green))
} You can get the full code here.
We get serial representation as a hex string that we wanted.
"00ff00"
Deserialization is also straightforward, because we implemented deserialize method.
@Serializable(with = ColorAsStringSerializer::class)
class Color(val rgb: Int)
fun main() {
val color = Json.decodeFromString<Color>("\"00ff00\"")
println(color.rgb) // prints 65280
} You can get the full code here.
It also works if we serialize or deserialize a different class with Color properties.
@Serializable(with = ColorAsStringSerializer::class)
data class Color(val rgb: Int)
@Serializable
data class Settings(val background: Color, val foreground: Color)
fun main() {
val data = Settings(Color(0xffffff), Color(0))
val string = Json.encodeToString(data)
println(string)
require(Json.decodeFromString<Settings>(string) == data)
} You can get the full code here.
Both Color properties are serialized as strings.
{"background":"ffffff","foreground":"000000"}
Now our challenge is to get Color serialized so that it is represented in JSON as if it is a class
with three properties: r, g, and b, so that JSON encodes it as an object.
The easiest way to achieve it is to define a surrogate class mimicking the serialized form of the Color that
we are going to use for its serialization. We also set SerialName of this surrogate class to Color, so
that if a format uses this name, the surrogate looks like it is a Color class.
The surrogate class can be private and can enforce all the constraints on the serial representation
of the class in its init block.
@Serializable
@SerialName("Color")
private class ColorSurrogate(val r: Int, val g: Int, val b: Int) {
init {
require(r in 0..255 && g in 0..255 && b in 0..255)
}
}An example of where the class name is used is shown in the Custom subclass serial name section in the chapter on polymorphism.
Now we can use a ColorSurrogate.serializer() function to retrieve a plugin-generated serializer for the
surrogate class.
An implementation of KSerializer for our original Color class is going to perform a conversion between
Color and ColorSurrogate, but delegate the actual serialization logic to the ColorSurrogate.serializer()
using encodeSerializableValue and
decodeSerializableValue, fully reusing an automatically
generated SerialDescriptor for the surrogate.
object ColorSerializer : KSerializer<Color> {
override val descriptor: SerialDescriptor = ColorSurrogate.serializer().descriptor
override fun serialize(encoder: Encoder, value: Color) {
val surrogate = ColorSurrogate((value.rgb shr 16) and 0xff, (value.rgb shr 8) and 0xff, value.rgb and 0xff)
encoder.encodeSerializableValue(ColorSurrogate.serializer(), surrogate)
}
override fun deserialize(decoder: Decoder): Color {
val surrogate = decoder.decodeSerializableValue(ColorSurrogate.serializer())
return Color((surrogate.r shl 16) or (surrogate.g shl 8) or surrogate.b)
}
}Bind ColorSerializer serializer to a Color class.
@Serializable(with = ColorSerializer::class)
class Color(val rgb: Int)Now we can enjoy the result of serialization of the Color class.
You can get the full code here.
{"r":0,"g":255,"b":0}
There are some cases where a surrogate solution does not fit. It can be due to performance reasons of avoiding additional allocation or where a configurable/dynamic set of properties of the resulting serial representation is required. In this case, we need to manually write a class serializer that mimics the behaviour of a serializer that is generated for a class.
object ColorAsObjectSerializer : KSerializer<Color> {Let's introduce it piece by piece. First, a descriptor is defined using buildClassSerialDescriptor builder. The element function in the builder DSL automatically fetches serializers for the corresponding fields by their type. The order of elements is important. They are indexed starting from zero.
override val descriptor: SerialDescriptor =
buildClassSerialDescriptor("Color") {
element<Int>("r")
element<Int>("g")
element<Int>("b")
}The "element" is a generic term here. What is an element of a descriptor depends of its SerialKind. Elements of a class descriptor are its properties, elements of a enum descriptor are its cases, etc.
Then we write the serialize function using encodeStructure DSL that provides access to
the CompositeEncoder in its block. The difference between Encoder and CompositeEncoder is that the latter
has encodeXxxElement functions that correspond to encodeXxx functions of the former. They must be called
in the same order as in the descriptor.
override fun serialize(encoder: Encoder, value: Color) =
encoder.encodeStructure(descriptor) {
encodeIntElement(descriptor, 0, (value.rgb shr 16) and 0xff)
encodeIntElement(descriptor, 1, (value.rgb shr 8) and 0xff)
encodeIntElement(descriptor, 2, value.rgb and 0xff)
}The most complex piece of code is the deserialize function. It shall support formats, like JSON, that
can decode properties in an arbitrary order. It starts with the call to decodeStructure to
get access to a CompositeDecoder. Inside of it, we write a loop that repeatedly calls
decodeElementIndex to decode the index of the next element, decode the corresponding
element using decodeIntElement in our example, and terminate a loop when
CompositeDecoder.DECODE_DONE is encountered.
override fun deserialize(decoder: Decoder): Color =
decoder.decodeStructure(descriptor) {
var r = -1
var g = -1
var b = -1
while(true) {
when (val index = decodeElementIndex(descriptor)) {
0 -> r = decodeIntElement(descriptor, 0)
1 -> g = decodeIntElement(descriptor, 1)
2 -> b = decodeIntElement(descriptor, 2)
CompositeDecoder.DECODE_DONE -> break
else -> error("Unexpected index: $index")
}
}
require(r in 0..255 && g in 0..255 && b in 0..255)
Color((r shl 16) or (g shl 8) or b)
}Bind the resulting serializer to the Color class and test its serialization/deserialization.
@Serializable(with = ColorAsObjectSerializer::class)
data class Color(val rgb: Int)
fun main() {
val color = Color(0x00ff00)
val string = Json.encodeToString(color)
println(string)
require(Json.decodeFromString<Color>(string) == color)
} You can get the full code here.
As before, we got the Color class represented as a JSON object with three keys:
{"r":0,"g":255,"b":0}
The implementation of the deserialize function from the previous section works with any format. However,
some formats either always store all the complex data in order or do it sometimes (for example, JSON always stores
collections in order). With these formats, the complex protocol of calling decodeElementIndex in the loop is
not needed and a faster implementation can be used if the CompositeDecoder.decodeSequentially function returns true.
The plugin-generated serializers are actually conceptually similar to the below code.
override fun deserialize(decoder: Decoder): Color =
decoder.decodeStructure(descriptor) {
var r = -1
var g = -1
var b = -1
if (decodeSequentially()) { // sequential decoding protocol
r = decodeIntElement(descriptor, 0)
g = decodeIntElement(descriptor, 1)
b = decodeIntElement(descriptor, 2)
} else while(true) {
when (val index = decodeElementIndex(descriptor)) {
0 -> r = decodeIntElement(descriptor, 0)
1 -> g = decodeIntElement(descriptor, 1)
2 -> b = decodeIntElement(descriptor, 2)
CompositeDecoder.DECODE_DONE -> break
else -> error("Unexpected index: $index")
}
}
require(r in 0..255 && g in 0..255 && b in 0..255)
Color((r shl 16) or (g shl 8) or b)
}You can get the full code here.
Sometimes an application has to work with an external type that is not serializable.
Let us use java.util.Date as an example. As before, we start by writing an implementation for KSerializer
for the class. Our goal is to get a Date serialized as the long number of milliseconds following the
approach from the Primitive serializer section.
In the following sections any kind of
Dateserializer would work. For example, if we wantDateto be serialized as an object, we would use an approach from the Composite serializer via surrogate section.
See also Deriving external serializer for another Kotlin class (experimental) when you need to serialize a 3rd-party Kotlin class which could have been serializable, but is not.
object DateAsLongSerializer : KSerializer<Date> {
override val descriptor: SerialDescriptor = PrimitiveSerialDescriptor("Date", PrimitiveKind.LONG)
override fun serialize(encoder: Encoder, value: Date) = encoder.encodeLong(value.time)
override fun deserialize(decoder: Decoder): Date = Date(decoder.decodeLong())
}We cannot bind the DateAsLongSerializer serializer to the Date class with the @Serializable annotation,
because we don't control the Date source code. There are several ways to work around that.
All encodeToXxx and decodeFromXxx functions have an overload with the first serializer parameter.
When a non-serializable class, like Date, is the top-level class being serialized we can use those.
fun main() {
val kotlin10ReleaseDate = SimpleDateFormat("yyyy-MM-ddX").parse("2016-02-15+00")
println(Json.encodeToString(DateAsLongSerializer, kotlin10ReleaseDate))
}You can get the full code here.
1455494400000
When a property of a non-serializable class, like Date, is serialized as part of a serializable class we must supply
its serializer, or the code does not compile. This is accomplished using the @Serializable annotation on a property.
@Serializable
class ProgrammingLanguage(
val name: String,
@Serializable(with = DateAsLongSerializer::class)
val stableReleaseDate: Date
)
fun main() {
val data = ProgrammingLanguage("Kotlin", SimpleDateFormat("yyyy-MM-ddX").parse("2016-02-15+00"))
println(Json.encodeToString(data))
}You can get the full code here.
The stableReleaseDate property is serialized with the serialization strategy that we specified for it:
{"name":"Kotlin","stableReleaseDate":1455494400000}
A serializer for a specific type, like Date, can be specified for a whole source code file with the file-level
UseSerializers annotation at the beginning of the file.
@file:UseSerializers(DateAsLongSerializer::class)Now a Date property can be used in a serializable class without additional annotations.
@Serializable
class ProgrammingLanguage(val name: String, val stableReleaseDate: Date)
fun main() {
val data = ProgrammingLanguage("Kotlin", SimpleDateFormat("yyyy-MM-ddX").parse("2016-02-15+00"))
println(Json.encodeToString(data))
}You can get the full code here.
{"name":"Kotlin","stableReleaseDate":1455494400000}
Let us take a look at the following example of the generic Box<T> class.
It is marked with @Serializable(with = BoxSerializer::class) as we plan to have a custom serialization
strategy for it.
@Serializable(with = BoxSerializer::class)
data class Box<T>(val contents: T) An implementation of KSerializer for a regular type is written as an object as we saw in this chapter
in the examples for the Color type. A generic class serializer is instantiated with serializers
for its generic parameters as we saw in the Plugin-generated generic serializer section.
A custom serializer for a generic class must be a class with a constructor that accepts as many KSerializer
parameters as the type has generic parameters. Let us write a Box<T> serializer that erases itself during
serialization, delegating everything to the underlying serializer of its data property.
class BoxSerializer<T>(private val dataSerializer: KSerializer<T>) : KSerializer<Box<T>> {
override val descriptor: SerialDescriptor = dataSerializer.descriptor
override fun serialize(encoder: Encoder, value: Box<T>) = dataSerializer.serialize(encoder, value.contents)
override fun deserialize(decoder: Decoder) = Box(dataSerializer.deserialize(decoder))
}Now we can serialize and deserialize Box<Project>.
@Serializable
data class Project(val name: String)
fun main() {
val box = Box(Project("kotlinx.serialization"))
val string = Json.encodeToString(box)
println(string)
println(Json.decodeFromString<Box<Project>>(string))
}You can get the full code here.
The resulting JSON looks like the Project class was serialized directly.
{"name":"kotlinx.serialization"}
Box(contents=Project(name=kotlinx.serialization))
The above custom serializers worked in the same way for every format. However, there might be format-specific features that a serializer implementation would like to take advantage of.
-
The Json transformations section of the Json chapter provides examples of serializers that utilize JSON-specific features.
-
A format implementation can have a format-specific representation for a type as explained in the Format-specific types section of the Alternative and custom formats (experimental) chapter.
This chapter proceeds to a generic approach of tweaking serialization strategy based on the context.
All the previous approaches to specifying the custom serialization strategies that we saw were static,
fully defined at compile-time. The exception was the Passing a serializer manually
approach, but it worked only on a top-level object that is being serialized. You might need to change the serialization
strategy of objects deep in the serialized object tree at run-time so that it is selected in a context-dependent way.
For example, you might want to represent java.util.Date in JSON format as an ISO 6801 string or as a long integer
depending on a version of a protocol you are serializing data for. This is called contextual serialization and
is supported by a built-in ContextualSerializer class. Usually, we don't have to use this serializer class explicitly
as there is the Contextual annotation providing a shortcut to
the @Serializable(with = ContextualSerializer::class) annotation
and the UseContextualSerialization annotation that can be used on a file-level just like
the UseSerializers annotation. Let's see an example utilizing the former.
@Serializable
class ProgrammingLanguage(
val name: String,
@Contextual
val stableReleaseDate: Date
)To actually serialize this class we must provide the corresponding context when calling encodeToXxx/decodeFromXxx
functions. Without it we'll get "Serializer for class 'Date' is not found" exception.
See here for the example that produces exception.
To provide a context, we define a SerializersModule instance that describes which serializers shall be used
at run-time to serialize which contextually-serializable classes. This is done using the
SerializersModule {} builder function, which provides SerializersModuleBuilder DSL to
register serializers. In the below example we use contextual function with the serializer. The corresponding
class this serializer is defined for is fetched automatically via the reified type parameter.
private val module = SerializersModule {
contextual(DateAsLongSerializer)
}Next we create an instance of Json format with this serializers module using Json {} builder function and the serializersModule property.
Details on custom JSON configurations can be found in the JSON configuration section.
val format = Json { serializersModule = module }Now we can serialize our data with this format.
fun main() {
val data = ProgrammingLanguage("Kotlin", SimpleDateFormat("yyyy-MM-ddX").parse("2016-02-15+00"))
println(format.encodeToString(data))
}You can get the full code here.
{"name":"Kotlin","stableReleaseDate":1455494400000}
Additional details on serialization modules are given in the Merging library serializers modules section of the Polymorphism chapter.
If a 3rd-party class to be serialized is a Kotlin class with properties-only primary constructor, a kind of
class which could have been made @Serializable, then you can generate an external serializer for it
using the Serializer annotation on an object with forClass property.
// NOT @Serializable
class Project(val name: String, val language: String)
@Serializer(forClass = Project::class)
object ProjectSerializerYou must all bind this serializer to a class using one of the approaches explained in this chapter. We'll follow the Passing a serializer manually way for this example.
fun main() {
val data = Project("kotlinx.serialization", "Kotlin")
println(Json.encodeToString(ProjectSerializer, data))
}You can get the full code here.
This gets all the Project properties serialized:
{"name":"kotlinx.serialization","language":"Kotlin"}
As we saw earlier, the regular @Serializer annotation creates a serializer so that
Backing fields are serialized. External serialization using
Serializer(forClass = ...) has no access to backing fields and works differently.
It serializes only accessible properties that have setter or are a part of the primary constructor.
The following example shows it.
// NOT @Serializable, will use external serializer
class Project(
// val in a primary constructor -- serialized
val name: String
) {
var stars: Int = 0 // property with getter & setter -- serialized
val path: String // getter only -- not serialized
get() = "kotlin/$name"
private var locked: Boolean = false // private, not accessible -- not serialized
}
@Serializer(forClass = Project::class)
object ProjectSerializer
fun main() {
val data = Project("kotlinx.serialization").apply { stars = 9000 }
println(Json.encodeToString(ProjectSerializer, data))
}You can get the full code here.
The output is shown below.
{"name":"kotlinx.serialization","stars":9000}
The next chapter covers Polymorphism.