Skip to content

Latest commit

 

History

History
879 lines (668 loc) · 34.6 KB

json.md

File metadata and controls

879 lines (668 loc) · 34.6 KB
Raw

JSON features

This is the fifth chapter of the Kotlin Serialization Guide. In this chapter we'll walk through various Json features.

Table of contents

Json configuration

By default, Json implementation is quite strict with respect to invalid inputs, enforces Kotlin type safety, and restricts Kotlin values that can be serialized so that the resulting JSON representations are standard. Many non-standard JSON features are supported by creating a custom instance of a JSON format.

JSON format configuration can be specified by creating your own Json class instance using an existing instance, such as a default Json object, and a Json() builder function. Additional parameters are specified in a block via JsonBuilder DSL. The resulting Json format instance is immutable and thread-safe; it can be simply stored in a top-level property.

It is recommended to store and reuse custom instances of formats for performance reasons as format implementations may cache format-specific additional information about the classes they serialize.

This chapter shows various configuration features that Json supports.

Pretty printing

JSON can be configured to pretty print the output by setting the prettyPrint property.

val format = Json { prettyPrint = true }

@Serializable 
data class Project(val name: String, val language: String)

fun main() {                                      
    val data = Project("kotlinx.serialization", "Kotlin")
    println(format.encodeToString(data))
}

You can get the full code here.

It gives the following nice result.

{
    "name": "kotlinx.serialization",
    "language": "Kotlin"
}

Lenient parsing

By default, Json parser enforces various JSON restrictions to be as specification-compliant as possible (see RFC-4627). Keys must be quoted, literals shall be unquoted. Those restrictions can be relaxed with the isLenient property. With isLenient = true we can parse quite freely-formatted data.

val format = Json { isLenient = true }

enum class Status { SUPPORTED }                                                     

@Serializable 
data class Project(val name: String, val status: Status, val votes: Int)
    
fun main() {             
    val data = format.decodeFromString<Project>("""
        { 
            name   : kotlinx.serialization,
            status : SUPPORTED,
            votes  : "9000"
        }
    """)
    println(data)
}

You can get the full code here.

We get the object, even though all keys, string and enum values are unquoted, while an integer was quoted.

Project(name=kotlinx.serialization, status=SUPPORTED, votes=9000)

Ignoring unknown keys

JSON format is often used to read the output of 3rd-party services or in otherwise highly-dynamic environment where new properties could be added as a part of API evolution. By default, unknown keys encountered during deserialization produces an error. This behavior can be configured with the ignoreUnknownKeys property.

val format = Json { ignoreUnknownKeys = true }

@Serializable 
data class Project(val name: String)
    
fun main() {             
    val data = format.decodeFromString<Project>("""
        {"name":"kotlinx.serialization","language":"Kotlin"}
    """)
    println(data)
}

You can get the full code here.

It decodes the object, despite the fact that it is missing the language property.

Project(name=kotlinx.serialization)

Coercing input values

JSON formats that are encountered in the wild can be flexible in terms of types and evolve quickly. This can lead to exceptions during decoding when the actual values do not match the expected values. By default Json implementation is strict with respect to input types as was demonstrated in the Type safety is enforced section. It can be somewhat relaxed using the coerceInputValues property.

This property only affects decoding. It treats a limited subset of invalid input values as if the corresponding property was missing and uses a default value of the corresponding property instead. The current list of supported invalid values is:

  • null inputs for non-nullable types.
  • Unknown values for enums.

This list may be expanded in the future, so that Json instance configured with this property becomes even more permissive to invalid value in the input, replacing them with defaults.

Let us take the example from the Type safety is enforced section.

val format = Json { coerceInputValues = true }

@Serializable 
data class Project(val name: String, val language: String = "Kotlin")

fun main() {
    val data = format.decodeFromString<Project>("""
        {"name":"kotlinx.serialization","language":null}
    """)
    println(data)
}

You can get the full code here.

We see that invalid null value for the language property was coerced into the default value.

Project(name=kotlinx.serialization, language=Kotlin)

Encoding defaults

Default values of properties don't have to be encoded, because they will be reconstructed during encoding anyway. It can be configured by the encodeDefaults property. This is especially useful for nullable properties with null defaults to avoid writing the corresponding null values.

val format = Json { encodeDefaults = false }

@Serializable 
class Project(
    val name: String, 
    val language: String = "Kotlin",
    val website: String? = null
)           

fun main() {
    val data = Project("kotlinx.serialization")
    println(format.encodeToString(data))
}

You can get the full code here.

Produces the following output which has only the name property:

{"name":"kotlinx.serialization"}

Allowing structured map keys

JSON format does not natively support the concept of a map with structured keys. Keys in JSON objects are strings and can be used to represent only primitives or enums by default. Non-standard support for structured keys can be enabled with the allowStructuredMapKeys property.

val format = Json { allowStructuredMapKeys = true }

@Serializable 
data class Project(val name: String)
    
fun main() {             
    val map = mapOf(
        Project("kotlinx.serialization") to "Serialization",
        Project("kotlinx.coroutines") to "Coroutines"
    )
    println(format.encodeToString(map))
}

You can get the full code here.

The map with structured keys gets represented as [key1, value1, key2, value2,...] JSON array.

[{"name":"kotlinx.serialization"},"Serialization",{"name":"kotlinx.coroutines"},"Coroutines"]

Allowing special floating-point values

By default, special floating-point values like Double.NaN and infinities are not supported in JSON, because the JSON specification prohibits it. But they can be enabled using the allowSpecialFloatingPointValues property.

val format = Json { allowSpecialFloatingPointValues = true }

@Serializable
class Data(
    val value: Double
)                     

fun main() {
    val data = Data(Double.NaN)
    println(format.encodeToString(data))
}

You can get the full code here.

This example produces the following non-stardard JSON output, yet it is a widely used encoding for special values in JVM world.

{"value":NaN}

Class discriminator

A key name that specifies a type when you have a polymorphic data can be specified with the classDiscriminator property.

val format = Json { classDiscriminator = "#class" }

@Serializable
sealed class Project {
    abstract val name: String
}
            
@Serializable         
@SerialName("owned")
class OwnedProject(override val name: String, val owner: String) : Project()

fun main() {
    val data: Project = OwnedProject("kotlinx.coroutines", "kotlin")
    println(format.encodeToString(data))
}

You can get the full code here.

In combination with an explicitly specified SerialName of the class it provides full control on the resulting JSON object.

{"#class":"owned","name":"kotlinx.coroutines","owner":"kotlin"}

Json elements

So far, we've been working with JSON format by converting objects to strings and back. However, JSON is often so flexible in practice that you might need to tweak the data before it can parse or otherwise work with such an unstructured data that it does not readily fit into the typesafe world of Kotlin serialization.

Parsing to Json element

A string can parsed into an instance of JsonElement with the Json.parseToJsonElement function. It is called neither decoding nor deserialization, because none of that happens in the process. Only JSON parser is being used here.

fun main() {
    val element = Json.parseToJsonElement("""
        {"name":"kotlinx.serialization","language":"Kotlin"}
    """)
    println(element)
}

You can get the full code here.

A JsonElement prints itself as a valid JSON.

{"name":"kotlinx.serialization","language":"Kotlin"}

Subtypes of Json elements

A JsonElement class has three direct subtypes, closely following JSON grammar.

  • JsonPrimitive represents all primitive JSON elements, such as string, number, boolean, and null. Each primitive has a simple string content. There is also a JsonPrimitive() constructor function overloaded to accept various primitive Kotlin types and to convert them to JsonPrimitive.

  • JsonArray represents a JSON [...] array. It is a Kotlin List of JsonElement.

  • JsonObject represents a JSON {...} object. It is a Kotlin Map from String key to JsonElement value.

The JsonElement class has jsonXxx extensions that cast it to its corresponding subtypes (jsonPrimitive, jsonArray, jsonObject). The JsonPrimitive class, in turn, has convenient converters to Kotlin primitive types (int, intOrNull, long, longOrNull, etc) that allow fluent code to work with JSON for which you know the structure of.

fun main() {
    val element = Json.parseToJsonElement("""
        {
            "name": "kotlinx.serialization",
            "forks": [{"votes": 42}, {"votes": 9000}, {}]
        }
    """)
    val sum = element
        .jsonObject["forks"]!!
        .jsonArray.sumOf { it.jsonObject["votes"]?.jsonPrimitive?.int ?: 0 }
    println(sum)
}

You can get the full code here.

The above example sums votes in all objects in the forks array, ignoring the objects that have no votes, but failing if the structure of the data is otherwise different.

9042

Json element builders

We can construct instances of specific JsonElement subtypes using the respective builder functions buildJsonArray and buildJsonObject. They provide a DSL to define the resulting structure that is similar to Kotlin standard library collection builders, but with some added JSON-specific convenience of more type-specific overloads and inner builder functions. The following example shows all the key features.

fun main() {
    val element = buildJsonObject {
        put("name", "kotlinx.serialization")
        putJsonObject("owner") {
            put("name", "kotlin")
        }
        putJsonArray("forks") {
            addJsonObject {
                put("votes", 42)
            }
            addJsonObject {
                put("votes", 9000)
            }
        }
    }
    println(element)
}

You can get the full code here.

At the end, we get a proper JSON string.

{"name":"kotlinx.serialization","owner":{"name":"kotlin"},"forks":[{"votes":42},{"votes":9000}]}

Decoding Json element

An instance of the JsonElement class can be decoded into a serializable object using the Json.decodeFromJsonElement function.

@Serializable 
data class Project(val name: String, val language: String)

fun main() {
    val element = buildJsonObject {
        put("name", "kotlinx.serialization")
        put("language", "Kotlin")
    }
    val data = Json.decodeFromJsonElement<Project>(element)
    println(data)
}

You can get the full code here.

The result is exactly what we would expect.

Project(name=kotlinx.serialization, language=Kotlin)

Json transformations

To affect the shape and contents of JSON output after serialization, or adapt input to deserialization, it is possible to write a custom serializer. However, it may not be convenient to carefully follow Encoder and Decoder calling conventions, especially for relatively small and easy tasks. For that purpose, Kotlin serialization provides an API that can reduce the burden of implementing a custom serializer to a problem of manipulating a Json elements tree.

You are still strongly advised to become familiar with the Serializers chapter, as it explains, among other things, how custom serializers are bound to classes.

Transformation capabilities are provided by the abstract JsonTransformingSerializer class which implements KSerializer. Instead of direct interaction with Encoder or Decoder, this class asks you to supply transformations for JSON tree represented by the JsonElement class using the transformSerialize and transformDeserialize methods. Let us take a look at the examples.

Array wrapping

The first example is our own implementation of JSON array wrapping for lists. Consider a REST API that returns a JSON array of User objects, or, if there is only one element in the result, then it is a single object, not wrapped into an array. In our data model, we use @Serializable annotation to specify a custom serializer for a users: List<User> property.

@Serializable 
data class Project(
    val name: String,
    @Serializable(with = UserListSerializer::class)      
    val users: List<User>
)

@Serializable
data class User(val name: String)

For now, we are only concerned with deserialization, so we implement UserListSerializer and override only the transformDeserialize function. The JsonTransformingSerializer constructor takes an original serializer as parameter and here we use the approach from the Constructing collection serializers section to create one.

object UserListSerializer : JsonTransformingSerializer<List<User>>(ListSerializer(User.serializer())) {
    // If response is not an array, then it is a single object that should be wrapped into the array
    override fun transformDeserialize(element: JsonElement): JsonElement =
        if (element !is JsonArray) JsonArray(listOf(element)) else element
}

Now we can test our code with a JSON array or a single JSON object as inputs.

fun main() {     
    println(Json.decodeFromString<Project>("""
        {"name":"kotlinx.serialization","users":{"name":"kotlin"}}
    """))
    println(Json.decodeFromString<Project>("""
        {"name":"kotlinx.serialization","users":[{"name":"kotlin"},{"name":"jetbrains"}]}
    """))
}

You can get the full code here.

The output shows that both cases are correctly deserialized into a Kotlin List.

Project(name=kotlinx.serialization, users=[User(name=kotlin)])
Project(name=kotlinx.serialization, users=[User(name=kotlin), User(name=jetbrains)])

Array unwrapping

We can also implement the transformSerialize function to unwrap a single-element list into a single JSON object during serialization.

    override fun transformSerialize(element: JsonElement): JsonElement {
        require(element is JsonArray) // we are using this serializer with lists only
        return element.singleOrNull() ?: element
    }

Now, when we start with a single-element list of objects in Kotlin.

fun main() {     
    val data = Project("kotlinx.serialization", listOf(User("kotlin")))
    println(Json.encodeToString(data))
}

You can get the full code here.

We end up with a single JSON object.

{"name":"kotlinx.serialization","users":{"name":"kotlin"}}

Manipulating default values

Another kind of useful transformation is omitting specific values from the output JSON, e.g. because it is treated as default when missing or for any other domain-specific reasons.

Suppose that our Project data model cannot specify a default value for the language property, but it has to omitted from the JSON when it is equal to Kotlin (we can all agree that Kotlin should be default anyway). We'll fix it by writing the special ProjectSerializer based on the Plugin-generated serializer for the Project class.

@Serializable
class Project(val name: String, val language: String)

object ProjectSerializer : JsonTransformingSerializer<Project>(Project.serializer()) {
    override fun transformSerialize(element: JsonElement): JsonElement =
        // Filter out top-level key value pair with the key "language" and the value "Kotlin"
        JsonObject(element.jsonObject.filterNot {
            (k, v) -> k == "language" && v.jsonPrimitive.content == "Kotlin"
        })
}

In the example below, we are serializing the Project class at the top-level, so we explicitly pass the above ProjectSerializer to Json.encodeToString function as was shown in the Passing a serializer manually section.

fun main() {
    val data = Project("kotlinx.serialization", "Kotlin")
    println(Json.encodeToString(data)) // using plugin-generated serializer
    println(Json.encodeToString(ProjectSerializer, data)) // using custom serializer
}

You can get the full code here.

We can clearly see the effect of the custom serializer.

{"name":"kotlinx.serialization","language":"Kotlin"}
{"name":"kotlinx.serialization"}

Content-based polymorphic deserialization

Typically, polymorphic serialization requires a dedicated "type" key (also known as class discriminator) in the incoming JSON object to determine the actual serializer which should be used to deserialize Kotlin class.

However, sometimes type property may not be present in the input, and it is expected to guess the actual type by the shape of JSON, for example by the presence of a specific key.

JsonContentPolymorphicSerializer provides a skeleton implementation for such a strategy. To use it, we override its selectDeserializer method. Let us start with the following class hierarchy.

Note, that is does not have to be sealed as recommended in the Sealed classes section, because we are not going to take advantage of the plugin-generated code that automatically selects the appropriate subclass, but are going to implement this code manually.

@Serializable
abstract class Project {
    abstract val name: String
}                   

@Serializable 
data class BasicProject(override val name: String): Project() 

            
@Serializable
data class OwnedProject(override val name: String, val owner: String) : Project()

We want to distinguish between the BasicProject and OwnedProject subclasses by the presence of the owner key in the JSON object.

object ProjectSerializer : JsonContentPolymorphicSerializer<Project>(Project::class) {
    override fun selectDeserializer(element: JsonElement) = when {
        "owner" in element.jsonObject -> OwnedProject.serializer()
        else -> BasicProject.serializer()
    }
}

We can serialize data with such serializer. In that case, either registered or the default serializer is selected for the actual type at runtime.

fun main() {
    val data = listOf(
        OwnedProject("kotlinx.serialization", "kotlin"),
        BasicProject("example")
    )
    val string = Json.encodeToString(ListSerializer(ProjectSerializer), data)
    println(string)
    println(Json.decodeFromString(ListSerializer(ProjectSerializer), string))
}

You can get the full code here.

No class discriminator is added in the JSON output.

[{"name":"kotlinx.serialization","owner":"kotlin"},{"name":"example"}]
[OwnedProject(name=kotlinx.serialization, owner=kotlin), BasicProject(name=example)]

Under the hood (experimental)

Although abstract serializers mentioned above can cover most of the cases, it is possible to implement similar machinery manually, using only the KSerializer class. If tweaking the abstract methods transformSerialize/transformDeserialize/selectDeserializer is not enough, then altering serialize/deserialize is a way to go.

There are several tidbits on custom serializers with Json.

Given all that, it is possible to implement two-stage conversion Decoder -> JsonElement -> value or
value -> JsonElement -> Encoder. For example, we can implement a fully custom serializer for the following Response class so that its Ok subclass is represented directly, but Error subclass by an object with the error message.

@Serializable(with = ResponseSerializer::class)
sealed class Response<out T> {
    data class Ok<out T>(val data: T) : Response<T>()
    data class Error(val message: String) : Response<Nothing>()
}

class ResponseSerializer<T>(private val dataSerializer: KSerializer<T>) : KSerializer<Response<T>> {
    override val descriptor: SerialDescriptor = buildSerialDescriptor("Response", PolymorphicKind.SEALED) {
        element("Ok", buildClassSerialDescriptor("Ok") {
            element<String>("message")
        })
        element("Error", dataSerializer.descriptor)
    }

    override fun deserialize(decoder: Decoder): Response<T> {
        // Decoder -> JsonDecoder
        require(decoder is JsonDecoder) // this class can be decoded only by Json
        // JsonDecoder -> JsonElement
        val element = decoder.decodeJsonElement()
        // JsonElement -> value
        if (element is JsonObject && "error" in element)
            return Response.Error(element["error"]!!.jsonPrimitive.content)
        return Response.Ok(decoder.json.decodeFromJsonElement(dataSerializer, element))
    }

    override fun serialize(encoder: Encoder, value: Response<T>) {
        // Encoder -> JsonEncoder
        require(encoder is JsonEncoder) // This class can be encoded only by Json
        // value -> JsonElement
        val element = when (value) {
            is Response.Ok -> encoder.json.encodeToJsonElement(dataSerializer, value.data)
            is Response.Error -> buildJsonObject { put("error", value.message) }
        }
        // JsonElement -> JsonEncoder
        encoder.encodeJsonElement(element)
    }
}

Armed with this serializable Response implementation we can take any serializable payload for its data and serialize/deserialize the corresponding responses.

@Serializable
data class Project(val name: String)

fun main() {
    val responses = listOf(
        Response.Ok(Project("kotlinx.serialization")),
        Response.Error("Not found")
    )
    val string = Json.encodeToString(responses)
    println(string)
    println(Json.decodeFromString<List<Response<Project>>>(string))
}

You can get the full code here.

This gives us fine-grained control on the representation of the Response class in our JSON output.

[{"name":"kotlinx.serialization"},{"error":"Not found"}]
[Ok(data=Project(name=kotlinx.serialization)), Error(message=Not found)]

The next chapter covers Alternative and custom formats (experimental).