Skip to content

Commit

Permalink
feat: Add naming cache
Browse files Browse the repository at this point in the history
  • Loading branch information
Chuckame committed Apr 10, 2024
1 parent f85e658 commit 2ecc97d
Show file tree
Hide file tree
Showing 18 changed files with 202 additions and 56 deletions.
118 changes: 110 additions & 8 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -120,13 +120,12 @@ Would normally have a schema like this:
}
```

However we can override the name and/or the namespace like this:
#### Overriding the class name and the namespace

```kotlin
package com.github.avrokotlin.avro4k.example

@AvroName("Wibble")
@AvroNamespace("com.other")
@SerialName("com.other.Wibble")
data class Foo(val a: String)
```

Expand All @@ -146,15 +145,67 @@ And then the generated schema looks like this:
}
```

Note: It is possible, but not necessary, to use both AvroName and AvroNamespace. You can just use either of them if you wish.

#### Overriding only the namespace

We can also just override the namespace while keeping the class name as record name:

```kotlin
package com.github.avrokotlin.avro4k.example

@AvroNamespaceOverride("com.other")
data class Foo(val a: String)
```

And then the generated schema looks like this:

```json
{
"type":"record",
"name":"Foo",
"namespace":"com.other",
"fields":[
{
"name":"a",
"type":"string"
}
]
}
```



#### Overriding only the name

We can just override the name while keeping the namespace. Note that you need to replicate the namespace in the `@SerialName` annotation:

```kotlin
package com.github.avrokotlin.avro4k.example

@SerialName("com.github.avrokotlin.avro4k.example.Wibble")
data class Foo(val a: String)
```

And then the generated schema looks like this:

```json
{
"type":"record",
"name":"Wibble",
"namespace":"com.github.avrokotlin.avro4k.example",
"fields":[
{
"name":"a",
"type":"string"
}
]
}
```


### Overriding a field name

The `@AvroName` annotation can also be used to override field names.
The `@SerialName` annotation can also be used to override field names.
This is useful when the record instances you are generating or reading need to have field names different from the Kotlin data classes.
For example if you are reading data generated by another system or another language.

Expand All @@ -163,7 +214,7 @@ Given the following class.
```kotlin
package com.github.avrokotlin.avro4k.example

data class Foo(val a: String, @AvroName("z") val b : String)
data class Foo(val a: String, @SerialName("z") val b : String)
```

Then the generated schema would look like this:
Expand All @@ -188,12 +239,64 @@ Then the generated schema would look like this:

Notice that the second field is z and not b.

Note: `@AvroName` does not add an alternative name for the field, but an override.
Note: `@SerialName` does not add an alternative name for the field, but an override.
If you wish to have alternatives then you should use `@AvroAlias`.


### Overriding the namespaces for all nested records, fixed and enums in a field

```kotlin
package com.github.avrokotlin.avro4k.example

data class Foo(@AvroNamespaceOverride("overridden") val nested: Bar)
data class Bar(val a: String)
```

Then the generated schema would look like this:

```json
{
"type":"record",
"name":"Foo",
"namespace":"com.github.avrokotlin.avro4k.example",
"fields":[
{
"name":"nested",
"type": {
"type":"record",
"name":"Bar",
"namespace":"overridden",
"fields":[
{
"name":"a",
"type":"string"
}
]
}
}
]
}
```

Notice that the second field is z and not b.

Note: `@SerialName` does not add an alternative name for the field, but an override.
If you wish to have alternatives then you should use `@AvroAlias`.



### Change the record/enum/fixed naming strategy
To change the naming strategy for records, enums and fixed types, create your own instance of `Avro` with the wanted naming strategies.

- `fieldNamingStrategy` is used for field names.
- `recordNamingStrategy` is used for record, enum and fixed names.

```kotlin
Avro(AvroConfiguration(
fieldNamingStrategy = /* ... */,
recordNamingStrategy = /* ... */,
))
```

### Adding properties and docs to a Schema

Expand Down Expand Up @@ -438,7 +541,6 @@ But if you want to remove a nullable field that is not optional, depending on th
So to mark a field as optional and facilitate avro contract evolution regarding compatibility checks, then set `default` to `null`.


## Types

Avro4k supports the Avro logical types out of the box as well as other common JDK types.
Expand Down
11 changes: 8 additions & 3 deletions src/main/kotlin/com/github/avrokotlin/avro4k/Avro.kt
Original file line number Diff line number Diff line change
Expand Up @@ -153,10 +153,15 @@ class AvroOutputStreamBuilder<T>(
}

@OptIn(ExperimentalSerializationApi::class)
class Avro(
override val serializersModule: SerializersModule = defaultModule,
internal val configuration: AvroConfiguration = AvroConfiguration(),
class Avro internal constructor(
internal val configuration: AvroInternalConfiguration,
override val serializersModule: SerializersModule,
) : SerialFormat, BinaryFormat {
constructor(
serializersModule: SerializersModule = defaultModule,
configuration: AvroConfiguration = AvroConfiguration(),
) : this(AvroInternalConfiguration(configuration), serializersModule)

constructor(configuration: AvroConfiguration) : this(defaultModule, configuration)

companion object {
Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,10 @@
package com.github.avrokotlin.avro4k

import com.github.avrokotlin.avro4k.schema.FieldNamingStrategy
import com.github.avrokotlin.avro4k.schema.RecordName
import com.github.avrokotlin.avro4k.schema.RecordNamingStrategy
import kotlinx.serialization.descriptors.SerialDescriptor
import java.util.concurrent.ConcurrentHashMap

data class AvroConfiguration(
val recordNamingStrategy: RecordNamingStrategy = RecordNamingStrategy.Default,
Expand All @@ -11,4 +14,44 @@ data class AvroConfiguration(
* When set to [true], the nullable fields that haven't any default value are set as null if the value is missing. It also adds `"default": null` to those fields when generating schema using avro4k.
*/
val implicitNulls: Boolean = false,
)
val namingCacheEnabled: Boolean = true,
)

class AvroInternalConfiguration private constructor(
val recordNamingStrategy: RecordNamingStrategy,
val fieldNamingStrategy: FieldNamingStrategy,
val implicitNulls: Boolean,
) {
constructor(configuration: AvroConfiguration) : this(
recordNamingStrategy = configuration.recordNamingStrategy.cachedIfNecessary(configuration.namingCacheEnabled),
fieldNamingStrategy = configuration.fieldNamingStrategy.cachedIfNecessary(configuration.namingCacheEnabled),
implicitNulls = configuration.implicitNulls
)
}

internal fun RecordNamingStrategy.cachedIfNecessary(cacheEnabled: Boolean): RecordNamingStrategy =
object : RecordNamingStrategy {
private val cache = ConcurrentHashMap<SerialDescriptor, RecordName>()

override fun resolve(
descriptor: SerialDescriptor,
serialName: String,
): RecordName =
cache.getOrPut(descriptor) {
this@cachedIfNecessary.resolve(descriptor, serialName)
}
}

internal fun FieldNamingStrategy.cachedIfNecessary(cacheEnabled: Boolean): FieldNamingStrategy =
object : FieldNamingStrategy {
private val cache = ConcurrentHashMap<Pair<SerialDescriptor, Int>, String>()

override fun resolve(
descriptor: SerialDescriptor,
elementIndex: Int,
serialName: String,
): String =
cache.getOrPut(descriptor to elementIndex) {
this@cachedIfNecessary.resolve(descriptor, elementIndex, serialName)
}
}
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
package com.github.avrokotlin.avro4k.decoder

import com.github.avrokotlin.avro4k.AvroConfiguration
import com.github.avrokotlin.avro4k.AvroInternalConfiguration
import kotlinx.serialization.DeserializationStrategy
import kotlinx.serialization.ExperimentalSerializationApi
import kotlinx.serialization.descriptors.PolymorphicKind
Expand All @@ -19,7 +19,7 @@ class ListDecoder(
private val schema: Schema,
private val array: List<Any?>,
override val serializersModule: SerializersModule,
private val configuration: AvroConfiguration,
private val configuration: AvroInternalConfiguration,
) : AbstractDecoder(), FieldDecoder {
init {
require(schema.type == Schema.Type.ARRAY)
Expand Down Expand Up @@ -85,7 +85,7 @@ class ListDecoder(
return when (descriptor.kind) {
StructureKind.CLASS -> RecordDecoder(descriptor, array[index] as GenericRecord, serializersModule, configuration)
StructureKind.LIST -> ListDecoder(schema.elementType, array[index] as GenericArray<*>, serializersModule, configuration)
StructureKind.MAP -> MapDecoder(descriptor, schema.elementType, array[index] as Map<String, *>, serializersModule, configuration)
StructureKind.MAP -> MapDecoder(schema.elementType, array[index] as Map<String, *>, serializersModule, configuration)
PolymorphicKind.SEALED, PolymorphicKind.OPEN -> UnionDecoder(descriptor, array[index] as GenericRecord, serializersModule, configuration)
else -> throw UnsupportedOperationException("Kind ${descriptor.kind} is currently not supported.")
}
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
package com.github.avrokotlin.avro4k.decoder

import com.github.avrokotlin.avro4k.AvroConfiguration
import com.github.avrokotlin.avro4k.AvroInternalConfiguration
import kotlinx.serialization.ExperimentalSerializationApi
import kotlinx.serialization.SerializationException
import kotlinx.serialization.descriptors.PolymorphicKind
Expand All @@ -17,11 +17,10 @@ import java.nio.ByteBuffer

@ExperimentalSerializationApi
class MapDecoder(
private val desc: SerialDescriptor,
private val schema: Schema,
map: Map<*, *>,
override val serializersModule: SerializersModule,
private val configuration: AvroConfiguration,
private val configuration: AvroInternalConfiguration,
) : AbstractDecoder(), CompositeDecoder {
init {
require(schema.type == Schema.Type.MAP)
Expand Down Expand Up @@ -146,7 +145,7 @@ class MapDecoder(
PrimitiveKind.BYTE -> ByteArrayDecoder((value() as ByteBuffer).array(), serializersModule)
else -> ListDecoder(schema.valueType, value() as GenericArray<*>, serializersModule, configuration)
}
StructureKind.MAP -> MapDecoder(descriptor, schema.valueType, value() as Map<String, *>, serializersModule, configuration)
StructureKind.MAP -> MapDecoder(schema.valueType, value() as Map<String, *>, serializersModule, configuration)
PolymorphicKind.SEALED, PolymorphicKind.OPEN -> UnionDecoder(descriptor, value() as GenericRecord, serializersModule, configuration)
else -> throw UnsupportedOperationException("Kind ${descriptor.kind} is currently not supported.")
}
Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
package com.github.avrokotlin.avro4k.decoder

import com.github.avrokotlin.avro4k.AnnotationExtractor
import com.github.avrokotlin.avro4k.AvroConfiguration
import com.github.avrokotlin.avro4k.AvroInternalConfiguration
import com.github.avrokotlin.avro4k.schema.extractNonNull
import kotlinx.serialization.ExperimentalSerializationApi
import kotlinx.serialization.SerializationException
Expand Down Expand Up @@ -31,7 +31,7 @@ class RecordDecoder(
private val desc: SerialDescriptor,
private val record: GenericRecord,
override val serializersModule: SerializersModule,
private val configuration: AvroConfiguration,
private val configuration: AvroInternalConfiguration,
) : AbstractDecoder(), FieldDecoder {
private var currentIndex = -1

Expand All @@ -42,7 +42,6 @@ class RecordDecoder(
StructureKind.CLASS -> RecordDecoder(descriptor, value as GenericRecord, serializersModule, configuration)
StructureKind.MAP ->
MapDecoder(
descriptor,
fieldSchema(),
value as Map<String, *>,
serializersModule,
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
package com.github.avrokotlin.avro4k.decoder

import com.github.avrokotlin.avro4k.AvroConfiguration
import com.github.avrokotlin.avro4k.AvroInternalConfiguration
import kotlinx.serialization.ExperimentalSerializationApi
import kotlinx.serialization.SerializationException
import kotlinx.serialization.descriptors.PolymorphicKind
Expand All @@ -16,7 +16,7 @@ import org.apache.avro.generic.GenericRecord
class RootRecordDecoder(
private val record: GenericRecord,
override val serializersModule: SerializersModule,
private val configuration: AvroConfiguration,
private val configuration: AvroInternalConfiguration,
) : AbstractDecoder() {
var decoded = false

Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
package com.github.avrokotlin.avro4k.decoder

import com.github.avrokotlin.avro4k.AvroConfiguration
import com.github.avrokotlin.avro4k.AvroInternalConfiguration
import com.github.avrokotlin.avro4k.possibleSerializationSubclasses
import com.github.avrokotlin.avro4k.schema.RecordName
import kotlinx.serialization.DeserializationStrategy
Expand All @@ -18,7 +18,7 @@ class UnionDecoder(
descriptor: SerialDescriptor,
private val value: GenericRecord,
override val serializersModule: SerializersModule,
private val configuration: AvroConfiguration,
private val configuration: AvroInternalConfiguration,
) : AbstractDecoder(), FieldDecoder {
private enum class DecoderState(val index: Int) {
BEFORE(0),
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
package com.github.avrokotlin.avro4k.encoder

import com.github.avrokotlin.avro4k.AvroConfiguration
import com.github.avrokotlin.avro4k.AvroInternalConfiguration
import kotlinx.serialization.ExperimentalSerializationApi
import kotlinx.serialization.descriptors.SerialDescriptor
import kotlinx.serialization.encoding.AbstractEncoder
Expand All @@ -15,7 +15,7 @@ import java.nio.ByteBuffer
class ListEncoder(
private val schema: Schema,
override val serializersModule: SerializersModule,
override val configuration: AvroConfiguration,
override val configuration: AvroInternalConfiguration,
private val callback: (GenericData.Array<Any?>) -> Unit,
) : AbstractEncoder(), StructureEncoder {
private val list = mutableListOf<Any?>()
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
package com.github.avrokotlin.avro4k.encoder

import com.github.avrokotlin.avro4k.AvroConfiguration
import com.github.avrokotlin.avro4k.AvroInternalConfiguration
import kotlinx.serialization.ExperimentalSerializationApi
import kotlinx.serialization.SerializationException
import kotlinx.serialization.descriptors.SerialDescriptor
Expand All @@ -16,7 +16,7 @@ import java.nio.ByteBuffer
class MapEncoder(
schema: Schema,
override val serializersModule: SerializersModule,
override val configuration: AvroConfiguration,
override val configuration: AvroInternalConfiguration,
private val callback: (Map<Utf8, *>) -> Unit,
) : AbstractEncoder(),
CompositeEncoder,
Expand Down
Loading

0 comments on commit 2ecc97d

Please sign in to comment.