Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Revamp naming strategy and related annotations #182

Merged
merged 2 commits into from
Apr 11, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
118 changes: 110 additions & 8 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -120,13 +120,12 @@ Would normally have a schema like this:
}
```

However we can override the name and/or the namespace like this:
#### Overriding the class name and the namespace

```kotlin
package com.github.avrokotlin.avro4k.example

@AvroName("Wibble")
@AvroNamespace("com.other")
@SerialName("com.other.Wibble")
data class Foo(val a: String)
```

Expand All @@ -146,15 +145,67 @@ And then the generated schema looks like this:
}
```

Note: It is possible, but not necessary, to use both AvroName and AvroNamespace. You can just use either of them if you wish.

#### Overriding only the namespace

We can also just override the namespace while keeping the class name as record name:

```kotlin
package com.github.avrokotlin.avro4k.example

@AvroNamespaceOverride("com.other")
data class Foo(val a: String)
```

And then the generated schema looks like this:

```json
{
"type":"record",
"name":"Foo",
"namespace":"com.other",
"fields":[
{
"name":"a",
"type":"string"
}
]
}
```



#### Overriding only the name

We can just override the name while keeping the namespace. Note that you need to replicate the namespace in the `@SerialName` annotation:

```kotlin
package com.github.avrokotlin.avro4k.example

@SerialName("com.github.avrokotlin.avro4k.example.Wibble")
data class Foo(val a: String)
```

And then the generated schema looks like this:

```json
{
"type":"record",
"name":"Wibble",
"namespace":"com.github.avrokotlin.avro4k.example",
"fields":[
{
"name":"a",
"type":"string"
}
]
}
```


### Overriding a field name

The `@AvroName` annotation can also be used to override field names.
The `@SerialName` annotation can also be used to override field names.
This is useful when the record instances you are generating or reading need to have field names different from the Kotlin data classes.
For example if you are reading data generated by another system or another language.

Expand All @@ -163,7 +214,7 @@ Given the following class.
```kotlin
package com.github.avrokotlin.avro4k.example

data class Foo(val a: String, @AvroName("z") val b : String)
data class Foo(val a: String, @SerialName("z") val b : String)
```

Then the generated schema would look like this:
Expand All @@ -188,12 +239,64 @@ Then the generated schema would look like this:

Notice that the second field is z and not b.

Note: `@AvroName` does not add an alternative name for the field, but an override.
Note: `@SerialName` does not add an alternative name for the field, but an override.
If you wish to have alternatives then you should use `@AvroAlias`.


### Overriding the namespaces for all nested records, fixed and enums in a field

```kotlin
package com.github.avrokotlin.avro4k.example

data class Foo(@AvroNamespaceOverride("overridden") val nested: Bar)
data class Bar(val a: String)
```

Then the generated schema would look like this:

```json
{
"type":"record",
"name":"Foo",
"namespace":"com.github.avrokotlin.avro4k.example",
"fields":[
{
"name":"nested",
"type": {
"type":"record",
"name":"Bar",
"namespace":"overridden",
"fields":[
{
"name":"a",
"type":"string"
}
]
}
}
]
}
```

Notice that the second field is z and not b.

Note: `@SerialName` does not add an alternative name for the field, but an override.
If you wish to have alternatives then you should use `@AvroAlias`.



### Change the record/enum/fixed naming strategy
To change the naming strategy for records, enums and fixed types, create your own instance of `Avro` with the wanted naming strategies.

- `fieldNamingStrategy` is used for field names.
- `recordNamingStrategy` is used for record, enum and fixed names.

```kotlin
Avro(AvroConfiguration(
fieldNamingStrategy = /* ... */,
recordNamingStrategy = /* ... */,
))
```

### Adding properties and docs to a Schema

Expand Down Expand Up @@ -438,7 +541,6 @@ But if you want to remove a nullable field that is not optional, depending on th

So to mark a field as optional and facilitate avro contract evolution regarding compatibility checks, then set `default` to `null`.


## Types

Avro4k supports the Avro logical types out of the box as well as other common JDK types.
Expand Down
10 changes: 5 additions & 5 deletions settings.gradle.kts
Original file line number Diff line number Diff line change
Expand Up @@ -10,24 +10,24 @@ rootProject.name = "avro4k-core"
dependencyResolutionManagement {
versionCatalogs {
create("libs") {
version("kotlin", "1.8.20")
version("kotlin", "1.9.22")
version("jvm", "18")

library("xerial-snappy", "org.xerial.snappy", "snappy-java").version("1.1.10.1")
library("apache-avro", "org.apache.avro", "avro").version("1.11.3")

val kotlinxSerialization = "1.5.0"
val kotlinxSerialization = "1.6.2"
library("kotlinx-serialization-core", "org.jetbrains.kotlinx", "kotlinx-serialization-core").version(kotlinxSerialization)
library("kotlinx-serialization-json", "org.jetbrains.kotlinx", "kotlinx-serialization-json").version(kotlinxSerialization)

val kotestVersion = "5.6.1"
val kotestVersion = "5.8.0"
library("kotest-core", "io.kotest", "kotest-assertions-core").version(kotestVersion)
library("kotest-json", "io.kotest", "kotest-assertions-json").version(kotestVersion)
library("kotest-junit5", "io.kotest", "kotest-runner-junit5").version(kotestVersion)
library("kotest-property", "io.kotest", "kotest-property").version(kotestVersion)

plugin("dokka", "org.jetbrains.dokka").version("1.8.10")
plugin("kotest", "io.kotest").version("0.4.10")
plugin("dokka", "org.jetbrains.dokka").version("1.9.10")
plugin("kotest", "io.kotest").version("0.4.11")
plugin("github-versions", "com.github.ben-manes.versions").version("0.46.0")
plugin("nexus-publish", "io.github.gradle-nexus.publish-plugin").version("1.3.0")
plugin("spotless", "com.diffplug.spotless").version("6.25.0")
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -19,18 +19,14 @@ class AnnotationExtractor(private val annotations: List<Annotation>) {

fun fixed(): Int? = annotations.filterIsInstance<AvroFixed>().firstOrNull()?.size

fun namespace(): String? = annotations.filterIsInstance<AvroNamespace>().firstOrNull()?.value

fun name(): String? = annotations.filterIsInstance<AvroName>().firstOrNull()?.value

fun doc(): String? = annotations.filterIsInstance<AvroDoc>().firstOrNull()?.value

fun aliases(): List<String> =
(
annotations.firstNotNullOfOrNull {
it as? AvroAlias
}?.value ?: emptyArray()
).asList() + (annotations.firstNotNullOfOrNull { it as? AvroAliases }?.value ?: emptyArray())
).asList()

fun props(): List<Pair<String, String>> = annotations.filterIsInstance<AvroProp>().map { it.key to it.value }

Expand Down
13 changes: 9 additions & 4 deletions src/main/kotlin/com/github/avrokotlin/avro4k/Avro.kt
Original file line number Diff line number Diff line change
Expand Up @@ -153,10 +153,15 @@ class AvroOutputStreamBuilder<T>(
}

@OptIn(ExperimentalSerializationApi::class)
class Avro(
override val serializersModule: SerializersModule = defaultModule,
internal val configuration: AvroConfiguration = AvroConfiguration(),
class Avro internal constructor(
internal val configuration: AvroInternalConfiguration,
override val serializersModule: SerializersModule,
) : SerialFormat, BinaryFormat {
constructor(
serializersModule: SerializersModule = defaultModule,
configuration: AvroConfiguration = AvroConfiguration(),
) : this(AvroInternalConfiguration(configuration), serializersModule)

constructor(configuration: AvroConfiguration) : this(defaultModule, configuration)

companion object {
Expand Down Expand Up @@ -280,7 +285,7 @@ class Avro(
obj: T,
): GenericRecord {
var record: Record? = null
val encoder = RootRecordEncoder(schema, serializersModule) { record = it }
val encoder = RootRecordEncoder(schema, serializersModule, configuration) { record = it }
encoder.encodeSerializableValue(serializer, obj)
return record!!
}
Expand Down
79 changes: 73 additions & 6 deletions src/main/kotlin/com/github/avrokotlin/avro4k/AvroConfiguration.kt
Original file line number Diff line number Diff line change
@@ -1,13 +1,80 @@
package com.github.avrokotlin.avro4k

import com.github.avrokotlin.avro4k.schema.DefaultNamingStrategy
import com.github.avrokotlin.avro4k.schema.NamingStrategy
import com.github.avrokotlin.avro4k.schema.FieldNamingStrategy
import com.github.avrokotlin.avro4k.schema.RecordName
import com.github.avrokotlin.avro4k.schema.RecordNamingStrategy
import kotlinx.serialization.descriptors.SerialDescriptor
import java.util.concurrent.ConcurrentHashMap

data class AvroConfiguration(
val namingStrategy: NamingStrategy = DefaultNamingStrategy,
/**
* By default, during decoding, any missing value for a nullable field without default [null] value (e.g. `val field: Type?` without `= null`) is failing.
* When set to [true], the nullable fields that haven't any default value are set as null if the value is missing. It also adds `"default": null` to those fields when generating schema using avro4k.
* The naming strategy to use for record's name and namespace. Also applied for fixed and enum types.
*
* Default: [RecordNamingStrategy.Builtins.FullyQualified]
*/
val recordNamingStrategy: RecordNamingStrategy = RecordNamingStrategy.Builtins.FullyQualified,
/**
* The naming strategy to use for field's name.
*
* Default: [FieldNamingStrategy.Builtins.NoOp]
*/
val fieldNamingStrategy: FieldNamingStrategy = FieldNamingStrategy.Builtins.NoOp,
/**
* By default, during decoding, any missing value for a nullable field without default `null` value (e.g. `val field: Type?` without `= null`) is failing.
* When set to `true`, the nullable fields that haven't any default value are set as null if the value is missing. It also adds `"default": null` to those fields when generating schema using avro4k.
*/
val implicitNulls: Boolean = false,
)
/**
* Enable caching of resolved names.
*
* Default: `true`
*/
val namingCacheEnabled: Boolean = true,
)

class AvroInternalConfiguration private constructor(
val recordNamingStrategy: RecordNamingStrategy,
val fieldNamingStrategy: FieldNamingStrategy,
val implicitNulls: Boolean,
) {
constructor(configuration: AvroConfiguration) : this(
recordNamingStrategy = configuration.recordNamingStrategy.cachedIfNecessary(configuration.namingCacheEnabled),
fieldNamingStrategy = configuration.fieldNamingStrategy.cachedIfNecessary(configuration.namingCacheEnabled),
implicitNulls = configuration.implicitNulls
)
}

internal fun RecordNamingStrategy.cachedIfNecessary(cacheEnabled: Boolean): RecordNamingStrategy =
if (!cacheEnabled) {
this
} else {
object : RecordNamingStrategy {
private val cache = ConcurrentHashMap<SerialDescriptor, RecordName>()

override fun resolve(
descriptor: SerialDescriptor,
serialName: String,
): RecordName =
cache.getOrPut(descriptor) {
this@cachedIfNecessary.resolve(descriptor, serialName)
}
}
}

internal fun FieldNamingStrategy.cachedIfNecessary(cacheEnabled: Boolean): FieldNamingStrategy =
if (!cacheEnabled) {
this
} else {
object : FieldNamingStrategy {
private val cache = ConcurrentHashMap<Pair<SerialDescriptor, Int>, String>()

override fun resolve(
descriptor: SerialDescriptor,
elementIndex: Int,
serialName: String,
): String =
cache.getOrPut(descriptor to elementIndex) {
this@cachedIfNecessary.resolve(descriptor, elementIndex, serialName)
}
}
}
32 changes: 0 additions & 32 deletions src/main/kotlin/com/github/avrokotlin/avro4k/FieldNaming.kt

This file was deleted.

Loading
Loading