Allow updating individual parts of the processor

[nomisRev]

Some ideas:

1. abstract class, Wrapper(lambdas): Interceptor, ... (should be compatible with Gradle ideally, especially in relation to the union ticket)
2. Split all interceptors into smaller groups for example:

interface OpenAPIInterceptor {
  interface Enum {
    toEnum(...): KModel
    enum(...): KModel
  }
  // interface Union (oneOf, ?), Object, etc
}

interface NamingStrategy {
  interface Enum { ... }
}

[nomisRev]

End-goal:

openKttp {
  enumFormatting = Formatting.SNAKE_CASE
  topLevelGeneration = NameGeneration.INDICES // NUMBERING, ???
}

// is allow overriding with functions in Gradle crazy? How can we allow registering user code? Kotlin Script?? 🤔

[hfhbd]

I created a similar tool for OpenAPI/Swagger/WSDL/XSD files and I just use service loader mechanism and interfaces. You need to create a custom gradle module and implement the transformer, and add it to a predefined Gradle configuration.

[nomisRev]

Hey @hfhbd,

I am not entirely sure I follow? 😅
Are you talking about this specific issue, or the entire project?

Any links you can share? I found this on your GitHub, https://github.com/hfhbd/serviceloader-gradle-plugin

[hfhbd]

@nomisRev I just talk about your design for this issue how to update/change the behavior. Instead using functions in Gradle, I chose to use transformers and each "function"/option to change the behavior is implemented as a service loaded at runtime. This is my whole pipeline:

flowchart TD
    Input(Schema File) -->|Read Schema|M
    M --> |Model-Transformer|M
    M[Model] --> IR(Format independent IR Tree)
    IR -->|IR-Transformer|IR
    IR -->C(CodeGen Tree)
    C -->|CodeGen-Transformer|C(CodeGen Tree)
    C --> K(Kotlin representation)
    K --> |KIR-Transformer|K
    K -->KP(Kotlin Poet)
    KP --> O(Output File)

Loading

And this is a (very basic) transformer:

package com.volkswagen.vocs.kfx.wsdl

interface WsdlTransformer {
    operator fun invoke(definitions: WSDL): WSDL
    operator fun invoke(schema: Schema): Schema
}

fun interface IrTransformer {
    operator fun invoke(irTree: IRTree): IRTree
}

@ServiceLoader(IrTransformer::class)
class FooInterfaceMessageBodyTransformer : IrTransformer {
    override fun invoke(irTree: IRTree): IRTree {
        return irTree.copy(
            classes = irTree.classes.mapTo(mutableSetOf()) {
                if (it.name == "FooInterfaceMessageBody") {
                    (it as IRTree.NormalClass).copy(
                        members = it.members.map {
                            if (it.name == "value") {
                                it.copy(
                                    xmlType = IRTree.XmlType.CData,
                                )
                            } else {
                                it
                            }
                        },
                    )
                } else {
                    it
                }
            },
        )
    }
}

Unfortunately, my project is not yet open sourced.

[nomisRev]

Ah okay, got it! Thank you for claryif'ing @hfhbd 🙏 That is an amazing idea, and is actually exactly what I was looking for complexer cases!

Unfortunately, my project is not yet open sourced.

Oh, I hope we don't conflict, or can collaborate somehow and both projects can complement each other. I seem you have a lot more intermediate phases in your pipeline.

I think your IR, CodeGen, and Kotlin representation phases are a single phase 'Model' phase in my project. Was this done to simplify, and models and their transformations? 🤔 My single transformation layer is quite complex, but it's like 650~ lines of code, so it's not so bad.

I'm not sure what the goals are of your library, but my goal here was just to focus on OpenAPI, in the future maybe AsyncAPI, and/or ReDoc. Primary goal to build a generate an opinionated HttpClient using Ktor, with full support for everything OpenAPI covers.

• Complex Types: oneOf, anyOf, allOf
• Complex (multiple) response types
• Formats: YAML, Json, XML

In the future also perhaps generate other tooling such as:

• IntelliJ Http
• (Ktor) Server generation
• Dynamic OpenAPI clients

[hfhbd]

Was this done to simplify, and models and their transformations?

Yes, to support different formats. I started with WSDL and XSD support, and extended it to Swagger and OpenAPI. Of course I also did start small with less phases but it turns out, you often need to transform different parts because each api is different, so I added more and more phases... Or sometimes you don't like the generated code and you want to add/remove prefixes etc.

I also focus on Kotlin and Ktor client + Ktor server implementation, and yes, handling different content types adds another complexity because you also need custom Ktor plugins to support call specific content negotiation.

I directly support XML (SOAP) and Json and I generate the client functions using HttpClient as receiver. So setting up the correct format/installing the plugin is up to the caller, this simplifies the generated code.

// public api, so safe to share

/**
 * Starts a new synchronization run using a DataProvider information to obtain the LDIF input, but
 * choose a configuration based on execution group.
 * @param groupName The name of execution group
 * @param start If true the created run will be enqueued to be started
 * @param test If true a dry run without any changes will be performed. This parameter requires the
 * start parameter to be set to true as well
 */
public suspend fun HttpClient.createSynchronizationRunWithExecutionGroupAndUrlInput(
  input: DataProvider,
  groupName: String? = null,
  start: Boolean? = null,
  test: Boolean? = null,
): SynchronizationRunWithConfiguration? {
  val response = post(urlString =
      """/synchronizationRuns/withExecutionGroupAndUrlInput""") {
    parameter("groupName", groupName)
    parameter("start", start)
    parameter("test", test)
    contentType(Json)
    setBody(body = input)
  }
  if (response.status.value == 404) {
    return null
  }
  return response.body<SynchronizationRunWithConfiguration>()
}

/**
 * Starts a new synchronization run using a DataProvider information to obtain the LDIF input
 */
public
    fun Route.createSynchronizationRunWithUrlInput(action: suspend ApplicationCall.(DataProvider) -> SynchronizationRunWithConfiguration) {
  route(path = """/synchronizationRuns/withUrlInput""") {
    contentType(Json) {
      post {
        val body = call.receive<DataProvider>()
        val response = call.action(body)
        call.response.status(OK)
        call.respond(response)
      }
    }
  }
}

// XML generated class:

@Serializable
@XmlSerialName(
  value = "FooResultSet",
  namespace = "http://xmldefs.example.com/Service/V2",
)
public data class ResultSet(
  /**
   * Blah blah blah
   */
  @XmlElement
  @XmlSerialName(
    value = "FResult",
    namespace = "http://xmldefs.example.com/Service/V2",
  )
  public val fResult: Double? = null,
  /**
   * F
   */
  @XmlElement
  @XmlSerialName(
    value = "ProtocolID",
    namespace = "http://xmldefs.example.com/Service/V2",
  )
  @MaxLength(inclusive = 255) // https://github.com/hfhbd/validation
  public val protocolID: Identifier? = null,
)

And yes, I need to extract the parameters on server side, it's on my todo list.

But in the end, the user still needs to setup the HttpClient with the correct plugins, like base url, authentication and formats, and on the server side, the user needs to setup the routing and the formats to the module too.

Maybe I can extract the Ktor plugins and publish them as open source.

[nomisRev]

Wow, impressive work @hfhbd. 👏 👏 👏

Sadly, it seems there is a lot of overlap 😞

Not sure what is the best way to proceed, I'm personally not a fan of "conflicting" libraries. What do you think?

Very surprised to find someone else working on this 😅 even though I wondered what some other people were doing in Kotlin. I've been wrestling and writing custom templates for OpenAPI Generator for almost 8 years, and toyed with this for several years due to lack of time.

But in the end, the user still needs to setup the HttpClient with the correct plugins, like base url, authentication and formats, and on the server side, the user needs to setup the routing and the formats to the module too.

Yes, those are things that all this need to be dealt with to some extends.

[hfhbd]

Not sure what is the best way to proceed, I'm personally not a fan of "conflicting" libraries. What do you think?

Let me check if I can open source it. As you might guess, I wrote the lib at Volkswagen, so don't expect an answer tomorrow.

But yes, it would be nice, if we can bundle our resources :)

[nomisRev]

That'd be nice indeed! Cool work you get to do at Volkswagen 😁

I'm close to finishing off what I currently need, and 'Dynamic OpenAPI clients' is next but that should also be refactor-able/buildable on your pipeline on top of IR, or KIR.