Generator for .proto files based on serializable Kotlin classes

[elizarov]

The idea is that you can have your Kotlin classes as your "master" code that conveniently declares your APIs and that you can analyze and refactor in IDE. Then, if you want your APIs to be used from other languages, you could generate a protobuf .proto files based on that and use them for other languages.

The design shall have at least two layers. On lower layer we need a format-agnostic way to traverse structure of serializable classes, that is ClassDesc shall contain the corresponding methods to make full reflection on the serial representation of classes possible. On the next layer, the format-specific code would be used. In this case, there shall be a tool that takes a set serializable classes (with an ability to scan a whole folder of compiled classes) and outputs a proto file.

[apatrida]

this would allow use of Kotlin serialization in conjunction with GRPC which requires the .proto files. https://grpc.io/

[cgruber]

An alternative (or companion idea) would be to accept @Schema("foo.proto") (handwave about whether that should be a path, a uri, or what). This could allow for (a) schema validation so that you know that your type is wire-compatible with the proto definition, (b) possibly omitting some annotations on fields if it can unambiguously attribute proto fields to properties, and (c) make this useable (and safe) in the context where the source of truth is a .proto, not the .kt file.

[elizarov]

@cgruber The idea with @Schema("foo.proto") is quite hard to align with format-agnostic nature of kotlin serialization. The validation has to happen at compile time, but serialization compiler plugin is not format-specific. It does not know and should not know anything about protobuf in particular. Protobuf support is runtime-only.

However, we can build a separate generation/validation tool that takes proto file and generates a Kotlin stub classes based on that, and, in addition to that, as proto file is being updated, performs compile-time validation of the correspondence between proto files and Kotlin files, if you plug the corresponding plugin into the build process.

[cgruber]

It wouldn't be inherent to kotlinx.serialization, necessarily, but an additional validator. As you say, it could be a completely separate tool. But for proto usage where proto's value is strong... cross-platform/language, using kotlin as a source of truth probably is going to be a blocker (we've used something similar to kx.serialization's apprach to java proto mapping, and it's caused all sorts of problems, when not paired with validation). So while yes, different project, I think it's something whose interactions should be thought about and referenced here. Even if the end aritifact in this project is just a documentation blurb and link.

…

On Wed, 29 Nov 2017 at 23:21 Roman Elizarov ***@***.***> wrote: @cgruber <https://github.com/cgruber> The idea with @Schema <https://github.com/schema>("foo.proto") is quite hard to align with format-agnostic nature of kotlin serialization. The validation has to happen at compile time, but serialization compiler plugin is not format-specific. It does not know and should not know anything about protobuf in particular. Protobuf support is runtime-only. However, we can build a separate generation/validation tool that takes proto file and generates a Kotlin stub classes based on that, and, in addition to that, as proto file is being updated, performs compile-time validation of the correspondence between proto files and Kotlin files, if you plug the corresponding plugin into the build process. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#34 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAUN4gUzD6OlDKkzz843ngFkw-vwYWcwks5s7ldlgaJpZM4Qp1DL> .

[elizarov]

@cgruber Thank. Those comments definitely belong to this issue. We do a need a two way (generation/validation) tool and being able to traverse Kotlin serialization schema is a key enabler for that.

[pdvrieze]

The infrastructure needed to do this is also required for more complex serializers. The actual information is actually embedded in the generated loaders/savers. It is possible to get almost all information (except for optionality) by emulating a loader but manually probing each field.

[leearmstrong]

Did anything ever happen in this?

I am just getting started with using Kotlin. At the moment I have a proto2 file I want to use and decode a payload but curious if I can generate the Kotlin code for the serialization?

[sandwwraith]

@leearmstrong This ticket is about the inverse process: generate proto2 files from Kotlin definitions. Unfortunately, as far as I know, there is no automated tool for your case.

[leearmstrong]

Ah ok, thank you!

[IgorKey]

Hi there!
Are there any updates? It was moved to 1.0. as i understand we are so close to 1.0...

[qwwdfsad]

Hi, unfortunately, it is not going to be fixed I'm 1.0.0, but it is something we are looking for after 1.0

[RdeWilde]

@shanshin Did you actually manage to get this working?

@elizarov your last reply was from 2017. I guess this is no longer on the roadmap?

[sandwwraith]

@RdeWilde See #477 (comment)