Derivation of codecs with Dotty (Scala 3)

[plokhotnyuk]

Currently jsoniter-scala uses some features of experimental Scala 2 macros which are not ported to Scala 3. They are described in the following PR to the Scala Migration Guide.

Two possible solutions:

1. Port Evals.eval to Scala 3
2. Redesign jsoniter-scala-macros API to avoid usage of Evals.eval calls and refactor jsoniter-scala-core implementation to avoid eval.eval calls (see dependency on expression-evaluator).

[domdorn]

As Scala 3 RC1 is out
( see https://mvnrepository.com/artifact/org.scala-lang/scala3-library_3.0.0-RC1/3.0.0-RC1 ), is there anything new about Jsoniter-Scala for Scala 3? I'm already thinking about moving some of my apps and if the macros don't work yet with Scala 3, I'd have to move the code to a separate module and compile them there with 2.13 (which I would like to avoid if possible).

Thx for the update!

[plokhotnyuk]

@domdorn Hi, Dominik! Moving of jsoniter-scala-macros API calls to the separate module in your project is a great idea.

It seems that Evals.eval will never be ported to Scala 3, so jsoniter-scala-macros API will be redesigned fully.

Moreover, in Scala 3 macros I haven't found an ability to pass a parameter which is more complex than a literal and can be used as a value in the compile time.

[domdorn]

Quick update:
I've split a new microservice project into multiple modules:

• web-requests // case classes for request/response
• web-json // jsoniter-scala codecs, depends on web-requests
• web // controllers, depends on web-requests and web-json and core
  and
• core module // actual business logic.
  web-requests and web-json have to be scala2, while the rest can be scala3.. so this way I'm able to use jsoniter and its macros.
  However, I'm unable to use any common classes that are in a scala3 package/module (e.g. that would reside in a domain module).

so at the moment, I'm able to use jsoniter-scala, but its quite a PITA and I basically have to have a migration-utility in my web-module that translates from the web-only domain classes to those of the system (which is normal, except that I usually have some classes that are shared amongst all layers, e.g. CorrelationId, UserId, ... )

I looked up the Evals.eval and looked through the code.. it is just used two times and - from my understanding - only for two very special cases

1. overriding a field name using the @named annotation
2. overriding mapping logic with own functions

If I'm right, number (2) could be overcome, if the function is in a different compilation-unit, right? If I'm right with this, could we maybe have a proof of concept of the macros with those two (or one) feature missing, and the rest already works?

[plokhotnyuk]

I think that using only literal constants for compile-time configuration is a great start point of derivation module for Scala 3.

Moreover MVP can be done for the default compile-time configuration.

BTW, in most cases configuration of name translation functions can be predefined in some enums.

[plokhotnyuk]

Core API is available for Scala 3 since v2.10.3.
JVM: https://repo1.maven.org/maven2/com/github/plokhotnyuk/jsoniter-scala/jsoniter-scala-core_3/2.10.3/
Scala.JS: https://repo1.maven.org/maven2/com/github/plokhotnyuk/jsoniter-scala/jsoniter-scala-core_sjs1_3/2.10.3/

[domdorn]

<3 will try it on one of my services asap!

[plokhotnyuk]

@domdorn BTW, some clever functions from the Macro API can be used with Scala 3, like here: https://scastie.scala-lang.org/g4WW4LhiQqSk61ZOkfNFhw

[domdorn]

I'm a little confused now.. you say the core-api is scala3 ready, the macros are not.. could u share a sample how I'd make a codec for e.g.

final case class Person(name: String, age: Int, address: Option[Address])
final case class Address(street: String, postalCode: String)

?

[plokhotnyuk]

@domdorn There is an example for similar data structure here.

As a workaround codecs can be generated as sources by Scala 2 compiler with turned -Xmacro-settings:print-codecs scalac option like here and then pasted manually to Scala 3 sources.

[domdorn]

hmm... my coworkers are going to be very upset if I commit this kind of code into our repos.. looks like we have to wait for the macros then :/

[rssh]

Here: https://github.com/rssh/jsoniter-scala/tree/scala3
is an attempt to port existing macros to scala3.

I make it compiled but not yet run. Note, that this attempt was porting with minimal changes, I have not tried to improve code organization.

Possible local next step -- look at the first errors in the tests compilation logs

sbt '++3.1.0' jsoniter-scala-macrosJVM/test 2>&1 | tee log

, extract this error into small reproducible examples and submit it to the dotty team or discover our own bug. repeat.

I think, porting existing code is possible but time-consuming [i.e. marathon, not sprint]. (I'm not sure that I can allocate an appropriate time budget for this, but maybe somebody can start from that point.

Only one thing, which is used in current macros and has no direct analog in dotty is access to the default values of primary constructors. (see scala/scala3#14078 ), eventually, it is possible to extend the tasty reflection API.

Also, I'm not sure that the current approach is ideal -- maybe better to write derivation from scratch, (not checked this, an idea to write all derivation inside some trait from which an instance of configuration is accessible).
Differences are not very big, from the other side -- code organization can be better) In case, if I have had enough time, I have started to explore this way.

In any case, It would be good to see branch 'scala3-experiment' here. (and maybe create a new subproject to explore dotty-only derivation)

[plokhotnyuk]

@rssh Hi, Ruslan!
You have done an amazing work!
I will DM you my propositions.

[domdorn]

awesome work @rssh !
@plokhotnyuk as you've merged this, can we get a new release on maven-central? I'm sure there are plenty of people waiting to try this out!

[plokhotnyuk]

@domdorn Yes, you can! It is available on the Maven Central. Please try and send your feedback.