Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Backward compatible variant for migration from Scala 2.12 to 2.13 #467

Open
wants to merge 4 commits into
base: develop
Choose a base branch
from

Conversation

vigoo
Copy link

@vigoo vigoo commented Aug 31, 2020

The default implementation in Chill does not maintain binary compatibility between Scala version 2.12 and 2.13. I guess this is an intentional decision, as even a separate test data set is provided for the two versions.

Earlier this year we migrated a live service to Scala 2.13 that was using Chill with Akka Persistence and clustering, so had to keep binary compatibility to do a live upgrade. This PR contains an alternative ScalaKryoInstantiator for Scala 2.13 that preserves compatibility with 2.12 in both directions. The compatibility does not mean binary equivalence in all cases but roundtrip serialization between the two Scala versions is possible.

This was heavily tested manually and runs since months in production but there are no tests included yet. If this is something you would like to have in the core library as it may help others migrations, I'm happy to add them.

@CLAassistant
Copy link

CLAassistant commented Aug 31, 2020

CLA assistant check
All committers have signed the CLA.

@codecov-commenter
Copy link

codecov-commenter commented Aug 31, 2020

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 91.69%. Comparing base (5fc80e9) to head (51ded44).
Report is 212 commits behind head on develop.

Additional details and impacted files
@@           Coverage Diff            @@
##           develop     #467   +/-   ##
========================================
  Coverage    91.69%   91.69%           
========================================
  Files           42       42           
  Lines         1396     1396           
  Branches        35       35           
========================================
  Hits          1280     1280           
  Misses         116      116           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@johnynek
Copy link
Collaborator

Thanks for the PR.

I'm willing to merge this, but I'd like to have some tests. For instance, one idea would be we could check in a file that has been serialized with one of all these items in 0.9.2 with 2.12 or something, and then assert that we deserialize it.

I get very nervous about code that people depend on that has no tests in the repo.

@vigoo
Copy link
Author

vigoo commented Aug 31, 2020

I fully agree and will add some tests!

@johnynek johnynek mentioned this pull request Aug 31, 2020
Copy link
Collaborator

@regadas regadas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this! Would you mind rebasing as well and formatting with scalafmt? +scalafmtAll

val nameId = classToNameId.get(placeholderType, -1)
if (nameId != -1) {
output.writeVarInt(nameId, true)
return
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎨 could we rewrite this without using return?


override def write(kryo: Kryo, output: Output, `object`: T): Unit = {
var length: Option[Int] = None
for (field <- getFields) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎨 use while?

val start = kryo.readClassAndObject(input)
val step = kryo.readClassAndObject(input)

for (field <- getFields) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎨 use while? yields better perf.

@vigoo
Copy link
Author

vigoo commented Oct 2, 2020

Update: I found some issues, will push fix and tests when have time.

@johnynek
Copy link
Collaborator

take a look at #514 --

that would break, significantly, binary compatibility, and I assume the serialized data compatibility.

you might want to weight in.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants