Correctly erase Scala 2 intersection types #11603

smarter · 2021-03-04T17:45:24Z

No description provided.

smarter · 2021-03-04T17:54:22Z

test performance please

dottybot · 2021-03-04T17:56:21Z

performance test scheduled: 1 job(s) in queue, 1 running.

griggt · 2021-03-04T18:44:16Z

Looks like this fixes #10827

dottybot · 2021-03-04T20:10:09Z

Performance test finished successfully:

Visit http://dotty-bench.epfl.ch/11603/ to see the changes.

Benchmarks is based on merging with master (601cc5b)

odersky

Overall, this looks great. I wonder whether @lrytz could also have a look. Or who else was working on Scala 2 erasure last? I only seem to remember that it changed a but over time

compiler/src/dotty/tools/dotc/core/TypeErasure.scala

compiler/src/dotty/tools/dotc/core/Signature.scala

compiler/src/dotty/tools/dotc/core/Denotations.scala

compiler/src/dotty/tools/dotc/core/TypeErasure.scala

compiler/src/dotty/tools/dotc/core/unpickleScala2/Scala2Erasure.scala

lrytz · 2021-03-05T20:39:41Z

I don't have such a deep background in Scala 2 erasure. But the work here looks really great, it's great to see the fixes, and really valueable to have that infrastructure in place for improving compat in the future without further complicating the Scala 3 implementation.

Structural members need to be selected by name, not by symbols. Also deleted a special case in finishSym that was dead code: refinement classes are classes so the `owner.isClass` branch was always taken.

Previously we only distinguished between Java and Scala erasure, now we have Java, Scala 2 and Scala 3 erasure modes. This commit only adds the machinery needed to make that distinction, latter commits in this PR will introduce differences between the Scala 2 and 3 modes.

For SymDenotations, it's important that we use a signature that matches how the type is erased to maintain the invariant that two overloads in the same owner have different signatures, but for non-SymDenotations we don't make this guarantee so we can choose whatever we want as long as it's consistent. I originally chose to treat them like SymDenotation for consistency, but we're about to add some complex (and therefore somewhat expensive) logic for dealing with Scala 2 intersection erasure, by using the Scala 3 signature for all non-SymDenotation we avoid calling this logic more than we need to.

Because our algorithm for erasing intersection types does not exactly match the one used by Scala 2, we could end up emitting calls to Scala 2 methods with the wrong bytecode signature, leading to NoSuchMethodError at runtime. We could try to exactly match what Scala 2 does, but it turns out that the Scala 2 logic heavily relies on implementation details which makes it extremely complex to reliably replicate. Therefore, this commit instead special-cases the erasure of Scala 2 intersections (just like we already special-case the erasure of Java intersections) and limits which Scala 2 intersection types we support to a subset that we can erase without too much complications (but even that still requires ~200 lines of code!). This means that we're now free to change the way we erase intersections in any way we want without introducing more compatibility problems (until 3.0.0 that is), I'll explore this in a follow-up PR. Fixes scala#4619. Fixes scala#9175.

Previously their signature was incorrectly computed (the value class was erased to its underlying type) which lead to valid Java overloads being seen as double definitions.

odersky · 2021-03-09T13:02:10Z

LGTM. @jsrd Do you want to take another look or can we merge this?

sjrd · 2021-03-09T13:08:51Z

IIUC in this PR there's nothing related to Scala.js union left, so as far as I'm concerned it can be merged.

Thanks to scala#11603, we can now freely how we erase our intersection types without compromising compatibility with Scala 2. We take advantage of this by designing a new erasedGlb algorithm which respects associativity and commutativity and can also erase Java intersections without any special case. This commit also improves the way we handle intersections in Java generic signature to produce more precise types when possible and to ensure that we never emit a type that would lead to javac emitting invalid bytecode (which can happen in Scala 2: scala/bug#12300).

Thanks to scala#11603, we can now freely decide how we erase our intersection types without compromising compatibility with Scala 2. We take advantage of this by designing a new erasedGlb algorithm which respects associativity and commutativity and can also erase Java intersections without any special case. This commit also improves the way we handle intersections in Java generic signature to produce more precise types when possible and to ensure that we never emit a type that would lead to javac emitting invalid bytecode (which can happen in Scala 2: scala/bug#12300).

Thanks to scala#11603, we can now freely decide how we erase our intersection types without compromising compatibility with Scala 2. We take advantage of this by designing a new erasedGlb algorithm which respects associativity and commutativity and can also erase Java intersections without any special case. This commit also improves the way we handle intersections in Java generic signature to produce more precise types when possible and to ensure that we never emit a type that would lead to javac emitting invalid bytecode (which can happen with Scala 2: scala/bug#12300).

Thanks to scala#11603, we can now freely decide how we erase our intersection types without compromising compatibility with Scala 2. We take advantage of this by designing a new erasedGlb algorithm which respects associativity and commutativity and can also erase Java intersections without any special case. This commit also improves the way we handle intersections in Java generic signature to produce more precise types when possible and to ensure that we never emit a type that would lead to javac emitting invalid bytecode (which can happen with Scala 2: scala/bug#12300). Fixes scala#5139.

Thanks to scala#11603, we can now freely decide how we erase our intersection types without compromising compatibility with Scala 2. We take advantage of this by designing a new erasedGlb algorithm which respects associativity and commutativity and can also erase Java intersections without any special case. Incidentally, this lets us reverse a recent change in scala-stm which was previously required due to two equivalent types ending up with different signatures. This commit also improves the way we handle intersections in Java generic signature to produce more precise types when possible and to ensure that we never emit a type that would lead to javac emitting invalid bytecode (which can happen with Scala 2: scala/bug#12300). Fixes scala#5139.

We sometimes need to erase `Array[T]` to `Object` instead of `Array[erasure(T)]` in method signatures, this happens when both primitive and reference arrays, or two different sort of primitive arrays could be passed as arguments because the lub of those types on the JVM is `Object`. But before this commit, we additionally erased Arrays whose element type is upper-bounded by a universal trait to Object (like `Array[_ <: Serializable]`), this isn't necessary since primitives do not extend those traits (except for compiler fictions like `Singleton`) and derived value classes in arrays are always boxed. Since having matching Scala 2 and 3 erasure is a lost cause (cf scala#11603), this commit align ourselves with Java which improves Java interop (because we can emit more accurate Java generic signatures) and should let us simplify erasure more in the future. It also turns out than even before this commit, we did not match Scala 2 erasure perfectly since we erase `Array[_ <: Int]` to `Array[Int]` whereas Scala 2 erases to `Object` (this is important since we want `IArray[Int]` to be erased to `Array[Int]`), so we need to special case Scala 2 array erasure anyway to handle this. This commit renames `isUnboundedGeneric` to `isGenericArrayElement` since the former was somewhat misleading, `T <: String | Any` is bounded, but `Array[T]` cannot be represented with a specific JVM array type.

Thanks to scala#11603, we can now freely decide how we erase our intersection types without compromising compatibility with Scala 2. We take advantage of this by designing a new erasedGlb algorithm which respects associativity and commutativity and can also erase Java intersections without any special case. Incidentally, this lets us reverse a recent change in scala-stm which was previously required due to two equivalent types ending up with different signatures. This commit also improves the way we handle intersections in Java generic signature to produce more precise types when possible and to ensure that we never emit a type that would lead to javac emitting invalid bytecode (which can happen with Scala 2: scala/bug#12300). Fixes scala#5139.

smarter force-pushed the scala2-intersection-erasure-3 branch from c54fc99 to 229d183 Compare March 4, 2021 17:46

smarter requested review from odersky and sjrd March 4, 2021 17:59

smarter assigned odersky Mar 4, 2021

smarter linked an issue Mar 4, 2021 that may be closed by this pull request

AbstractMethodError using Scala 2 http4s library/withDottyCompat #10827

Closed

odersky requested changes Mar 5, 2021

View reviewed changes

odersky assigned smarter and unassigned odersky Mar 5, 2021

smarter added 5 commits March 8, 2021 12:54

Scala2Unpickler: Support structural member selection

c08543a

Structural members need to be selected by name, not by symbols. Also deleted a special case in finishSym that was dead code: refinement classes are classes so the `owner.isClass` branch was always taken.

Properly handle value classes appearing in Java code

943636f

Previously their signature was incorrectly computed (the value class was erased to its underlying type) which lead to valid Java overloads being seen as double definitions.

smarter force-pushed the scala2-intersection-erasure-3 branch from 229d183 to 943636f Compare March 8, 2021 12:43

smarter changed the title ~~Correctly erase Scala 2 intersection types; first step towards first-class Scala.js unions~~ Correctly erase Scala 2 intersection types Mar 8, 2021

smarter assigned odersky and unassigned smarter Mar 8, 2021

odersky approved these changes Mar 9, 2021

View reviewed changes

odersky assigned sjrd and unassigned odersky Mar 9, 2021

smarter merged commit c7054c2 into scala:master Mar 9, 2021

smarter deleted the scala2-intersection-erasure-3 branch March 9, 2021 13:37

lrytz mentioned this pull request Mar 17, 2021

The Scaladoc {{{...}}} code block syntax isn’t working #11728

Closed

smarter mentioned this pull request Mar 18, 2021

Associative, commutative, Java-friendly intersection erasure #11808

Merged

smarter mentioned this pull request Mar 22, 2021

Array erasure: Better Java and Scala 2 compat #11846

Merged

bishabosha mentioned this pull request Mar 30, 2021

Signature is different if method compiled by Scala 2 #11949

Closed

Kordyjan added this to the 3.0.0 milestone Aug 2, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Correctly erase Scala 2 intersection types #11603

Correctly erase Scala 2 intersection types #11603

smarter commented Mar 4, 2021

smarter commented Mar 4, 2021

dottybot commented Mar 4, 2021

griggt commented Mar 4, 2021

dottybot commented Mar 4, 2021

odersky left a comment

lrytz commented Mar 5, 2021

odersky commented Mar 9, 2021

sjrd commented Mar 9, 2021

Correctly erase Scala 2 intersection types #11603

Correctly erase Scala 2 intersection types #11603

Conversation

smarter commented Mar 4, 2021

smarter commented Mar 4, 2021

dottybot commented Mar 4, 2021

griggt commented Mar 4, 2021

dottybot commented Mar 4, 2021

odersky left a comment

Choose a reason for hiding this comment

lrytz commented Mar 5, 2021

odersky commented Mar 9, 2021

sjrd commented Mar 9, 2021