Incorrect widening of covariant type arguments passed to a polymorphic function

[neko-kai]

Compiler version

3.2.1, 3.2.2-RC1, 3.3.1-RC4

Minimized code

Runnable: https://scastie.scala-lang.org/9rXw6Hg8RRSmRkk1b4NemA

case class ZIO[-R, +E, +A]()
object ZIO {
  val interrupt: ZIO[Any, Nothing, Nothing] = ZIO()
}

inline def fromHas[R, E, A](z: ZIO[R, E, A]): String = {
  compiletime.codeOf(identity(z))
}

@main def main: Unit = {
  println(fromHas(ZIO.interrupt))
}

Output

identity[ZIO[Any, Any, Any]](ZIO.interrupt)

Expectation

identity[ZIO[Any, Nothing, Nothing]](ZIO.interrupt)

Expected the type E and A parameters of ZIO.interrupt to be preserved, instead of widened all the way to Any, as in Scala 2.

Note: only covariant parameters were widened - contravariant R parameter was not widened from Any to Nothing.

This code is a minimized version of real library code in https://github.com/izumi/izumi which we're trying to port to Scala 3

[szymon-rd]

Thank you for reporting and minimization!

[DmytroOrlov]

@szymon-rd can you please take a look? as issue affects many Izumi users, and instead of concise:

make[MyEffect].fromZEnv(MyEffect.make)
make[MyResource].fromZEnv(MyResource.make)

we often need to write more explicit boilerplate like:

  def provideZEnv[R: ZEnvConstructor, E: Tag, A: Tag](fn: ZIO[R, E, A]): Functoid[IO[E, A]] =
    ZEnvConstructor[R].map(fn.provideEnvironment(_))

  def provideZEnv[R: ZEnvConstructor, E: Tag, A: Tag](fn: ZManaged[R, E, A]): Functoid[Managed[E, A]] =
    ZEnvConstructor[R].map(fn.provideEnvironment(_))

  make[MyEffect].fromEffect(provideZEnv(MyEffect.make))
  make[MyResource].fromResource(provideZEnv(MyResource.make))

[szymon-rd]

@DmytroOrlov Could you please provide an example project where this problem occurs? It would really help us understand the goal and structure of the API with this problem. I looked into that, and it's how the current inference behaves. Given a HOT with covariant/contravariant type parameters, the compiler will often try to come up with as general parameter types and as specific return types as possible. It's not a good idea to rely solely on the preservation of type parameters that are covariant/contravariant during type inference.

However, if you give me a simple project where this is problematic so I understand the problem fully, I may help you come up with a solution.

[neko-kai]

@szymon-rd
The example is at the top, the real API is this and is essentially very similar. Note that the inferrence issue doesn't happen in a similar method that uses HKT instead of separate type parameter here def fromEffect[F[_], I](instance: F[I])

come up with as general parameter types

I'm not sure that's exactly what's happening, because the contravariant parameter is not widened from Any to Nothing, only the covariant parameter is widened from Nothing to Any.

[szymon-rd]

Sorry, but I am still missing a few steps on how you went from inlined fromHas with compiletime.codeOf to the code you've sent that you use in your library. Inference is contextual, and printing the .codeOf in inlined def shows just one of the ways that the inference may behave. If you show a full example of what you want to achieve for your users, just a short example of working code using your library as it has to be now vs as you'd like it to be, I will be able to be sure that we are talking about the same problem.

[neko-kai]

@szymon-rd
Hi, the inference in this case doesn't change between example above and real usage, despite the example not using implicits, just compiletime.codeOf. An example with real libraries where this error happens:

import distage.{Id, Injector, ModuleDef}
import izumi.functional.bio.UnsafeRun2
import zio.{Task, ZIO}

@main def main: Unit = {
  val printEffect: ZIO[Any, Nothing, Unit] = ZIO.succeed(println("Hello World"))
  // val printEffect: ZIO[Any, Throwable, Unit] = ZIO.succeed(println("Hello World")) // using Throwable error instead of Nothing works fine

  val module = new ModuleDef {
    make[Unit].fromZEnv(printEffect) // not ok, inferred type ZIO[Any, Any, Unit], runtime error
    make[Unit].named("fromEffect").fromEffect(printEffect) // this is ok, inferred type ZIO[Any, Nothing, Unit]
  }

  val program = Injector[Task]().produceRun(module) {
    (u1: Unit, u2: Unit @Id("fromEffect")) =>
      ZIO.unit
  }

  UnsafeRun2.createZIO().unsafeRun(program)
}

Scastie (Scala 3.3.1-RC4) - https://scastie.scala-lang.org/jdqXjAyRT0ScO50vCOEwTg
This infers correctly on Scala 2 - https://scastie.scala-lang.org/hWm5TTQAQBOzeSwvfTK18g

Note that the incorrect widening only happens when the error type of printEffect is Nothing - if it's Throwable it is not widened to Any.

Note that the error happens at runtime, not compiletime:

- {type.scala.Unit} (main.scala:10) := effect[λ %1 → zio.ZIO[-scala.Any,+scala.Any,+1]]{effect.{type.scala.Unit}/zio.ZIO[-scala.Any,+scala.Any,+scala.Unit]}
    
    Can't execute an effect in `λ %1 → zio.ZIO[-scala.Any,+scala.Any,+1]` which is neither equivalent to `izumi.fundamentals.platform.Identity`, nor a subtype of the Injector's effect type: `λ %0 → zio.ZIO[-scala.Any,+java.lang.Throwable,+0]`
    
      - To execute `make[_].fromEffect` and `make[_].fromResource` bindings for effects other than `Identity`, you must parameterize the `Injector` with the corresponding effect type when creating it, as in `Injector[F]()`.
      - Subtype type constructors are allowed. e.g. when using ZIO you can execute effects in `IO[Nothing, _]` when using an `Injector[IO[Throwable, _]]()`.

But the cause is due to inference – the E: Tag implicit macro in def fromZEnv[R: ZEnvConstructor, E: Tag, I <: T: Tag](effect: ZIO[R, E, I]): AfterBind = { produces Tag[Any] instead of Tag[Nothing] because Nothing was widened to Any. This in turn causes an incorrect effect type to be assigned to the binding, λ %1 → zio.ZIO[-scala.Any,+scala.Any,+1] instead of λ %1 → zio.ZIO[-scala.Any,+scala.Nothing,+1].
The inference issue is confirmed to not be caused by the implicit itself, since i happens in the first example with no implicits – with simple codeOf.

[szymon-rd]

@neko-kai Have you considered tricking the inference a bit?

case class ZIO[-R, +E, +A]()
object ZIO {
  val interrupt: ZIO[Any, Nothing, Nothing] = ZIO()
}

inline def fromHas[T, R, E, A](z: T)(using T <:< ZIO[R, E, A]): String = {
  compiletime.codeOf(identity(z))
}

@main def main: Unit = {
  println(fromHas(ZIO.interrupt))
}

This approach gives you more control over the inference. (cc: @prolativ thanks for help). In this approach, the inference has no way of knowing the variance.

[neko-kai]

@szymon-rd Thank you for the workaround! However, I believe this issue is a bug - it didn't happen on Scala 2 and it only happens with Nothing not any other type.

[szymon-rd]

@sjrd Do you know why only Nothing is treated differently and widened?