Lazy doesn't propagate type-member info about derived type-class instances

[ryan-williams]

scalafiddle

Scala 2.12.8, Shapeless 2.3.3 (also tried from #797 branch)

Description

Derivations involving shapeless.Lazy don't correctly propagate type-member info in derived type-class instances.

Source

import shapeless.{ the, Lazy }

// shapeless.Lazy interferes with derivations of type-classes with type-members

trait CC  // stand-in for a case-class that we with to derive a typeclass instance `TC` for
trait L   // stand-in for a generic HList Repr of the case-class `CC`
trait O   // output-type for `L` (and ideally `CC`) from the typeclass `TC` below

// Imitation of shapeless.Generic.Aux
// - sole instance binds L as CC's "Repr"
// - theoretically easier to resolve as the "Repr" type-member is instead a type-param
trait G[T, Repr]; object G { implicit val g: G[CC, L] = null }

// Typeclass with one instance mapping `L` to `O`
trait TC[In] { type Out }
object TC {
  type Aux[In, Out_] = TC[In] { type Out = Out_ }
  implicit val base: TC.Aux[L, O] = null
}

// A derivation of typeclass `TC` for the case-class `CC` in terms of its Repr `L` and an existing TC[L], which reuses the same `Out` type
// THe "eager" version works as expected, but the "lazy" version does not.
object derive {
  implicit def eager_[T](implicit g: G[T, L], tc:      TC[L]) : TC.Aux[T, tc      .Out] = null  // 👌🏻
  implicit def lazy_ [T](implicit g: G[T, L], tc: Lazy[TC[L]]): TC.Aux[T, tc.value.Out] = null  // 🐉
}

object test {
  {
    import derive.eager_
    the[TC    [CC   ]]  // ✅
    the[TC.Aux[CC, O]]  // ✅
  }

  {
    import derive.lazy_
    the[TC    [CC   ]]                   // ✅
    the[TC.Aux[CC, O]]                   // 🚫: Lazy breaks the derivation's awareness of the "output" type, for some reason

    // Further checks:
    the[TC    [CC     ]]: TC.Aux[CC, O]  // ✅
    the[TC    [CC     ]](lazy_)          // ✅
    the[TC.Aux[CC, O]](lazy_)            // 🚫: "error: polymorphic expression cannot be instantiated to expected type; "
    the[TC.Aux[CC, O]](lazy_[CC])        // ✅
  }
}

Console

import shapeless._
{ import derive.eager_; the[TC    [CC   ]]                }  // ✅: res0: TC.Aux[CC,TC.base.Out] = null
{ import derive.eager_; the[TC.Aux[CC, O]]                }  // ✅: res1: TC.Aux[CC,TC.base.Out] = null
{ import derive.lazy_ ; the[TC    [CC   ]]                }  // ✅: res2: TC[CC]{type Out = O} = null
{ import derive.lazy_ ; the[TC    [CC   ]]: TC.Aux[CC, O] }  // ✅: res3: TC.Aux[CC,O] = null
{ import derive.lazy_ ; the[TC.Aux[CC, O]]                }  // 🚫
// <console>:15: error: could not find implicit value for parameter t: TC.Aux[CC,O]
//   { import derive.lazy_ ; the[TC.Aux[CC, O]]           }  // 🚫
//                              ^

The difference between res2/res3 (Lazy summons that don't specify the output type, and therefore work) and res0/res1 ("eager" summons that omit and specify the output type, resp.) are intriguing. The latter seem to keep an awareness of the Aux alias and origin of the output type (TC.base.Out), whereas the former seem to have a subtly different representation.

[ryan-williams]

A longer, mostly-unrelated write-up of my upstream use-case that led to this issue is in this gist, along with a previous version of this issue from when I'd not narrowed down the problem as much.

[joroKr21]

Hmm tc.value.Out looks suspicious to me. I don't think value has a refined type?

[ryan-williams]

I don't totally know what that means, but it sounds plausible!

Is there a different way to propagate output-types through (potentially recursive) derivations?

[joroKr21]

I can check tomorrow but maybe it's a type inference issue (e.g. doing an extra call to generic constructor such as Lazy would widen the type. Try to construct the expression that would be implicitly resolved by hand.

The solution is usually to use an extra type parameter and the Aux pattern.

[milessabin]

I think that @joroKr21 is looking in the right place. Try,

implicit def lazy_ [T, O](implicit g: G[T, L], tc: Lazy[Aux[L, O]]): TC.Aux[T, O] = null

[joroKr21]

Confirmed that Aux works:

implicit def lazy_ [T, O](implicit g: G[T, L], tc: Lazy[TC.Aux[L, O]]): TC.Aux[T, O] = null

This is not an issue.