Add ContT monad #2506

johnynek · 2018-09-16T21:09:14Z

ContT is a challenging Monad to implement if you need to implement tailRecM.

The addition of the Defer typeclass solves this problem. This PR replaces the unmerged #1400.

djspiewak · 2018-09-16T22:13:03Z

Not to be Debbie Downer, but this (like all Cont implementations) still doesn’t have a stack safe tailRecM given a truly arbitrary value of Cont. Specifically, arbitrary left-associated flattens sequenced through continue should blow the stack. This is encodable via tailRecM just as it is via flatMap. There are older examples of this in the cats-effect history (IO is basically Defer + Cont).

To be clear, I don’t think that’s a problem. We softened the laws on tailRecM specifically to allow this situation. Just pointing it out. I should be able to contrive a full example in terms of this when I’m not on my phone.

johnynek · 2018-09-16T22:43:00Z

It’s an interesting question how many current Monads may fail this test with better generators.

It may be that this isn’t stack safe with more aggressive generators but the use of AndThen doesn’t make it totally obvious to me that is the case. The WithCont/MapCont isn’t safe I guess in recursions.

In any case, without Defer this implementation fails, which I think shows a nice example of using Defer.

kailuowang · 2018-09-17T15:10:45Z

to save you some time digging travis log, scalastyle was complaining

[info]scalastyle using config /home/travis/build/typelevel/cats/scalastyle-config.xml
[error]/home/travis/build/typelevel/cats/core/src/main/scala/cats/data/ContT.scala:50:8: Public method must have explicit type
[error]/home/travis/build/typelevel/cats/core/src/main/scala/cats/data/ContT.scala:54:8: Public method must have explicit type
[error]/home/travis/build/typelevel/cats/core/src/main/scala/cats/data/ContT.scala:58:8: Public method must have explicit type
[error]/home/travis/build/typelevel/cats/core/src/main/scala/cats/data/ContT.scala:9:51: Use 'type class' instead of typeclass (See #441)

core/src/main/scala/cats/data/ContT.scala

johnynek · 2018-09-18T00:11:02Z

okay, interestingly, using Defer on each call in map and flatMap makes this a StackSafeMonad, but I went ahead and kept the custom implementation which I think should be more efficient.

What do you think now @djspiewak ? I think this may actually be fully stack safe (at the expense of requiring to defer inside each map/flatMap).

djspiewak · 2018-09-18T00:13:57Z

Reading and pondering… :-D

djspiewak · 2018-09-18T00:23:06Z

core/src/main/scala/cats/data/ContT.scala

+    ContT[M, A, C] { fn2 =>
+      val contRun: ContT[M, A, C] => M[A] = (_.run(fn2))
+      val fn3: B => M[A] = fnAndThen.andThen(contRun)
+      M.defer(run(fn3))


I stand very much corrected here. I'm convinced that this is stack-safe. I owe you an apology, @johnynek. Dear internets, please link here if you're looking for a handy example of me being wrong.

When I first reviewed this PR on my phone, it looked like a slightly different version of the trick you used in #1400, but this is considerably better. You're basically inheriting the stack-safety of the underlying monad by explicitly hitting a stack-safe join at this point and returning to its trampoline (which it must have due to Defer). I'm not sure you even need the AndThen, since the stack is cut by the defer.

We all make mistakes here and there. I very much admire folks who feel no shame about it. Kudos to you.

djspiewak · 2018-09-18T00:23:24Z

core/src/main/scala/cats/data/ContT.scala

+  def later[M[_], A, B](fn: => (B => M[A]) => M[A]): ContT[M, A, B] =
+    DeferCont(() => FromFn(AndThen(fn)))
+
+  def tailRecM[M[_], A, B, C](a: A)(fn: A => ContT[M, C, Either[A, B]])(implicit M: Defer[M]): ContT[M, C, B] =


Can just delegate to flatMap now. Probably slightly slower than this though.

djspiewak · 2018-09-18T00:27:25Z

core/src/main/scala/cats/data/ContT.scala

+        DeferCont(() => c)
+    }
+
+  implicit def catsDataContTMonad[M[_]: Defer, A]: Monad[ContT[M, A, ?]] =


Random thought…

What if we implicitly prioritize a bit here? ContT is still useful with very short bind chains even when the underlying monad is not Defer, it just wouldn't be stack-safe. So make the higher priority Monad instance a Monad with Defer and require the Defer[M] constraint. Provide a lower priority instance which doesn't require or provide the Defer[M] (and thus also has a stack-unsafe tailRecM). Obviously this means removing the flatMap and map implementations from the ContT class itself and only making them visible via implicit enrichment (if the imports are a problem for usability, we can with MonadSyntax on the companion object).

The advantage would be a more broadly applicable ContT (admittedly, I'm not sure how much people care about continuations on the JVM outside of async effects, so… maybe not a big advantage?) which transparently preserves stack-safety whenever possible, and gracefully degrades whenever not. Maybe this is taking the "tailRecM doesn't really need to be stack-safe to be lawful" idea too far though.

djspiewak · 2018-09-18T00:28:39Z

core/src/main/scala/cats/data/ContT.scala

+object ContT {
+
+  // Note, we only have two instances of ContT in order to be gentle on the JVM JIT
+  // which treats classes with more than two subclasses differently


You can just make ContT take runAndThen as a parameter to get the same benefit without the subtype limit. :-) A more impactful micro-optimization would be to make the implementations within ContT marked as final, since then it won't matter how many subtypes you have.

djspiewak · 2018-09-18T00:31:04Z

core/src/main/scala/cats/data/ContT.scala

+    apply { cb => cb(b) }
+
+  def apply[M[_], A, B](fn: (B => M[A]) => M[A]): ContT[M, A, B] =
+    FromFn(AndThen(fn))


I'd also like to see an applyPure or something like that defined as:

def applyPure[M[_]: Applicative, A, B](fn: (B => A) => A): ContT[M, A, B] = apply[M, A, B](_(fn.andThen(_.pure)).pure)

Or thereabouts…

actually, I don't think we can do that I think we need Applicative and Comonad (we need to go B => M[A] to B => A to call fn.

Oh I see what you mean. That's definitely unfortunate. I'm not sure the function would be useful at all requiring Comonad

easel

This one's largely above my head, but given that, I went looking for the documentation to figure it out and didn't find any. Probably would be good to document the motivating use case and how to use it before merging.

ceedubs · 2018-10-06T13:29:14Z

I'd also be interested in the motivation for this. I've read about Cont and ContT a few times, but I've never seen them used in the wild. As someone who doesn't have much intuition for them, in the use cases that people describe it seems to me like a pull- based stream API like fs2 would be a cleaner solution. But I'm definitely open to learning more about this approach. Do you have a current use-case for this?

johnynek · 2018-10-07T00:44:54Z

I mostly did it to show we can get a safe tailRecM.

Whatever. I’ll let y’all close it if you don’t want it. Not here to try to change minds about it.

rossabaker · 2018-10-07T01:32:23Z

@iravid has an interesting proof of concept that shows an akka-http-like DSL expressed with ContT.

kailuowang

I am not very familiar with the subject matter, but the code looks sane. Since the API surface is small and it's a data structure rather than typeclass, I vote that we merge and let it out into the wild to test.

kailuowang · 2018-11-08T18:55:01Z

@djspiewak is there anything else you would like to be addressed in this PR?

lemastero · 2018-11-15T21:22:39Z

I'd also be interested in the motivation for this. I've read about Cont and ContT a few times, but I've never seen them used in the wild. As someone who doesn't have much intuition for them, in the use cases that people describe it seems to me like a pull- based stream API like fs2 would be a cleaner solution. But I'm definitely open to learning more about this approach. Do you have a current use-case for this?

TLDR; there are many :)

People do mind blowing things with ContT. Usually they write interpreters, games, web servers, streaming libraries:

Last one is special ❤️ to me - That is how I got hooked by FP. Long before I learned Scala. When ContT lands in Cats, I will revisit my plan to write optimizing Lisp compiler using CPS in Scala :)

There are more sophisticated applications as well: Is there a real-world applicability for the continuation monad outside of academic use?

And it is good to have this implemented in library, with PR that passed review 👀 like this one. Otherwise everyone will need to write it by himself.

djspiewak · 2018-11-15T22:47:48Z

@kailuowang Sorry missed this. Nothing more. I'm 👍

kailuowang · 2018-11-16T01:10:14Z

Thanks @djspiewak! I am merging.

fommil · 2018-11-16T16:01:30Z

a stack safe ContT in Scala is an achievement, well done @johnynek !

SemanticBeeng · 2018-11-20T10:45:17Z

There are more sophisticated applications as well: [Is there a real-world applicability for the continuation monad outside of academic use?]

Sorry if being lazy but how to best compare ConT monad and Scala delimited continuations ?
Asking in context of Lantern https://github.com/feiwang3311/Lantern.

Add ContT monad

dd59a53

johnynek assigned non Sep 16, 2018

kailuowang added this to the 1.5 milestone Sep 17, 2018

johnynek added 2 commits September 17, 2018 09:31

improve implementation

adc96ca

be a bit more efficient with allocations

bda7d91

johnynek commented Sep 17, 2018

View reviewed changes

core/src/main/scala/cats/data/ContT.scala Outdated Show resolved Hide resolved

defer on each call that returns M[A]

bcf6b53

djspiewak reviewed Sep 18, 2018

View reviewed changes

use final, covariant on B

ed1cebc

easel suggested changes Sep 25, 2018

View reviewed changes

kailuowang approved these changes Nov 6, 2018

View reviewed changes

kailuowang modified the milestones: 1.5-RC0, 1.5.0-RC1 Nov 16, 2018

kailuowang merged commit 7b456fe into master Nov 16, 2018

kailuowang deleted the oscar/contt branch November 16, 2018 16:14

kailuowang added the enhancement label Nov 19, 2018

bblfish mentioned this pull request Aug 22, 2019

Continuation monads #700

Open

johnynek mentioned this pull request Aug 9, 2021

Stack-safety issues in Kleisli instances #3947

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add ContT monad #2506

Add ContT monad #2506

johnynek commented Sep 16, 2018

djspiewak commented Sep 16, 2018 •

edited

Loading

johnynek commented Sep 16, 2018

kailuowang commented Sep 17, 2018

johnynek commented Sep 18, 2018

djspiewak commented Sep 18, 2018

djspiewak Sep 18, 2018

johnynek Sep 18, 2018

djspiewak Sep 18, 2018

djspiewak Sep 18, 2018

djspiewak Sep 18, 2018

djspiewak Sep 18, 2018 •

edited

Loading

johnynek Sep 18, 2018

djspiewak Sep 18, 2018

easel left a comment

ceedubs commented Oct 6, 2018

johnynek commented Oct 7, 2018

rossabaker commented Oct 7, 2018

kailuowang left a comment •

edited

Loading

kailuowang commented Nov 8, 2018

lemastero commented Nov 15, 2018

djspiewak commented Nov 15, 2018

kailuowang commented Nov 16, 2018

fommil commented Nov 16, 2018

SemanticBeeng commented Nov 20, 2018

Add ContT monad #2506

Add ContT monad #2506

Conversation

johnynek commented Sep 16, 2018

djspiewak commented Sep 16, 2018 • edited Loading

johnynek commented Sep 16, 2018

kailuowang commented Sep 17, 2018

johnynek commented Sep 18, 2018

djspiewak commented Sep 18, 2018

djspiewak Sep 18, 2018

Choose a reason for hiding this comment

johnynek Sep 18, 2018

Choose a reason for hiding this comment

djspiewak Sep 18, 2018

Choose a reason for hiding this comment

djspiewak Sep 18, 2018

Choose a reason for hiding this comment

djspiewak Sep 18, 2018

Choose a reason for hiding this comment

djspiewak Sep 18, 2018 • edited Loading

Choose a reason for hiding this comment

johnynek Sep 18, 2018

Choose a reason for hiding this comment

djspiewak Sep 18, 2018

Choose a reason for hiding this comment

easel left a comment

Choose a reason for hiding this comment

ceedubs commented Oct 6, 2018

johnynek commented Oct 7, 2018

rossabaker commented Oct 7, 2018

kailuowang left a comment • edited Loading

Choose a reason for hiding this comment

kailuowang commented Nov 8, 2018

lemastero commented Nov 15, 2018

djspiewak commented Nov 15, 2018

kailuowang commented Nov 16, 2018

fommil commented Nov 16, 2018

SemanticBeeng commented Nov 20, 2018

djspiewak commented Sep 16, 2018 •

edited

Loading

djspiewak Sep 18, 2018 •

edited

Loading

kailuowang left a comment •

edited

Loading