Add ability to create unpersistent versions of persistent behaviors #31464

leviramsey · 2022-07-07T23:09:12Z

Provides an alternative interpreter for EventSourcedBehavior and DurableStateBehavior that does not actually involve any persistence: persistence operations are exposed via a side-channel from the actor.

The major benefit of this approach is that the lower-overhead fully-synchronous BehaviorTestKit can now be used to test many such behaviors (conditional, of course, on the behavior in question not performing operations which are unsupported in the BehaviorTestKit: #30050 removes ActorContext.ask from that set of operations). This lower overhead results in somewhat faster tests (see https://gist.github.com/leviramsey/741e64fc9de472e7782aad3ddf40804c) and makes things like property-based testing of persistent behaviors practical.

Other minor benefits of this approach might include:

same testing style for both durable-state and event-sourcing
better ergonomics for behaviors which delay replies until some other message is received (stash, later performing an ask) than the EventSourcedBehaviorTestKit
middle ground for testing mostly domain logic along with some of how the domain logic affects the world outside (compare with an approach that, e.g. has domain command handlers returning a sequence of events and some reification of a program with side effects (e.g. a free monad))

patriknw

This is looking good as an alternative synchronous testkit similar to BehaviorTestKit.

"Unpersistent" sounds a bit funny to me.

leviramsey · 2022-07-08T14:25:54Z

Yeah, I bounced between unpersistent and nonpersistent... NotPersistent could also be an option.

leviramsey · 2022-07-08T14:26:39Z

akka-persistence/src/main/scala-3/akka/persistence/TraitOrder.scala

Inadvertently did an applyCodeStyle while in Scala-3 mode...

leviramsey · 2022-09-09T15:46:55Z

One bit of functionality that's not implemented yet is the lastSequenceNumber. The easiest way to do that could be to have the unpersistent and persistent versions extend a HasSequenceNumber trait (to avoid having EventSourcedBehavior.lastSequenceNumber have to discriminate between possible implementations), which would have binary compatibility implications?

johanandren · 2022-09-09T15:55:12Z

Running.WithSeqNrAccessible already is a trait, and is completely internal so should be fine to use in more places, couldn't that work out?

leviramsey · 2022-09-09T17:05:03Z

Okay, hadn't noticed that @johanandren

lightbend-cla-validator · 2022-09-13T15:43:34Z

Hi @leviramsey,

Thank you for your contribution! We really value the time you've taken to put this together.

We see that you have signed the Lightbend Contributors License Agreement before, however, the CLA has changed since you last signed it.
Please review the new CLA and sign it before we proceed with reviewing this pull request:

https://www.lightbend.com/contribute/cla

leviramsey · 2022-09-13T15:45:22Z

rebasing on BSL...

akka-persistence-testkit/src/main/scala/akka/persistence/testkit/ChangePersisted.scala

akka-persistence-testkit/src/main/scala/akka/persistence/testkit/internal/Unpersistent.scala

patriknw

Looking good. I guess I'm slowly getting used to the naming Unpersistent.

akka-persistence-testkit/src/main/scala/akka/persistence/testkit/ChangePersisted.scala

akka-persistence-testkit/src/main/scala/akka/persistence/testkit/internal/Unpersistent.scala

patriknw · 2022-09-21T14:08:38Z

...sistence-testkit/src/main/scala/akka/persistence/testkit/scaladsl/UnpersistentBehavior.scala

+object UnpersistentBehavior {
+
+  type BehaviorAndChanges[Command, Event, State] =
+    (Behavior[Command], ConcurrentLinkedQueue[ChangePersisted[State, Event]])


could be more clear to define this as a case class instead of a tuple

...sistence-testkit/src/main/scala/akka/persistence/testkit/scaladsl/UnpersistentBehavior.scala

akka-persistence-testkit/src/main/scala/akka/persistence/testkit/ChangePersisted.scala

leviramsey · 2022-09-21T17:51:38Z

...sistence-testkit/src/main/scala/akka/persistence/testkit/scaladsl/UnpersistentBehavior.scala

+   */
+  def fromEventSourced[Command, Event, State](
+      behavior: Behavior[Command],
+      fromStateAndOffset: Option[(State, Long)] = None): (Behavior[Command], PersistenceProbe[Event], PersistenceProbe[State]) = {


The javadsl exposes a wrapper class instead of a Tuple3, though in the Java API, that exposes a "guaranteed-to-fail" probe for durable state. Could get around that with 4 substantially-identical result types ("Java or Scala" x "Event-sourced or Durable-state"), but that strikes me as a "cure might be worse than the disease" situation

...sistence-testkit/src/main/scala/akka/persistence/testkit/scaladsl/UnpersistentBehavior.scala

leviramsey · 2022-09-21T17:55:32Z

Have not yet adjusted any of the tests ~~or docs~~ for the new PersistenceProbe API, btw.

...rsistence-testkit/src/main/scala/akka/persistence/testkit/javadsl/UnpersistentBehavior.scala

akka-persistence-testkit/src/main/scala/akka/persistence/testkit/internal/Unpersistent.scala

patriknw · 2022-10-07T09:52:36Z

@leviramsey Where are we here? Ready for final review?

leviramsey · 2022-10-07T14:02:33Z

Basically just docs (including adding a testing section for durable state in general, covering some of the other approaches). The code stuff should be OK to review now.

(or should we delay docs for another PR?)

johanandren

Another round of review, mostly looking at the API, would be good with a small Java test to see the ergonomics using that API as well (even if I think they'll be pretty 1:1)

...rsistence-testkit/src/main/scala/akka/persistence/testkit/javadsl/UnpersistentBehavior.scala

...-testkit/src/test/scala/akka/persistence/testkit/scaladsl/UnpersistentDurableStateSpec.scala

johanandren · 2022-10-07T14:39:48Z

...-testkit/src/test/scala/akka/persistence/testkit/scaladsl/UnpersistentDurableStateSpec.scala

+      val recoveryDone = TestInbox[Done]()
+      val behavior = BehaviorUnderTest("test-1", recoveryDone.ref)
+
+      val (unpersistent, probe) = UnpersistentBehavior.fromDurableState[Command, State](behavior)


I know it is just testkit and not production API but returning tuples is not great DX, let's make a result type for Scala as well instead.

So with a result type patterned on the Java API, you end up with code like

val unpersistent = UnpersistentBehavior.fromDurableState[Command, State](behavior) val probe = unpersistent.stateProbe val testkit = unpersistent.behaviorTestKit

or with an extractor, something like the tuple return

val UnpersistentBehavior.DurableState(testkit, probe) = UnpersistentBehavior.fromDurableState[Command, State](behavior)

OTOH, in the Scala API, maybe it makes the most sense to have the result type take a (BehaviorTestKit[Command], PersistenceProbe[Event], PersistenceProbe[State]) => Unit (or otherwise for durable state) in addition to the accessors?

So it would be like

UnpersistentBehavior.fromDurableState[Command, State](behavior) { (testkit, probe) => // body of test }

That does make the Java API rather different from the Scala API, but I don't get the sense that that sort of resource-like API is at all idiomatic in Java.

A case class so that it is possible to use the unapply if you like, or the object with named fields if you rater like that sounds good to me (better than 3-tuple).

Edit: ignore, comment on previous state of PR, not sure why GitHub didn't show me the latest one.

patriknw

Looking very good. "Only" reference docs remaining.
And a few nitpicks...

...rsistence-testkit/src/main/scala/akka/persistence/testkit/javadsl/UnpersistentBehavior.scala

...sistence-testkit/src/main/scala/akka/persistence/testkit/scaladsl/UnpersistentBehavior.scala

...rsistence-testkit/src/main/scala/akka/persistence/testkit/javadsl/UnpersistentBehavior.scala

akka-persistence-testkit/src/main/scala/akka/persistence/testkit/internal/Unpersistent.scala

...rsistence-testkit/src/main/scala/akka/persistence/testkit/javadsl/UnpersistentBehavior.scala

…xample

leviramsey · 2022-10-17T00:51:57Z

For the doc structure, I'm thinking to make typed/persistence-testing.md be like typed/testing.md with links to the different options (as well as mentioning explicitly that in many cases the event-handling logic can be tested as just a function):

typed/persistence-testing-unpersistent.md
typed/persistence-testing-eventsourcedbehaviortestkit.md
typed/persistence-testing-persistencetestkit.md

And also adding a couple of durable state testing pages:

typed/persistence-testing-durablestate.md
typed/persistence-testing-durablestate-unpersistent.md
typed/persistence-testing-durablestate-async.md

johanandren · 2022-10-17T06:32:42Z

Sounds good, but I think it would also be ok to scope the doc effort down and add a section to the existing testing page for typed persistence. Up to you what you have time to complete, we can always add more docs later if needed.

leviramsey · 2022-10-17T23:53:01Z

For now, just adding the section to typed (event-sourced) persistence.

Documenting the testing of durable state behaviors in general is a gap in the docs (AFAIK, the main way before this to test them is to combine the PersistenceTestKitPlugin with PersistenceTestKitDurableStateStorePlugin and configure the ActorTestKit appropriately). Documenting the previous method and this can be a separate issue I guess.

If #31673 is merged before this, I will probably adjust the doc snippets to use that (or vice versa, do it in that PR).

johanandren

Some tiny things then this looks good to go

...sistence-testkit/src/main/scala/akka/persistence/testkit/scaladsl/UnpersistentBehavior.scala

akka-persistence-testkit/src/main/scala/akka/persistence/testkit/internal/Unpersistent.scala

johanandren · 2022-10-18T07:16:54Z

...-testkit/src/test/scala/akka/persistence/testkit/scaladsl/UnpersistentDurableStateSpec.scala

+      val recoveryDone = TestInbox[Done]()
+      val behavior = BehaviorUnderTest("test-1", recoveryDone.ref)
+
+      val (unpersistent, probe) = UnpersistentBehavior.fromDurableState[Command, State](behavior)


A case class so that it is possible to use the unapply if you like, or the object with named fields if you rater like that sounds good to me (better than 3-tuple).

Edit: ignore, comment on previous state of PR, not sure why GitHub didn't show me the latest one.

patriknw

LGTM

johanandren

LGTM (I don't think my comments are blocking merge on second thought)

patriknw reviewed Jul 8, 2022

View reviewed changes

leviramsey commented Jul 8, 2022

View reviewed changes

leviramsey added 3 commits September 13, 2022 11:43

first cut at unpersistent event-sourced and durable-state behaviors

204fac2

code style

c8685be

scala 3

507f252

leviramsey force-pushed the unpersistent branch from 051d4f4 to 507f252 Compare September 13, 2022 15:43

Support querying sequence/revision number

82a2e68

johanandren reviewed Sep 15, 2022

View reviewed changes

WIP: move to PersistenceProbe

5b7071b

patriknw reviewed Sep 21, 2022

View reviewed changes

Still WIP: feedback

4a41553

leviramsey commented Sep 21, 2022

View reviewed changes

...sistence-testkit/src/main/scala/akka/persistence/testkit/scaladsl/UnpersistentBehavior.scala Show resolved Hide resolved

leviramsey commented Sep 21, 2022

View reviewed changes

...rsistence-testkit/src/main/scala/akka/persistence/testkit/javadsl/UnpersistentBehavior.scala Outdated Show resolved Hide resolved

johanandren reviewed Sep 23, 2022

View reviewed changes

akka-persistence-testkit/src/main/scala/akka/persistence/testkit/internal/Unpersistent.scala Outdated Show resolved Hide resolved

leviramsey added 5 commits September 23, 2022 09:38

remove ChangePersisted

8903838

scaladsl.PersistenceEffect, s/offset/sequenceNr/ in public APIs

c16b8c7

actually finish the scaladsl.PersistenceEffect migration

4fc1147

style

5d8c76f

tests now use new API

0df8cc7

leviramsey marked this pull request as ready for review October 7, 2022 14:02

johanandren reviewed Oct 7, 2022

View reviewed changes

patriknw added the 2 - pick next Used to mark issues which are next up in the queue to be worked on. The tag is non-binding label Oct 11, 2022

Merge branch 'main' into unpersistent

38f4921

patriknw reviewed Oct 13, 2022

View reviewed changes

leviramsey added 7 commits October 14, 2022 10:35

Merge branch 'main' into unpersistent

51da903

feedback and API tweaks (especially in Scala)

82be6e4

stylin

7d992f7

Another stylish straggler?

ac6d23d

Add Unpersistent counterpart to javadsl EventSourcedBehaviorTestKit e…

436e04b

…xample

scaladsl counterpart to EventSourcedBehaviorTestKit example

7fea99c

format etc.

59ecfe1

add event-sourced docs

31ac089

johanandren reviewed Oct 18, 2022

View reviewed changes

patriknw approved these changes Oct 18, 2022

View reviewed changes

johanandren approved these changes Oct 18, 2022

View reviewed changes

johanandren merged commit e7448f8 into akka:main Oct 18, 2022

johanandren added this to the 2.7.0 milestone Oct 18, 2022

johanandren mentioned this pull request Oct 18, 2022

fix: Unpersistent now compiling on 2.12 #31684

Merged

leviramsey deleted the unpersistent branch October 18, 2022 10:25

leviramsey mentioned this pull request Oct 18, 2022

fix: Unpersistent order of event application #31686

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add ability to create unpersistent versions of persistent behaviors #31464

Add ability to create unpersistent versions of persistent behaviors #31464

leviramsey commented Jul 7, 2022

patriknw left a comment

leviramsey commented Jul 8, 2022

leviramsey Jul 8, 2022

leviramsey commented Sep 9, 2022

johanandren commented Sep 9, 2022

leviramsey commented Sep 9, 2022

lightbend-cla-validator commented Sep 13, 2022

leviramsey commented Sep 13, 2022

patriknw left a comment

patriknw Sep 21, 2022

leviramsey Sep 21, 2022

leviramsey commented Sep 21, 2022 •

edited

Loading

patriknw commented Oct 7, 2022

leviramsey commented Oct 7, 2022

johanandren left a comment

johanandren Oct 7, 2022

leviramsey Oct 14, 2022 •

edited

Loading

leviramsey Oct 15, 2022

johanandren Oct 18, 2022 •

edited

Loading

patriknw left a comment

leviramsey commented Oct 17, 2022

johanandren commented Oct 17, 2022

leviramsey commented Oct 17, 2022

johanandren left a comment •

edited

Loading

johanandren Oct 18, 2022 •

edited

Loading

patriknw left a comment

johanandren left a comment

Add ability to create unpersistent versions of persistent behaviors #31464

Add ability to create unpersistent versions of persistent behaviors #31464

Conversation

leviramsey commented Jul 7, 2022

patriknw left a comment

Choose a reason for hiding this comment

leviramsey commented Jul 8, 2022

leviramsey Jul 8, 2022

Choose a reason for hiding this comment

leviramsey commented Sep 9, 2022

johanandren commented Sep 9, 2022

leviramsey commented Sep 9, 2022

lightbend-cla-validator commented Sep 13, 2022

leviramsey commented Sep 13, 2022

patriknw left a comment

Choose a reason for hiding this comment

patriknw Sep 21, 2022

Choose a reason for hiding this comment

leviramsey Sep 21, 2022

Choose a reason for hiding this comment

leviramsey commented Sep 21, 2022 • edited Loading

patriknw commented Oct 7, 2022

leviramsey commented Oct 7, 2022

johanandren left a comment

Choose a reason for hiding this comment

johanandren Oct 7, 2022

Choose a reason for hiding this comment

leviramsey Oct 14, 2022 • edited Loading

Choose a reason for hiding this comment

leviramsey Oct 15, 2022

Choose a reason for hiding this comment

johanandren Oct 18, 2022 • edited Loading

Choose a reason for hiding this comment

patriknw left a comment

Choose a reason for hiding this comment

leviramsey commented Oct 17, 2022

johanandren commented Oct 17, 2022

leviramsey commented Oct 17, 2022

johanandren left a comment • edited Loading

Choose a reason for hiding this comment

johanandren Oct 18, 2022 • edited Loading

Choose a reason for hiding this comment

patriknw left a comment

Choose a reason for hiding this comment

johanandren left a comment

Choose a reason for hiding this comment

leviramsey commented Sep 21, 2022 •

edited

Loading

leviramsey Oct 14, 2022 •

edited

Loading

johanandren Oct 18, 2022 •

edited

Loading

johanandren left a comment •

edited

Loading

johanandren Oct 18, 2022 •

edited

Loading