TestJournal and TestSnaphostStore #3881

valdisz · 2019-08-08T17:26:30Z

PR addresses issue #3877

Added 3 new projects:

Akka.Persistence.TestKit
Akka.Persistence.TestKit.Tests
Akka.Persistence.TestKit.Xunit2

It adds a new journal (TestJournal) and a new snapshot store (TestSnaphostStore). It is possible to intercept calls to them and apply user-defined behavior, i.e. to simulate network problems or anything else.

It adds new test base class PersistenceTestKit (Xunit2 based) to simplify usage. API loos like this:

public class PersistentActorSpec : PersistenceTestKit
{
    [Fact]
    public async Task actor_must_fail_when_journal_will_fail_saving_message()
    {
        await WithJournalWrite(write => write.Fail(), () =>
        {
            var actor = ActorOf(() => new PersistActor());
            Watch(actor);

            actor.Tell("write", TestActor);
            ExpectTerminated(actor);
        });
    }
}

Aaronontheweb · 2019-08-08T17:42:17Z

Build failed due to TestJournalSpec.cs(28,13): error CS0165: Use of unassigned local variable 'journal' [D:\a\1\s\src\core\Akka.Persistence.TestKit.Tests\Akka.Persistence.TestKit.Tests.csproj]

Aaronontheweb

I know this is a very early preview and you have more iterations you're working on, but I wanted to pass along some comments.

src/Akka.sln

src/core/Akka.Persistence.TestKit.Tests/Akka.Persistence.TestKit.Tests.csproj

src/core/Akka.Persistence.TestKit.Tests/test-journal.conf

src/core/Akka.Persistence.TestKit/IJournalWriteInterceptor.cs

Aaronontheweb

Looking good so far - I think we should add a new base class on top of the TestKitBase that makes it more concise to use some of these Journal failure methods when writing a test, The underlying TestJournal itself looks great though.

src/core/Akka.Persistence.TestKit/JournalInterceptors.cs

src/core/Akka.Persistence.TestKit/TestJournal.cs

valdisz · 2019-08-11T12:01:35Z

@Aaronontheweb I have implemented PersistenceTestKit. Is it what you were expecting?

valdisz · 2019-08-12T08:26:00Z

One test in a totally different project (which was not modified) is failing. No idea why.

Aaronontheweb · 2019-08-12T16:18:00Z

@valdisz we have a number of test failures that are due to concurrent tests with hard deadlines blocking threads, We have an open proposal to fix this by moving everything to an async / await model in #3854 - we're going to have to address it in a future release. Don't worry about it for the time being.

Aaronontheweb

Have some more comments I want to add, but I need to do some work with this code locally on my machine first.

src/core/Akka.Persistence.TestKit/PersistenceTestKit.cs

Aaronontheweb

Added some examples of the type of functionality I'd like to see built into the PersistentTestkit class, and some recommendations on moving away from blocking threads and towards async / await implementations

src/core/Akka.Persistence.TestKit.Tests/TestJournalSpec.cs

src/core/Akka.Persistence.TestKit/PersistenceTestKit.cs

src/core/Akka.Persistence.TestKit/JournalWriteBehavior.cs

src/core/Akka.Persistence.TestKit/JournalRecoveryBehavior.cs

Aaronontheweb · 2019-08-13T16:14:15Z

@valdisz this is looking good - what are next steps for implementing this? More overloads on the PersistenceTestKit class?

valdisz · 2019-08-13T16:19:04Z

@Aaronontheweb more unit tests for existing code (some are still needed), then SnaphsotStore and then XML DOC.

Aaronontheweb

This looks great - have some minor changes we need to address (left comments for them) but otherwise I think this is ready to go.

We'll work on getting documentation added to the website for this built into a separate pull request and we'll also plan on releasing this as part of the v1.4.0-beta2 launch, so users can start using it and giving us some feedback. We'll probably consume it inside the Akka.Persistence.Extras project as a starting point.

src/core/Akka.Persistence.TestKit.Tests/TestSnapshotStoreSpec.cs

src/core/Akka.Persistence.TestKit.Xunit2/Akka.Persistence.TestKit.Xunit2.csproj

Aaronontheweb · 2019-08-21T09:40:17Z

src/core/Akka.Persistence.TestKit.Xunit2/PersistenceTestKit.cs

+    /// This class represents an Akka.NET Persistence TestKit that uses <a href="https://xunit.github.io/">xUnit</a>
+    /// as its testing framework.
+    /// </summary>
+    public abstract class PersistenceTestKit : PersistenceTestKitBase


Need to add a constructor overload here which takes an ITestOutputHelper for XUnit - that's how we're able to inject the XUnit output capture system into the Akka.NET logging infrastructure. Here's an example of a unit test where a user would do that:

https://github.com/akkadotnet/akka.net/blob/dev/src/core/Akka.Remote.Tests/RemotingSpec.cs#L31

To correctly handle ITestOutputHelper in test kit I changed the base class of PersistenceTestKit to TestKit. This means that PersistenceTestKitBase no longer exists other test frameworks will not have Test Kit class with persistence testing API out of the box for now.

src/core/Akka.Persistence.TestKit/Akka.Persistence.TestKit.csproj

src/core/Akka.Persistence.TestKit/Journal/JournalRecoveryBehavior.cs

src/core/Akka.Persistence.TestKit/PersistenceTestKitBase.cs

…from xUnit

valdisz · 2019-08-23T09:04:24Z

PR is ready, all work is done. If any issues are found, please comment, otherwise, merge.

Danthar · 2019-08-25T10:50:39Z

We got some errors in Tests, in the MNTK tests related to persistence are failing with casting errors. However as far as i can tell that should not be code this PR changed.

Did we change an IList to an SortedSet somewhere in this PR ?

valdisz · 2019-08-25T20:09:11Z

We got some errors in Tests, in the MNTK tests related to persistence are failing with casting errors. However as far as i can tell that should not be code this PR changed.

Did we change an IList to an SortedSet somewhere in this PR ?

This PR does not change anything except adding 3 new projects to the solution.
For me, it is hard to guess why that particular test failed.
I can dig in, but that will take some time.

Danthar · 2019-08-25T20:13:35Z

Yeah I could not find it either. Hence my question. Just checking if I'm not crazy. So this means a PR was merged which caused problems. Odd that ci didn't catch it. I got some time tomorrow ill see if I can find the offending PR. That is if no one else beats me to it. Op zo 25 aug. 2019 22:09 schreef Valdis Zobēla <notifications@github.com>:

…

We got some errors in Tests, in the MNTK tests related to persistence are failing with casting errors. However as far as i can tell that should not be code this PR changed. Did we change an IList to an SortedSet somewhere in this PR ? This PR does not change anything except adding 3 new projects to the solution. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#3881?email_source=notifications&email_token=AAHQ3F5ASNYCGAVCHTJBCVLQGLRGXA5CNFSM4IKM2MFKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD5C2Y4A#issuecomment-524659824>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAHQ3FZXBLLNYTZ5UJHY62TQGLRGXANCNFSM4IKM2MFA> .

Danthar · 2019-08-26T07:59:22Z

Think i found the issue. Problem happens during json serialization/deserialization. Im thinking this PR is the culprit #3671 Because i cannot find anything else that might cause this.

valdisz · 2019-08-26T10:50:04Z

Surprisingly my next PR does not fail any builds. #3889

Danthar · 2019-08-26T11:17:39Z

That is because we have a system. Which detects what code is affected and then only builds and runs the projects with the affected code. Cuts back on ci time. Since your other PR is a docs only PR. Only the docs project is checked. Op ma 26 aug. 2019 12:50 schreef Valdis Zobēla <notifications@github.com>:

…

Surprisingly my next PR does not fail any builds. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#3881?email_source=notifications&email_token=AAHQ3FZFPJNCGWHQOPUN6ZTQGOYOBA5CNFSM4IKM2MFKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD5EAPOQ#issuecomment-524814266>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAHQ3F2DIYYXSQ7XM4JYU43QGOYOBANCNFSM4IKM2MFA> .

Aaronontheweb

LGTM

Aaronontheweb · 2019-08-26T18:21:52Z

I'll take a look at that failing spec - I think it's due to the JSON.NET upgrade. CI didn't run because I think the common.props file was the only thing that was upgraded. Incrementalist will need to be upgraded to detect that a shared project property file changed.

Aaronontheweb · 2019-08-26T20:28:36Z

#3892

Aaronontheweb · 2019-08-27T17:01:22Z

Having a persistent failure with MNTR specs on .NET Framework only:

##teamcity[testFailed name='Akka.Cluster.Tools.Tests.MultiNode.PublishSubscribe.DistributedPubSubMediatorSpec.DistributedPubSubMediatorSpecs' message='|[NODE2:second|]|[12:33 PM|]: SPEC FAILED: Akka.Cluster.Tools.Tests.MultiNode.PublishSubscribe.DistributedPubSubMediatorSpec.DistributedPubSubMediatorSpecs|r FAIL' details='' timestamp='2019-08-27T12:33:42.162+0000' flowId='11']
[INFO][8/27/2019 12:33:42 PM][Thread 0003][[akka://TestRunnerLogging/user/TcpLogger#1268474739]] Node disconnected on [akka://TestRunnerLogging/system/IO-TCP/$a/$o#557351663]

Unknown message: Node\PublishSubscribe\DistributedPubSubMediatorSpec.cs:line 751
   at Akka.Cluster.Tools.Tests.MultiNode.PublishSubscribe.DistributedPubSubMediatorSpec.<DistributedPubSubMediator_must_remove_entries_when_node_is_removed>b__39_0() in D:\a\1\s\src\contrib\cluster\Akka.Cluster.Tools.Tests.MultiNode\PublishSubscribe\DistributedPubSubMediatorSpec.cs:line 752
   at Akka.TestKit.TestKitBase.<>c__DisplayClass141_0.<Within>b__0() in D:\a\1\s\src\core\Akka.TestKit\TestKitBase_Within.cs:line 48
   at Akka.TestKit.T
Unknown message: estKitBase.Within[T](TimeSpan min, TimeSpan max, Func`1 function, String hint, Nullable`1 epsilonValue) in D:\a\1\s\src\core\Akka.TestKit\TestKitBase_Within.cs:line 101
   at Akka.TestKit.TestKitBase.Within(TimeSpan min, TimeSpan max, Action action, String hint, Nullable`1 epsilonValue) in D:\a\1\s\src\core\Akka.TestKit\TestKitBase_Within.cs:line 49
   at Akka.TestKit.TestKitBase.Within(TimeSpan max, Action action, Nullable`1 epsilonValue) in D:\a\1\s\src\core\Akka.TestKit\TestKitBase_Within.cs:line 32
 
Unknown message:   at Akka.Cluster.Tools.Tests.MultiNode.PublishSubscribe.DistributedPubSubMediatorSpec.DistributedPubSubMediator_must_remove_entries_when_node_is_removed() in D:\a\1\s\src\contrib\cluster\Akka.Cluster.Tools.Tests.MultiNode\PublishSubscribe\DistributedPubSubMediatorSpec.cs:line 761
   at Akka.Cluster.Tools.Tests.MultiNode.PublishSubscribe.DistributedPubSubMediatorSpec.DistributedPubSubMediatorSpecs() in D:\a\1\s\src\contrib\cluster\Akka.Cluster.Tools.Tests.MultiNode\PublishSubscribe\DistributedPubSubMediato
Unknown message: nToken) in D:\a\1\s\src\core\Akka\Actor\Futures.cs:line 128

##teamcity[testFailed name='Akka.Cluster.Tools.Tests.MultiNode.PublishSubscribe.DistributedPubSubMediatorSpec.DistributedPubSubMediatorSpecs' message='|[NODE1:first|]|[12:33 PM|]: SPEC FAILED: Akka.Cluster.Tools.Tests.MultiNode.PublishSubscribe.DistributedPubSubMediatorSpec.DistributedPubSubMediatorSpecs|r FAIL' details='' timestamp='2019-08-27T12:33:47.471+0000' flowId='11']
##teamcity[testFailed name='Akka.Cluster.Tools.Tests.MultiNode.PublishSubscribe.DistributedPubSubMediatorSpec.DistributedPubSubMediatorSpecs' message='|[NODE1:first|]|[12:33 PM|]: SPEC FAILED: Message: Timeout after 00:00:05 seconds|r' details='' timestamp='2019-08-27T12:33:47.471+0000' flowId='11']
##teamcity[testFailed name='Akka.Cluster.Tools.Tests.MultiNode.PublishSubscribe.DistributedPubSubMediatorSpec.DistributedPubSubMediatorSpecs' message='|[NODE1:first|]|[12:33 PM|]: SPEC FAILED: StackTrace:    at System.Threading.Tasks.Task`1.GetResultCore(Boolean waitCompletionNotification)|r' details='' timestamp='2019-08-27T12:33:47.471+0000' flowId='11']
##teamcity[testFailed name='Akka.Cluster.Tools.Tests.MultiNode.PublishSubscribe.DistributedPubSubMediatorSpec.DistributedPubSubMediatorSpecs' message='|[NODE1:first|]|[12:33 PM|]: SPEC FAILED: StackTrace:    at Akka.Actor.Futures.Ask(ICanTell self, Func`2 messageFactory, IActorRefProvider provider, Nullable`1 timeout, CancellationToken cancellationToken) in D:\a\1\s\src\core\Akka\Actor\Futures.cs:line 187|r' details='' timestamp='2019-08-27T12:33:47.471+0000' flowId='11']

Going to take a closer look at this - don't think it's related to these changes.

* TestJournal with Write interception and various failure strategies * Akka.Persistence.TestKit implementation

valdisz changed the title ~~Feedback needed on feature addressing issue #3877~~ Feedback needed on feature addressing specialised Test Journal Aug 8, 2019

Aaronontheweb reviewed Aug 8, 2019

View reviewed changes

valdisz force-pushed the feature/persistence-testkit branch 2 times, most recently from d0aa37f to f640b69 Compare August 9, 2019 21:40

TestJournal with Write intreception and varios failure strategies

46c77c4

valdisz force-pushed the feature/persistence-testkit branch from f640b69 to 46c77c4 Compare August 9, 2019 21:49

valdisz added 2 commits August 10, 2019 15:21

unit tests for message interceptors

10ac5b2

TestJournal OnRecovery behavior

42d7f8c

Aaronontheweb reviewed Aug 10, 2019

View reviewed changes

Recovery unit tests

9dbb8ed

valdisz changed the title ~~Feedback needed on feature addressing specialised Test Journal~~ TestJournal and TestSnaphostStore Aug 11, 2019

PersistenceTestKit base for easier test setup

eaa701c

Aaronontheweb reviewed Aug 12, 2019

View reviewed changes

src/core/Akka.Persistence.TestKit/PersistenceTestKit.cs Outdated Show resolved Hide resolved

Aaronontheweb added 2 commits August 12, 2019 11:22

Merge branch 'dev' into feature/persistence-testkit

4fb4ea9

added example of built-in TestKit method

8e0f81e

Aaronontheweb requested changes Aug 12, 2019

View reviewed changes

use Async api whenver possible

e32dfff

valdisz commented Aug 13, 2019

View reviewed changes

src/core/Akka.Persistence.TestKit/JournalRecoveryBehavior.cs Outdated Show resolved Hide resolved

Merge branch 'dev' into feature/persistence-testkit

17ac3b3

(wip) TestSnapshotStore

aceb9aa

valdisz force-pushed the feature/persistence-testkit branch from 7ed6cef to aceb9aa Compare August 14, 2019 06:27

valdisz added 3 commits August 14, 2019 19:35

TestSnapshotStore interceptors

55b9f92

unit tests for TestSnapshotStore

610999f

separate package with Xunit2 integration

c97481a

valdisz added 2 commits August 18, 2019 00:30

XML doc

a85d950

Journal Write and Recovery behavior documentation

a14b2c9

Aaronontheweb requested changes Aug 21, 2019

View reviewed changes

valdisz added 4 commits August 23, 2019 00:03

removed unneeded statements from csproj files

c05ec27

PersistenceTestKitBase is removed to correctly add ITestOutputHelper …

c4cebe0

…from xUnit

added mor xml docs

1147968

Merge branch 'dev' into feature/persistence-testkit

fa4c150

valdisz marked this pull request as ready for review August 23, 2019 09:03

Aaronontheweb approved these changes Aug 26, 2019

View reviewed changes

Merge branch 'dev' into feature/persistence-testkit

7d50ddd

Aaronontheweb mentioned this pull request Aug 28, 2019

resolved lock-up with DistributedPubSubMediatorSpecs #3896

Merged

Aaronontheweb added 4 commits August 30, 2019 17:34

Merge branch 'dev' into feature/persistence-testkit

bd299f2

Merge branch 'dev' into feature/persistence-testkit

0218d9a

Merge branch 'dev' into feature/persistence-testkit

b823a45

Merge branch 'dev' into feature/persistence-testkit

bc6962c

Aaronontheweb merged commit 37be6a5 into akkadotnet:dev Aug 31, 2019

valdisz deleted the feature/persistence-testkit branch September 22, 2019 20:01

Aaronontheweb pushed a commit to Aaronontheweb/akka.net that referenced this pull request Sep 23, 2019

TestJournal and TestSnaphostStore (akkadotnet#3881)

5cbdd58

* TestJournal with Write interception and various failure strategies * Akka.Persistence.TestKit implementation

Aaronontheweb mentioned this pull request Oct 21, 2019

Akka.Persistence.TestKit implementation #3877

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

TestJournal and TestSnaphostStore #3881

TestJournal and TestSnaphostStore #3881

valdisz commented Aug 8, 2019 •

edited

Loading

Aaronontheweb commented Aug 8, 2019

Aaronontheweb left a comment

Aaronontheweb left a comment

valdisz commented Aug 11, 2019

valdisz commented Aug 12, 2019

Aaronontheweb commented Aug 12, 2019

Aaronontheweb left a comment

Aaronontheweb left a comment

Aaronontheweb commented Aug 13, 2019

valdisz commented Aug 13, 2019

Aaronontheweb left a comment

Aaronontheweb Aug 21, 2019

valdisz Aug 22, 2019

valdisz commented Aug 23, 2019

Danthar commented Aug 25, 2019

valdisz commented Aug 25, 2019 •

edited

Loading

Danthar commented Aug 25, 2019 via email

Danthar commented Aug 26, 2019

valdisz commented Aug 26, 2019 •

edited

Loading

Danthar commented Aug 26, 2019 via email

Aaronontheweb left a comment

Aaronontheweb commented Aug 26, 2019

Aaronontheweb commented Aug 26, 2019

Aaronontheweb commented Aug 27, 2019

TestJournal and TestSnaphostStore #3881

TestJournal and TestSnaphostStore #3881

Conversation

valdisz commented Aug 8, 2019 • edited Loading

Aaronontheweb commented Aug 8, 2019

Aaronontheweb left a comment

Choose a reason for hiding this comment

Aaronontheweb left a comment

Choose a reason for hiding this comment

valdisz commented Aug 11, 2019

valdisz commented Aug 12, 2019

Aaronontheweb commented Aug 12, 2019

Aaronontheweb left a comment

Choose a reason for hiding this comment

Aaronontheweb left a comment

Choose a reason for hiding this comment

Aaronontheweb commented Aug 13, 2019

valdisz commented Aug 13, 2019

Aaronontheweb left a comment

Choose a reason for hiding this comment

Aaronontheweb Aug 21, 2019

Choose a reason for hiding this comment

valdisz Aug 22, 2019

Choose a reason for hiding this comment

valdisz commented Aug 23, 2019

Danthar commented Aug 25, 2019

valdisz commented Aug 25, 2019 • edited Loading

Danthar commented Aug 25, 2019 via email

Danthar commented Aug 26, 2019

valdisz commented Aug 26, 2019 • edited Loading

Danthar commented Aug 26, 2019 via email

Aaronontheweb left a comment

Choose a reason for hiding this comment

Aaronontheweb commented Aug 26, 2019

Aaronontheweb commented Aug 26, 2019

Aaronontheweb commented Aug 27, 2019

valdisz commented Aug 8, 2019 •

edited

Loading

valdisz commented Aug 25, 2019 •

edited

Loading

valdisz commented Aug 26, 2019 •

edited

Loading