Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

v1.4.19 Akka.NET Release #4987

Merged
merged 74 commits into from
Apr 28, 2021
Merged

v1.4.19 Akka.NET Release #4987

merged 74 commits into from
Apr 28, 2021

Conversation

Aaronontheweb
Copy link
Member

1.4.19 April 28 2021

Maintenance Release for Akka.NET 1.4

Akka.NET v1.4.19 is a substantial release that includes a number of critical Akka.Cluster fixes, baseline Akka.NET performance improvements, and entirely new dispatcher that has shown to improve performance when used across all of the major actor groups that run both inside the /user hierarchy and the /system actor hierarchy as well.

Akka.Cluster Improvements
One of the most demanding issues of the v1.4.19 release was "Akka.Cluster: quarantining / reachability changes appear to be extremely sensitive" - and this is because debugging this issue touched so many different parts of Akka.Cluster.

We ultimately solved the problem - it is now quite feasible to rapidly scale an Akka.NET cluster from ~10 nodes to 50+ nodes without having a huge number of quarantines, accidentally downed nodes, and so on.

Here's the full set of fixes that went into resolving this issue:

Akka.Cluster is now much more robust, faster, and capable of scaling up and down much more efficiently than in previous releases.

ChannelExecutor and Akka Performance Improvements
In addition to improving Akka.Cluster, we also made substantial improvements to constructs found inside Akka.NET core itself:

In Akka.NET v1.4.19 we introduce an opt-in feature, the ChannelExecutor - a new dispatcher type that re-uses the same configuration as a ForkJoinDispatcher but runs entirely on top of the .NET ThreadPool and is able to take advantage of dynamic thread pool scaling to size / resize workloads on the fly.

In order to get the most use out of the ChannelExecutor, the default actor dispatcher, the internal dispatcher, and the Akka.Remote dispatchers all need to run on it - and you can see the latest configuration settings and guidance for that here in our documentation: https://getakka.net/articles/actors/dispatchers.html#channelexecutor

But a copy of today's configuration is included below - you can enable this feature inside your Akka.NET applications via the following HOCON:

akka.actor.default-dispatcher = {
    executor = channel-executor
    fork-join-executor { #channelexecutor will re-use these settings
      parallelism-min = 2
      parallelism-factor = 1
      parallelism-max = 64
    }
}

akka.actor.internal-dispatcher = {
    executor = channel-executor
    throughput = 5
    fork-join-executor {
      parallelism-min = 4
      parallelism-factor = 1.0
      parallelism-max = 64
    }
}

akka.remote.default-remote-dispatcher {
    type = Dispatcher
    executor = channel-executor
    fork-join-executor {
      parallelism-min = 2
      parallelism-factor = 0.5
      parallelism-max = 16
    }
}

akka.remote.backoff-remote-dispatcher {
  executor = channel-executor
  fork-join-executor {
    parallelism-min = 2
    parallelism-max = 2
  }
}

We are looking for feedback on how well the ChannelExecutor works in real world applications here: #4983

Hyperion v0.10 and Improvements
We also released Hyperion v0.10.0 and v0.10.1 as part of the Akka.NET v1.4.19 sprint, and this includes some useful changes for Akka.NET users who are trying to build cross-platform (.NET Framework + .NET Core / .NET 5) applications and need to handle all of the idiosyncrasies those platforms introduced by changing the default namespaces on primitive types such as string and int.

We have also introduced a new Setup type designed to make it easy to resolve some of these "cross platform" serialization concerns programmatically when configuring Hyperion for use inside Akka.NET:

#if NETFRAMEWORK
var hyperionSetup = HyperionSerializerSetup.Empty
    .WithPackageNameOverrides(new Func<string, string>[]
    {
        str => str.Contains("System.Private.CoreLib,%core%")
            ? str.Replace("System.Private.CoreLib,%core%", "mscorlib,%core%") : str
    }
#elif NETCOREAPP
var hyperionSetup = HyperionSerializerSetup.Empty
    .WithPackageNameOverrides(new Func<string, string>[]
    {
        str => str.Contains("mscorlib,%core%")
            ? str.Replace("mscorlib,%core%", "System.Private.CoreLib,%core%") : str
    }
#endif

var bootstrap = BootstrapSetup.Create().And(hyperionSetup);
var system = ActorSystem.Create("actorSystem", bootstrap);

See the full documentation for this feature here: https://getakka.net/articles/networking/serialization.html#cross-platform-serialization-compatibility-in-hyperion

To see the full set of fixes in Akka.NET v1.4.19, please see the milestone on Github.

COMMITS LOC+ LOC- AUTHOR
38 6092 4422 Aaron Stannard
13 2231 596 Gregorius Soedharmo
10 15 14 dependabot-preview[bot]
3 512 306 zbynek001
3 417 1 Ismael Hamed
1 5 5 Erik Følstad
1 5 19 Arjen Smits
1 27 1 Anton V. Ilyin
1 21 33 Igor
1 1 1 Cagatay YILDIZOGLU

Aaronontheweb and others added 30 commits March 23, 2021 19:27
* close #4860 - use local deploy for TcpManager child actors.

* Use local deploy for TcpIncomingConnection.

* Use local deploy for Udp actors.

Co-authored-by: Erik Folstad <erikmafo@gmail.com>
Co-authored-by: Aaron Stannard <aaron@petabridge.com>
…9.17

Bump Hyperion from 0.9.16 to 0.9.17
Bumps [Newtonsoft.Json](https://github.com/JamesNK/Newtonsoft.Json) from 12.0.3 to 13.0.1.
- [Release notes](https://github.com/JamesNK/Newtonsoft.Json/releases)
- [Commits](JamesNK/Newtonsoft.Json@12.0.3...13.0.1)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>

Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com>
Fix ClusterMetricsExtensionSpec racy spec
* Make sure that FileSubscriber shuts down cleanly when it dies

* Make sure that file all sink spec release the file handle if it fails
akka/akka#28341) (#4889)

* for example the Cluster InitJoin message is marked with DeadLetterSuppression
  but was anyway logged because sent with actorSelection
* for other WrappedMessage than ActorSelectionMessage we shouldn't unwrap and publish
  the inner in SuppressedDeadLetter because that might loose some information
* therefore those are silenced in the DeadLetterListener instead

Better deadLetter logging of wrapped messages (migrated from akka/akka#28253)

Logging of UnhandledMessage (migrated from akka/akka#28414)
* make use of the existing logging of dead letter
  also for UnhandledMessage

Add Dropped to Akka.Actor (migrated partially from akka/akka#27160)
Log Dropped from DeadLetterListener
* add CultureInfo for Turkish OS

added English CultureInfo to fix ToUpper function causes error on Turkish OS. "warning"->"WARNİNG"

* fix LogLevel TR char error

Co-authored-by: Aaron Stannard <aaron@petabridge.com>
…ations to complete (#4891)

* Harden FileSink unit tests by using AwaitAssert to wait for file operations to complete

* Use AwaitResult to improve readability

Co-authored-by: Aaron Stannard <aaron@petabridge.com>
* assertion failed: Nodes not part of cluster have marked the Gossip as seen
* trying to mark the Gossip as seen before it has joined, which may happen
  if CoordinatedShutdown is running before the node has joined

migrated from akka/akka#26835
* snapshot RecoveryTick ignored, part of akka/akka#20753

* lastSequenceNr should reflect the snapshot sequence and not start with 0 when journal is empty. Migrated from akka/akka#27496

* Enforce valid seqnr for deletes, migrated from akka/akka#25488

* api approval
* Add Setup class for NewtonSoftJsonSerializer

* Use Setup as a settings modifier instead of a settings factory

* Update spec

* Update API Approval list

* Unit test can inject null ActorSystem into the serializer causing the Setup system to throw a NRE

* Add documentation
Bumps [Google.Protobuf](https://github.com/protocolbuffers/protobuf) from 3.15.6 to 3.15.7.
- [Release notes](https://github.com/protocolbuffers/protobuf/releases)
- [Changelog](https://github.com/protocolbuffers/protobuf/blob/master/generate_changelog.py)
- [Commits](protocolbuffers/protobuf@v3.15.6...v3.15.7)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>

Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com>
…operties (#4902)

* close #4901 - replace reflection magic in MNTR with reading of MultiNodeConfig properties

* fixed outdated DiscoverySpec
This format error would cause the StandardOutLogger to throw a `FormatException` internally
made `HeartbeatHistory` into a `readonly struct` and cleaned up some other old LINQ calls inside the data structure
* Add -Dmultinode.include and -Dmultinode.exclude filter feature

* Add documentation

* Fix typos and makes sentences more readable

* Make the sample command line wrap instead of running off the screen

* Change include and exclude filtering by method name instead (requested)
Bumps [Google.Protobuf](https://github.com/protocolbuffers/protobuf) from 3.15.7 to 3.15.8.
- [Release notes](https://github.com/protocolbuffers/protobuf/releases)
- [Changelog](https://github.com/protocolbuffers/protobuf/blob/master/generate_changelog.py)
- [Commits](protocolbuffers/protobuf@v3.15.7...v3.15.8)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>

Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com>
Aaronontheweb and others added 29 commits April 19, 2021 21:19
…des (#4946)

* Configure duration for applying `MemberStatus.WeaklyUp`  to joining nodes

port of akka/akka#29665

* fixed validation check for TimeSpan duration passed in via HOCON

* harden ClusterLogSpecs
* added VectorClock benchmark

* fixed broken benchmark comparisons
* Performance optimize `VectorClock`

* don't cache MD5, but dispose of it

* guarantee disposal of iterators during VectorClock.Compare

* switch to local function for `VectorClock.CompareNext`

* fixed a comparison bug in how versions where compared

* minor cleanup

* replace `KeyValuePair<TKey,TValue>` with `ValueTuple<TKey,TValue>`

Reduced allocations by 90%, decreased execution time from 100ms to ~40ms
* harden RestartFirstSeedNodeSpec

* validate that we have complete seed node list prior to test
* added benchmark for HeartbeatNodeRing performance

* switched to local function

No perf change

* approve Akka.Benchmarks friend assembly for Akka.Cluster

* remove HeartbeatNodeRing.NodeRing() allocation and make field immutable

* made it so Akka.Util.Internal.ArrayExtensions.From no longer allocates (much)

* added some descriptive comments on HeartbeatNodeRing.Receivers

* Replaced `Lazy<T>` with `Option<T>` and a similar lazy initialization check

 Improved throughput by ~10% on larger collections and further reduced memory allocation.

* changed return types to `IImmutableSet`

Did this in order to reduce allocations from constantly converting back and forth from `ImmutableSortedSet<T>` and `ImmutableHashSet<T>` - that way we can just use whatever the underlying collection type is.

* converted `HeartbeatNodeRing` into a `struct`

improved performance some, but I don't want to lump it in with other changes just in case
* Add the groundwork for generalized crossplatform support.

* Update Hyperion to 0.10.0

* Convert adapter class to lambda object

* Add HyperionSerializerSetup setup class

* Add unit test spec

* Improve specs, add comments

* Add documentation

* Add copyright header.

* Change readonly fields to readonly properties.
* removed confusing and conflicting examples in the `ReceiveActor` documentation
* Embedded reference to "how actors restart" YouTube video in supervision docs
…4960)

Did this to make it easier to search for output logs produced during each phase of the `StressSpec`
* Bump Hyperion from 0.10.0 to 0.10.1

Bumps [Hyperion](https://github.com/akkadotnet/Hyperion) from 0.10.0 to 0.10.1.
- [Release notes](https://github.com/akkadotnet/Hyperion/releases)
- [Changelog](https://github.com/akkadotnet/Hyperion/blob/dev/RELEASE_NOTES.md)
- [Commits](akkadotnet/Hyperion@0.10.0...0.10.1)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>

* Fix dependabot Hyperion issue (#4961)

* Update Akka.Remote.Tests.csproj to use common.props

* Update HyperionSerializer to reflect recent hyperion changes

Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com>
Co-authored-by: Gregorius Soedharmo <arkatufus@yahoo.com>
* added memory metrics to `ActorSelection` benchmarks

* added ActorSelection benchmark

* ramped up the iteration counts

* validate that double wildcard can't be used outside of leaf node

* improve allocations on create

* minor cleanup

* create emptyRef only when needed via local function

* made `Iterator` into `struct`

* approved public API changes
* reduced iteration count to speed up benchmarks

* optimize some System.Collections.Immutable invocations to allocate less

* cleanup dictionary construction

* fixed multiple enumeration bug in `Reachability`
* cleaned up SpawnActorBenchmarks

* cleaned up SpawnActor benchmarks
This error has no impact on extremely busy actors, but for actors who have to process small bursts of messages this can make the difference between getting everything done in one dispatch vs. doing it in two.
port of akka/akka#20093

Might be responsible for some quarantines in Akka.Cluster / Akka.Remote when nodes are restarting on identical addresses.
* Added test cases where close would be called twice

* Bugfix UnfoldResource closed resource twice on failure
Co-authored-by: Aaron Stannard <aaron@petabridge.com>
* refactor Gossip class into `MembershipState`

port of akka/akka#23291

* completed `MembershipState` port

* fixed some downed observers calls

* forgot to copy gossip upon `Welcome` from Leader

* forgot to copy `MembershipState` while calling `UpdateLatestGossip`

* refactored all DOWN-ing logic to live inside `Gossip` class

* added some additional methods onto `MembershipState`

* fixed ValidNodeForGossip bug

* fixed equality check for Reachability

 should be quality by reference, not by value

Co-authored-by: Gregorius Soedharmo <arkatufus@yahoo.com>
* Fix serialization verification problem with Akka.IO messages

* Wrap naked SocketAsyncEventArgs in a struct that inherits INoSerializationVerificationNeeded

* Make the wrapper struct readonly

* Expand exception message with their actor types

* Update API Approver list
* Update DData reference HOCON config to follow JVM

* Clean up ReplicatorSettings, add sensible default values.

* Add DurableData spec that uses ORDictionary with POCO values

* Add a special case for Replicator to suppress messages during load

* Slight change in public API, behaviour is identical.

* Change Replicator class to UntypedActor

* Code cleanup - Change fields to properties

* Update API Approver list
fixed `A_dispatcher_must_handle_queuing_from_multiple_threads` - we were using the wrong message type the entire time, and the previous instance caused `Thread.Sleep` to be called repeatedly.
Cluster singletons won't create duplicates in a cluster split scenario, and it's much safer to run them _with_ a split brain resolver on than without. This documentation was just out of date.
* added `ChannelExecutor` dispatcher - uses `FixedConcurrencyTaskScheduler` internally - have `ActorSystem.Create` call `ThreadPool.SetMinThreads(0,0)` to improve performance across the board.

* fixed documentation errors
Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com>
* added v1.4.19 release notes
@Aaronontheweb Aaronontheweb merged commit 7a34a14 into master Apr 28, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

9 participants