fix concurrency issues in ActivationCollector

[DeagleGross]

I've managed to write a test, which consistenly reproduces the following error when trying to DeactivateInDueTimeOrder:

System.ArgumentException: The index is equal to or greater than the length of the array, or the number of elements in the dictionary is greater than the available space from index to the end of the destination array.
at System.Collections.Concurrent.ConcurrentDictionary2.System.Collections.Generic.ICollection<System.Collections.Generic.KeyValuePair<TKey,TValue>>.CopyTo(KeyValuePair2[] array, Int32 index)
at System.Linq.Enumerable.ICollectionToArray[TSource](ICollection1 collection) at System.Linq.Enumerable.ToArray[TSource](IEnumerable1 source)
at System.Linq.Enumerable.OrderedIterator`2.MoveNext()
at Orleans.Runtime.ActivationCollector.DeactivateInDueTimeOrder(Int32 count, CancellationToken cancellationToken) in D:\vrecluse\orleans\src\Orleans.Runtime\Catalog\ActivationCollector.cs:line 375
at Orleans.Runtime.ActivationCollector.RunMemoryBasedDeactivationLoop() in D:\vrecluse\orleans\src\Orleans.Runtime\Catalog\ActivationCollector.cs:line 604

Then made a fix to snapshot bucket collection before performing .OrderBy() to not fall into this trap. Fix done by Copilot in #9704 is roughly what we need, but test was not reproducing correctly, so I had to manually validate and adjust it.

Fixes #9686

Microsoft Reviewers: Open in CodeFlow

[Copilot]

Pull Request Overview

This PR fixes a concurrency issue in the ActivationCollector.DeactivateInDueTimeOrder method by creating a snapshot of the buckets dictionary before iteration, preventing collection modification exceptions during enumeration.

• Refactored test code to use a shared PrepareActivation helper method
• Added a comprehensive concurrency test (DeactivateInDueTimeOrder_ConcurrentModification_ShouldNotThrow)
• Fixed the race condition by taking a snapshot of buckets before ordering and iteration

Reviewed Changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.

File	Description
src/Orleans.Runtime/Catalog/ActivationCollector.cs	Fixed concurrency issue by snapshotting the buckets dictionary before ordering
test/NonSilo.Tests/Runtime/ActivationCollectorTests.cs	Refactored existing test to use helper methods and added comprehensive concurrency test