Fixed folder output, barrier spec failure handling issues in MNTR #2297
Conversation
made barrier failures more explicit hardened singleton manager startup spec
Aaronontheweb
force-pushed
the
mntr-spec-debugging
branch
from
bf6115a to
c7e0a9f
Compare
Aaronontheweb
changed the title
| @@ -31,6 +31,7 @@ public ClusterSingletonManagerStartupConfig() | |||
| CommonConfig = ConfigurationFactory.ParseString(@" | |||
| akka.loglevel = INFO | |||
| akka.actor.provider = ""Akka.Cluster.ClusterActorRefProvider, Akka.Cluster"" | |||
| akka.remote.retry-gate-closed-for = 1s #fast restart | |||
There was a problem hiding this comment.
Hardening the ClusterSingletonManagerStartupSpec - in the event two nodes try to connect to eachother at the same time (happens sometimes during Akka.Cluster formation) this lowers the retry gate from 5s to 1s so the spec has an opportunity to retry the association before hitting the deadline.
| @@ -156,7 +156,7 @@ static void Main(string[] args) | |||
| } -Dmultinode.listen-address={listenAddress} -Dmultinode.listen-port={listenPort}"; | |||
| var nodeIndex = nodeTest.Node; | |||
| //TODO: might need to do some validation here to avoid the 260 character max path error on Windows | |||
| var folder = Directory.CreateDirectory(Path.Combine(OutputDirectory, nodeTest.MethodName)); | |||
| var folder = Directory.CreateDirectory(Path.Combine(OutputDirectory, nodeTest.TestName)); | |||
There was a problem hiding this comment.
Uses the correct test name for the folders used for each of the individual node logs -this prevents two implementations of the same spec from overriding eachother's output.
| @@ -244,7 +244,7 @@ public override int GetHashCode() | |||
| public sealed class WrongBarrierException : Exception | |||
| { | |||
| public WrongBarrierException(string barrier, IActorRef client, Data barrierData) | |||
| : base($"tried to enter '{barrier}' while we were waiting for '{barrierData.Barrier}'") | |||
| : base($"[{client}] tried to enter '{barrier}' while we were waiting for '{barrierData.Barrier}'") | |||
There was a problem hiding this comment.
Clearer barrier exception error message.
| @@ -136,7 +136,7 @@ public void Enter(TimeSpan timeout, ImmutableList<string> names) | |||
| var askTimeout = barrierTimeout + Settings.QueryTimeout; | |||
| // Need to force barrier to wait here, so we can pass along a "fail barrier" message in the event | |||
| // of a failed operation | |||
| _client.Ask(new ToServer<EnterBarrier>(new EnterBarrier(name, barrierTimeout)), askTimeout).Wait(); | |||
| var result = _client.Ask(new ToServer<EnterBarrier>(new EnterBarrier(name, barrierTimeout)), askTimeout).Result; | |||
There was a problem hiding this comment.
We've had false positives in the MNTR previously where specs were still passing even though barriers were failing. This change ensures that an exception is thrown instead, failing the spec for this node if a barrier operation fails.
Working on resolviong #2296