Issue with finding akka-pubsub serializer

[joshgarnett]

Akka 1.3 + OSX

After testing a new build of Akka 1.3, my application ran into the following error:

[ERROR][8/25/2017 10:05:55 PM][Thread 0012][[akka://Cluster/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FCluster%40127.0.0.1%3A8091-1/endpointWriter#1752422679]] AssociationError [akka.tcp://Cluster@127.0.0.1:9189] -> akka.tcp://Cluster@127.0.0.1:8091: Error [Cannot find serializer with id [9]. The most probable reason is that the configuration entry 'akka.actor.serializers' is not in sync between the two systems.] [  at Akka.Serialization.Serialization.Deserialize (System.Byte[] bytes, System.Int32 serializerId, System.String manifest) [0x0002a] in <2643adcfd3b440da90f2f03df412dc4a>:0
  at Akka.Remote.Serialization.WrappedPayloadSupport.PayloadFrom (Akka.Remote.Serialization.Proto.Msg.Payload payload) [0x0003c] in <ab652cda59424e9e90ef31348362e877>:0
  at Akka.Remote.Serialization.MessageContainerSerializer.FromBinary (System.Byte[] bytes, System.Type type) [0x00018] in <ab652cda59424e9e90ef31348362e877>:0
  at Akka.Serialization.Serialization.Deserialize (System.Byte[] bytes, System.Int32 serializerId, System.String manifest) [0x0004e] in <2643adcfd3b440da90f2f03df412dc4a>:0
  at Akka.Remote.MessageSerializer.Deserialize (Akka.Actor.ActorSystem system, Akka.Remote.Serialization.Proto.Msg.Payload messageProtocol) [0x00032] in <ab652cda59424e9e90ef31348362e877>:0
  at Akka.Remote.DefaultMessageDispatcher.Dispatch (Akka.Actor.IInternalActorRef recipient, Akka.Actor.Address recipientAddress, Akka.Remote.Serialization.Proto.Msg.Payload message, Akka.Actor.IActorRef senderOption) [0x00000] in <ab652cda59424e9e90ef31348362e877>:0
  at Akka.Remote.EndpointReader.<Reading>b__11_1 (Akka.Remote.Transport.InboundPayload inbound) [0x000ec] in <ab652cda59424e9e90ef31348362e877>:0
  at (wrapper dynamic-method) System.Object:lambda_method (System.Runtime.CompilerServices.Closure,object,System.Action`1<Akka.Remote.Transport.Disassociated>,System.Action`1<Akka.Remote.Transport.InboundPayload>,System.Action`1<Akka.Remote.EndpointWriter/StopReading>)
  at Akka.Tools.MatchHandler.PartialHandlerArgumentsCapture`4[T,T1,T2,T3].Handle (T value) [0x00000] in <2643adcfd3b440da90f2f03df412dc4a>:0
  at Akka.Actor.ReceiveActor.ExecutePartialMessageHandler (System.Object message, Akka.Tools.MatchHandler.PartialAction`1[T] partialAction) [0x00000] in <2643adcfd3b440da90f2f03df412dc4a>:0
  at Akka.Actor.ReceiveActor.OnReceive (System.Object message) [0x00000] in <2643adcfd3b440da90f2f03df412dc4a>:0
  at Akka.Actor.UntypedActor.Receive (System.Object message) [0x00000] in <2643adcfd3b440da90f2f03df412dc4a>:0
  at Akka.Actor.ActorBase.AroundReceive (Akka.Actor.Receive receive, System.Object message) [0x00000] in <2643adcfd3b440da90f2f03df412dc4a>:0
  at Akka.Actor.ActorCell.ReceiveMessage (System.Object message) [0x00011] in <2643adcfd3b440da90f2f03df412dc4a>:0
  at Akka.Actor.ActorCell.Invoke (Akka.Actor.Envelope envelope) [0x00057] in <2643adcfd3b440da90f2f03df412dc4a>:0 ]

If I manually add the serializer to my app.config with the following the error goes away:

akka.actor {
  serializers {
    akka-pubsub = "Akka.Cluster.Tools.PublishSubscribe.Serialization.DistributedPubSubMessageSerializer, Akka.Cluster.Tools"
  }
  serialization-bindings {
    "Akka.Cluster.Tools.PublishSubscribe.IDistributedPubSubMessage, Akka.Cluster.Tools" = akka-pubsub
    "Akka.Cluster.Tools.PublishSubscribe.Internal.SendToOneSubscriber, Akka.Cluster.Tools" = akka-pubsub
  }
  serialization-identifiers {
    "Akka.Cluster.Tools.PublishSubscribe.Serialization.DistributedPubSubMessageSerializer, Akka.Cluster.Tools" = 9
  }
}

It's possible there is an ordering issue when loading the configs that cause the serializer to not be loaded properly.

[Horusiath]

Have you tried to supply DistributedPubSub.DefaultConfig() as fallback for your actor system configuration?

[joshgarnett]

Hey @Horusiath I have not, though my assumption would be this shouldn't be required as Akka should be properly combining the core akka configs in a sane way.

[vasily-kirichenko]

I have the same error and #3031 (comment) has not helped. Is there a WOKRING example cluster project to copy the config from?

I have this config on both nodes:

serializers {
              hyperion = "Akka.Serialization.HyperionSerializer, Akka.Serialization.Hyperion"
            }
            serialization-bindings {
              "System.Object" = hyperion
            }

[vasily-kirichenko]

It was Akkling System.create, which adds FsPickler serializer for expressions, after switching to ActorSystem.Create the error gone.

[leo12chandu]

I have the same problem that josh mentioned. When I add the pubsub serializers he pasted in app.config, it goes away. However, we plan to use hyperion serializer. Is there a way to get the ClusterClient working with hyperion serializer? Do I have to do a fallback with DistributedPubSub.DefaultConfig() as Horusiath suggested?

[vasily-kirichenko]

@leo12chandu I use the above config (using Hyperion for everything) and pub sub does work (without WithFallback you mentioned).

[leo12chandu]

@vasily-kirichenko - That did the trick. Thank You!