Feature: multi-homed transport, virtual addresses and broadcast

[Zetanova]

In the process of implantation a mesh networking over UDP
see: https://github.com/Zetanova/PNet.Mesh

I come across a strong demand to have the possibility to address nodes by multiple addresses
on the same Akka.Remote.Transport Scheme

Even a workaround to register the same Akka.Remote.Transport twice with different schemes will not work,
because of the current behavior to resolve the response address by the incoming senders address.

Following components would profit from a multi-homed transport feature:
*) Broadcast messages over UDP.
UDP has native multicast/broadcast support.

*) The ClusterClient
A client could connect to a round-robing address like over a NLB or DNS

*) Virtual Addresses
Well-Known Addresses could be used, that are not bound to the Network layer.
Akka.Remote would need to resolve them like with a DNS hostname.

*) A floating Address could be implemented.
Clusters and Roles could have a dedicated Address as a singleton.
Like in a Failover cluster or keepalived, an addresses that lives only on the leader

*) Routers and DistributedPubSub
Routers could create virtual addresses and senders could use multicast
on transport level instead of of unicast
ScatterGatherFirstCompleted could only have a single routee as the broadcast address.

*) Events and Streams
Communication over multicast/broadcast

*) Discovery
Akka.Cluster and Akka.Discovery could use it for messaging.

*) Distributed Data
Broadcast messaging for Akka.DistributedData replication.

[Zetanova]

*) Mixing of hostname and FQDN would be possible.
Related issue: #5083

[Zetanova]

An idea to get it implemented, is to add an abstract protocol that acks as a routing plane
and relays the messages to the proper protocol and destination

Description

A node has multiple protocols active with addresses:
akka.tcp://akka@node-0.akka.ns1.cluster.local:2552/
akka.udp://akka@node-0.akka.ns1.cluster.local:2552/
akka.something://akka@node-0.akka.ns1.cluster.local:2553/

The abstract protocol 'net' can be added this would then add following address:
akka.net://akka@virtualAddressId:0/
where the virtualAddressId is a GUID to the node (or incarnation)
but virtualAddressId can also be used to address a Role
where port 0 would address all role-nodes and port 2 would address the role-leader
and port 1 would address all role-nodes excluding the current node.
The some can be used for cluster-singletons and cluster-routers

Akka.Cluster and Akka.Remote would work and see always the akka.net:// addresses
and the new AkkaNetProtocolTransport would use the existing Transports like DotNettyTransport
to send and receive the normal akka remote messages and also to build an internal routing table
with peer discovery, peer-sharing and/or broadcast messages.

Problem

The only problem that I see for this implementation is in Akka.Serialization.
The current Transports would need to be modified/hacked so that they Akka.Serialization
produces full sender addresses of the different protocol akka.net:// and not for the current Transport Protocol.

Example: Sending a message to a remote node

IActorRef remote; //to remote address `akka.net://akka@c0b22851c4ad4667841bae8705dea511:0/user/myActor1`
IActorRef self = Self; //from local address '/user/myActor2'
IActorRef target; //local address '/user/myActor3'
var message = new MyMessage(target); //the message

remote.Tell(message, self);
/*
the real transport is going over DotNettyTransport to "real" address 'akka.tcp://akka@remote-node:2552/`

self would need to be translated to 'akka.net://akka@ed6e814a690a4b7ba076e15e49a5a4ba:0/user/myActor2' 
and NOT to 'akka.tcp://akka@local-node:2552/user/myActor2'

target need to be translated to 'akka.net://akka@ed6e814a690a4b7ba076e15e49a5a4ba:0/user/myActor3' 
and NOT to 'akka.tcp://akka@local-node:2552/user/myActor3'
*/

[Zetanova]

@Aaronontheweb How does have a good knowledge to resolve the problem above?

[Zetanova]

The Key to resolve it is with Akka.Serialization.Serialization.CurrentTransportInformation
If the akka-net-trasnport can set the own information bevor it passes the message to the DotNettyTransport,
in theory DotNettyTransport would serialize akka.net:// addresses then.

I will make a test in the near feature

[Zetanova]

@Aaronontheweb You mention in your last comm. standup the ms-quick implementation and some time ago that akka artery is coming ...
Can we plan for virtual-transport-router, it does not require much extending. Basicly the above mention of Akka.Serialization.Serialization.CurrentTransportInformation would be required to make it today and hopefully transparently.

The virtual-transport-reouter would be the base to enable multi-homed transport, super-clusters, nat-traversel and so on.

Example would be to add an akka.nats:// transport over https://nats.io message-queue and use it to communicate the connection for other transports like vanilla akka.tcp and activate/connect it on runtime