why is the needed two have kind of actor references RemoteActorRef and ActorRef,

[criloz]

They all wrap an ActorID, so they have knowledge when the actor is local or remote?, it feels that the distributed case is treated as second class citizen.

Wouldn't be better to just have a uniform API for all the actors, not matter if they are local or remote?

In case a local actor is needed because the message is not serializable, an as-local function could be implemented.

ActorRef::as_local()->Option<LocalActor>

Also, which is the best way to implement something like creating a mpsc channel that work in the distributed case I want to send to a remote actor a message and need the actor to be able to reply many messages into a stream.

[tqwewe]

In case a local actor is needed because the message is not serializable, an as-local function could be implemented.

I guess this is the biggest reason. Your suggestion for .as_local() -> Option<LocalActorRef<T>> could work too, but its just an alternative approach I guess. Currently the local ActorRef also has fields for its links, a way to kill the actor, and a way to wait for the actor to finish starting up.

In my personal code, I don't typically make my messages serializable if they don't need to be, so I'd have to be calling .to_local() quite often.

Your suggestion could work however, but just isn't the approach I went with when implementing distributed actors.

Is there an issue in particular you could describe with the current approach of having ActorRef and RemoteActorRef be separate?

Also, which is the best way to implement something like creating a mpsc channel that work in the distributed case I want to send to a remote actor a message and need the actor to be able to reply many messages into a stream.

Do you mean the reply type would be a stream of replies? I guess the simplest approach would be to instead send the replies as messages to an actor directly. But perhaps there could be a ReplyStream type added to kameo to help with this.

[criloz]

Yeah, send the replies directly to an actor is one alternative the problem is that the sender will need to know the type of the actor, the option that I found is to create a distributed actor version of UnboundedReceiver and UnboundedSender, that act as intermediaries, then creteate_distribuited channel create locally an UnboundedReceiverActor and forward all the messages to a mpsc::UnboundedReceiver, it returns (InputRef, mpsc::UnboundedReceiver), where the input ref have a reference to the UnboundedReceiverActor and can be sent to other nodes, in the other node InputRef.load()->mpsc::UnboundedSenderActor will create the UnboundedSenderActor than can be used to implement things like subscription to actor state changes or subscribe to changes in a specific range of a kv store.

[criloz]

Another way, but maybe hacky to handle this could be allowing reusing message IDs, so could it be possible to create generic Actors

#[Actor]
struct KvStoreAnySubscriber{}

#[message_id=1]
impl Message<OnUpdate> for KvStoreAnySubscriber{ ...}

#[Actor]
struct RealActor{}

#[message_id=1]
impl Message<OnUpdate> for RealActor{ ...}

#[Actor]
struct KvStore{
 notify: HashMap<PartitionKey, Vec<RemoteActorRef<AnyActor>>>
}

[criloz]

@tqwewe another option is to provide other families of references that only depend on the message type, but it looks that it will be difficult because of the mailbox field

pub struct ActorRef<A: Actor> {
    id: ActorID,
    mailbox: A::Mailbox,
    abort_handle: AbortHandle,
    pub(crate) links: Links,
    pub(crate) startup_semaphore: Arc<Semaphore>,
}

pub struct MesageRef<M> {
    id: ActorID,
    ...
}