Regarding its suitability for a kind of multi-threaded actor structure

[joseluis]

I found your project and I'm liking very very much what I see. But I'm still trying to wrap my head around it and I'd like to confirm the right approach to do what I want to build.

In essence, I need to have a group for actors with a "supervisor" that coordinates their work, like a tree of height 1. But the thing is one (or more) of those actors could be internally another group of actors with its own supervisor. And that group could be in the same thread, or have a dedicated separate thread. This structure can be repeated several times.

I'm still not sure what the best aproach would be using stakker, or even if stakker is the right tool for this, even after reading the why single threaded faq. I think that I'm forced enable the multi-thread feature, but also since I read there's no inter-thread message-passing layer yet, I'd like to ask for your opinion on how you see my case fits your library before I find myself bumping walls...

thank you very much in advance for any insights

[uazu]

The whole heirarchy of actors is easy to do within a single thread. For running another heirarchy in another thread, the supervisor actor in the first thread can start that other thread and start a stakker runtime and whatever actors are required in that thread. Maybe the PipedThread implementation could give you some ideas about making sure that things clean up correctly. Maybe PipedThread itself would be sufficient. So that part is doable.

The big question is what kind of communication will you require between the actors in the first and second threads? Stakker doesn't (yet) provide a way to proxy actor calls remotely (i.e. between threads or between machines). So you'll need to use mutex-protected shared memory, or queues, or channels, or some other means to pass data one way and the other. If the communication is relatively simple this should be straightforward. But if your actors will need to make a lot of calls backwards and forwards between the threads, and between different actors, it could be a bit more involved to set up as things stand right now. (Note that you can use any kind of inter-thread crate that suits your use case, and use Stakker's Waker to notify an actor in the target thread that it needs to look at the shared resource (i.e. queue, memory, channel, whatever).)

If you describe what kind of calling patterns you'd expect to see between the threads, maybe it would help me understand the requirements. This would also help for when I come to design and implement remote calls.

[joseluis]

Thank you for the quick response. So far it looks very promising... I think PipedThread could be enough...

At the most basic, the supervisor of the parent group needs to receive a message from the supervisor of the child group of actors in the second thread, containing the result of their work. And it may be also necessary to send several messages back and forth between the supervisors of the parent group and the child group, in both directions.

Also, each group of actors has its own private data, which must be able to be shared with the child actors (be them actors, groups in the same thread, or in another thread). At the very least as read-only data, since modifications could be done via messages... Maybe that will keep things simpler.

I like that I can use a Waker too... Hmmm... I see there are many possibilities. I just don't want to make things more complicated than they need to be, by adding too many things from the beginning, and that minimum is what I'm trying to figure out...

Just to confirm. What are the features that I'd need to enable in order for this to work?

[uazu]

You'll just need to enable "multi-thread" to run one Stakker per thread. Later on if you're interested in maximum performance it might be worth benchmarking with and without the "inline-deferrer" feature, which changes how the defer queue is reached (thread-local or inline reference). I'm not sure which will work out faster. It depends on how thread-locals are implemented on the platform.

For passing back results from the child thread, PipedThread makes that easy.

Is your second thread (or child thread) doing I/O, e.g. using MIO or whatever? Or is it purely dealing with work fed to it from the parent thread? If it's doing I/O then it will have its own Waker, and you can use a channel to send messages to it. If it's not doing I/O then you can use the PipedThread built-in channel to send it messages and do processing, although that would be a more synchronous flavour of operation (i.e. receive message, do processing, send back result).

I guess you might be wondering how to get data out of the actor system in the child thread, in the case of passing data back through the PipedThread's channel. You have control of the main loop, so you can arrange whatever signal you like, e.g. use cx.shutdown() to stop it, then do Actor::query to get data out of the actor holding the result. Or you could use an Rc<Cell<bool>> or Rc<RefCell<VecDeque<T>>> or whatever to pass flags or data from the actor up to the main loop code.

For shared read-only data, Arc will take care of that. If you want shared modifiable data then you're talking about mutexes and so on. Whether to use mutexes or messages for cross-thread modifications is a design decision depending on the patterns of modification you expect. If there is too much contention and synchronization then you lose the benefits of multiple threads, and so on ... I'm sure you know the trade-offs.

[joseluis]

That's very useful information, thank you.

I'm now designing a proof-of-concept implementation while learning the library, and pondering the structural choices...

I've still not decided whether it's better to differentiate at least between 2 kinds of workers: one that contains a stakker runtime and another that doesn't, and that way I don't have to use Option, I can avoid unnecessary matches and returning Result for many functions... or whether to use the same type for everything, with optional fields... or whether to use separate generic types with a common trait bound... This is one possibility I'm considering, but will probably change:

pub struct ThreadWorker {
    runtime: Stakker,
    actor: ActorOwn<ThreadWorkerData>, // this would be the 'supervisor'
    // ...
}

impl ThreadWorker {
    // here I can have fn wrappers over the runtime, for better calling ergonomics.
    // E.g. `dpt.run()` instead `dpt.runtime().run(Instant::now(), false)`...
}

pub ThreadWorkerData {
    // This could be stored in many different ways. Using separate vecs for simplicity
    workers: Vec<Worker>,
    thread_workers: Vec<ThreadWorker>,
    // ...
}

pub struct Worker {
    actor: ActorOwn<WorkerData>,
    // ...
}

pub WorkerData {
    workers: Vec<Worker>,
    thread_workers: Vec<ThreadWorker>,
    /// ...
}

Some of the child threads will do IO, and some would do cpu intensive operations over data already in memory... Although from the POV of the parent worker, operations should be nonblocking. The idea is that the parent worker supervises the work of its children, and periodically polls its workers. I still don't have very clear in my mind how different it will be in practice the polling of the child workers in the same thread, vs the workers in their own thread...

I'll have to play around with PipedThread, save the thread, do some messaging... and see how the design evolves. I see there are some PipeThread tests I can extract usage information from...

BTW I've noticed there are no standalone examples in the repository, and the only few existing ones are lost inside the extensive documentation, or as unit tests. Don't you like to have a examples folder? or would you welcome a PR adding some very simple ones? I may be able to do some while I'm learning...

[uazu]

Personally I think I'd use a different type for each distinct role. Then if I found myself repeating myself or wanting to copy/paste code, I'd look for how to pull things together (e.g. share a struct for common functionality, merge implementations into a single type, or whatever seemed best). But to start off, think about dividing responsibility into actors and coding up a type for each of them.

I notice that in your example above, all the workers have the same type. It may be that you know the types of your child workers in practice (i.e. hardcoded types) in which case, fine. (That's how most of my code against Stakker is.) But if you need the workers to implement a trait, then see the notes on that in the guide.

Regarding polling workers, that shouldn't be necessary. The workers should report back when they're done, probably via a Ret or Fwd. You'll also get a notification if a worker fails. However if you're polling for progress information, then yes that could make sense.

I don't have anything against examples/. Yes, the existing small examples could go in there. I find it hard to create convincingly-motivated small examples, though. The benefits only become obvious with bigger projects. I thought probably it would be better to produce some full applications that would serve as larger examples. However, if you'd like to submit some examples then that sounds good to me. Thanks!