Connecting multiple buses to Fleet Manager in one process.

[levirak]

I'm in the process of investigating what it would take for us to use Fleet Manager with our current setup, but I'm running onto a pretty big problem: we can't find a good way to connect more than one bus per process to Fleet Manager.

What I gather is that Rebus.FleetManager makes use of a buffer directory to queue up messages and store some auxiliary data. The problem is that each instance of a bus that has Fleet Manager enabled apparently expects total control of that directory for its entire lifetime. The effect of this is that we cannot use more than one bus in a single process without giving them separate directories.

And while we can change the buffer directory, how we use our buses makes it hard to ensure that each bus gets a unique buffer directory and gets the same directory every time. While we could do this assignment manually, we're looking for a more robust solution.

Do you have any suggestions on how to handle connecting multiple buses in the same process to Fleet Manager?

[mookid8000]

Yes! (but you'll have to wait a while for it)

I'm currently in the process of moving the HTTP forwarder part of the FM driver out and into an instance-per--URL-and-API-key model. This would mean that each forwarding buffer would be shared among all in-process Rebus instances sharing the same FM URL and API key.

This is not a big change, but it requires some pretty strict control of keeping track of how many buses are using a buffer, so that it can be properly flushed when the last bus shuts down.

I can't say how long it will take for this to be finished, but if you're interested in trying it out, I'd be happy to keep you in the loop and provide you with a prerelease or two to try.

[mookid8000]

When re-visiting the local buffering mechanism, maybe this PR has been merged? If so, maybe it would make sense to use FASTER log instead of the self-made buffering thing....?

[mookid8000]

File-based buffering has now been removed, drastically simplifying running multiple Rebus instances in the same process! 🙂