-
Notifications
You must be signed in to change notification settings - Fork 176
RabbitMQ
On the use of messaging services (i.e. RabbitMQ)
- Authors: Federico Stagni, Zoltan Mathe, Wojciech Ian Kremzien, ...
Messaging systems like RabbitMQ can be useful for performing several tasks in DIRAC. This RFC is created for discussing use cases, and for integration within the DIRAC framework.
RFC 21 on the Evolution of Transformation System already gives an example of a possible use case. The idea is to achieve data driven through usage of MQ like RabbitMQ. Details are in the RFC.
RabbitMQ is the market's leader when coming to interactions with monitoring tools.
DIRAC does not provide, at the moment, a pilots logging system. Authors think that such system could be provided using a message queuing system. RabbitMQ appears to be the most suitable candidate.
Executors work in a similar fashion to MQ systems. It should be studied if MQs can do the same job, but better.
From now on, we use RabbitMQ's jargon, and we consider specifically the case of integration of RabbitMQ within the DIRAC framework.
RabbitMQ has a server and a client part. The server part is host of the queues, the client part is for sending (RMQ Producers) and receiving (RMQ Consumers) messages to the queues inside the servers. Accessing queues can be assimilated to accessing databases (or, better, accessing services that expose database functionalities). RMQ producers are clients, while RMQ Consumers can be considered, in DIRAC jargon, like any other DIRAC component (e.g. like agents, or better like executors).
RMQ consumers can be run with runsv.