Distributed system for processing data based on bull.js

[tomeof]

Hi, bull.js looks very interesting. I would like to ask if it is possible to implement a system based on bull.js.
Requirements:

• A stateless REST service in node.js (e.g. express.js)
• A distributed system to process the input data (based on bull.js)

One computer is going to be the producer and several computers are going to be the consumers.

Simple Example: A client makes a request and sends a list of integers (e.g. 2,1,4,8,9,7,5,2,3,4,1,2). The service should return the sum of the squares of its' input numbers (2^2 + 1^2 + ...). The distributed system should split the input into 3 groups of 4 integers (for example), create a job for each group (3 jobs) and process the jobs to different servers (part of the distributed system).

Questions:

• How this system knows when the processing of a request has finished? I mean when it knows that the 3 jobs have finished in order to sum their results and return the value to the client? Should for every client request a new queue created? Is there a way to "tag" jobs so we know they are coming from the same client request?
• Assuming we get concurrent requests from various clients: How this system knows which sub-sums (or which job results) should be aggregated to return the correct sum to each client?

[manast]

That's seems like a use-case suitable for flows: https://docs.bullmq.io/guide/flows

[tomeof]

Cool, thanks!