Implement node:cluster module

[Tushar-Chavan14]

What version of Bun is running?

0.5.7

What platform is your computer?

Linux 6.2.0-76060200-generic x86_64 x86_64

What steps can reproduce the bug?

bun i rate-limiter-flexible

import { RateLimiterMemory } from "rate-limiter-flexible";

const rateLimiter = new RateLimiterMemory({
points: 5,
duration: 120,
});

What is the expected behavior?

No response

What do you see instead?

No response

Additional information

No response

[Jarred-Sumner]

We haven't implemented the Node.js cluster module yet.

[ThatOneCalculator]

Is there currently any other way to do multiprocessing/multithreading with bun? This seems like it should be a very high priority item to support.

[xb1itz]

+1 on this. The lack of node:cluster implementation is probably the main reason we are not switching to Bun right now.

[CharnSrinivas]

+1 on this. The lack of node:cluster implementation is probably the main reason we are not switching to Bun right now.
Same here i am shifting back to node since i get know that node:cluster is not available yet

[melroy89]

I got indeed:

error: Could not resolve Node.js builtin: "cluster". To use Node.js builtins, set target to 'node' or 'bun'

const cluster = require('cluster')

I was very surprised to see this error. EDIT: I know cluster is not yet implemented. But then don't call it bun 1.0 yet.

[joshholly]

I ran bun --bun run dev to run my nextJS app. And I got this exact error.

[ThatOneCalculator]

Well, node:cluster isn't implemented yet, so that's why you're seeing the error.

[ElYaiko]

+1

[masfahru]

For now (Bun v.1.0.3), I can imitate the behavior of the node:cluster in Bun to spawn as many server instances as the number of CPU cores, using this script.

spawn.ts

import os from "node:os";

const numCPUs = os.cpus().length;
for (let i = 0; i < numCPUs; i++) {
  Bun.spawn(["bun", "index.ts"], {
    stdio: ["inherit", "inherit", "inherit"],
    env: { ...process.env },
  });
}

index.ts

Bun.serve({
  port: 8080,
  reusePort: true, // allow Bun server instances to use same port 
  fetch(req: Request) {
    // ...

run bun spawn.ts

[calebeaires]

@masfahru,

I performed the test and observed that the process ID remains the same every time I access the endpoint and print it. It appears that the access point is not redistributing properly.

 if (process.pid) {
            console.log(process.pid);
 }

index.ts

const server = Bun.serve({
    port: 3000,
    // @ts-ignore
    reusePort: true, // allow Bun server instances to use same port
    fetch(request) {
        if (process.pid) {
            console.log(process.pid);
        }
        return new Response("Welcome to Bun!");
    },
});

console.log(`Listening on localhost:${server.port}`);

cluster.ts

import os from "node:os";

const numCPUs = os.cpus().length;
for (let i = 0; i < numCPUs; i++) {
    Bun.spawn(["bun", "index.ts"], {
        stdio: ["inherit", "inherit", "inherit"],
        env: { ...process.env },
    });
}

[masfahru]

@calebeaires

SO_REUSEPORT is a builtin way to do load balancing, but it only really works properly on Linux.

Sorry, I don't know how load balancing works in Bun. To fulfill my curiosity about Bun server performance, I have tested the script in the TechEmpower Framework Benchmark, and the result is amazing. Bun is able to serve up to 2.7M requests per second for plain text test.

https://www.techempower.com/benchmarks/#section=test&runid=235acf72-7de8-4f61-baca-36c2df35a5c0&test=plaintext&l=zik0zj-33z

[calebeaires]

@masfahru, you're right. I've tested in a linux machine and works like a charm.

[ThatOneCalculator]

@calebeaires Would you mind sharing the details on what hardware/setup was used to achieve those numbers?

[calebeaires]

On M1 Mac OS 8 cores, it does not work. On linux, Ubuntu 22, 2 cores I could get a good result. Sending concurrent requests to Bun server that console log the process.id numbers, we get different results.

[ednt]

+1 on this. The lack of node:cluster implementation is probably the main reason we are not switching to Bun right now.

THIS!

[Matthew-Tersigni]

+1 I Can't fully convert my node apps to bun without this feature.

[ThatOneCalculator]

Can we please keep this thread productive, such as discussing potential implementations?

[ndaidong]

I'm looking forward to seeing this feature implemented in Bun.

[ganeshkbhat]

+1 for this.

I am also recommend implementing cluster_thread module which is genuinely a good starting point for creating clusters of threads like clusters of processes in node.js. Currently a multi-threading option is a little buggy. There is no random round robin as well in Windows due to absence of file descriptor module [presumed - check] in windows environment.

This is the Node.js Suggestion recommendation. nodejs/node#48350

@masfahru,

I performed the test and observed that the process ID remains the same every time I access the endpoint and print it. It appears that the access point is not redistributing properly.

 if (process.pid) {
            console.log(process.pid);
 }

index.ts

const server = Bun.serve({
    port: 3000,
    // @ts-ignore
    reusePort: true, // allow Bun server instances to use same port
    fetch(request) {
        if (process.pid) {
            console.log(process.pid);
        }
        return new Response("Welcome to Bun!");
    },
});

console.log(`Listening on localhost:${server.port}`);

cluster.ts

import os from "node:os";

const numCPUs = os.cpus().length;
for (let i = 0; i < numCPUs; i++) {
    Bun.spawn(["bun", "index.ts"], {
        stdio: ["inherit", "inherit", "inherit"],
        env: { ...process.env },
    });
}

[rgillan]

So in my opinion having used node:cluster for many years (and the cluster module pre integration), the ability to reuse sockets in the Bun.serve implementation pretty much removes the need for node:cluster support in Bun for most of the use cases. The IPC channel support is identical syntax, and things like http/s and websocket server port sharing just work out of the box on Linux. Spawn separate workers and reuse the ports. Reach out if you need any guidance but we've pretty much deprecated the need for the cluster capability with what Bun already has implemented. Cheers Rob

What if I want to have multiple instances of smtp-server running on port 25? Only other option would be nginx, but that comes with its own set of issues.

Understood, and yes you'd need to handle that externally (nginx, possibly pm2 would be another option but not sure) Another approach we have done in the nodejs world is have a single process handle the raw port, then workers doing what's needed. I've modified my comment to say some rather than most

[Ivanmatthew]

+1 on this. The lack of node:cluster implementation is probably the main reason we are not switching to Bun right now.

+1

So in my opinion having used node:cluster for many years (and the cluster module pre integration), the ability to reuse sockets in the Bun.serve implementation pretty much removes the need for node:cluster support in Bun for some of the use cases. The IPC channel support is identical syntax, and things like http/s and websocket server port sharing just work out of the box on Linux. Spawn separate workers and reuse the ports. Reach out if you need any guidance but we've pretty much deprecated the need for the cluster capability with what Bun already has implemented. Cheers Rob

The use-case would still be to reinforce Bun's statement of being a "drop-in" replacement for Node, since most of the functionality that entails the cluster module is apparently already implemented, I hope some of the code can be reused for a node:cluster module (mock?).

[snorremd]

On M1 Mac OS 8 cores, it does not work. On linux, Ubuntu 22, 2 cores I could get a good result. Sending concurrent requests to Bun server that console log the process.id numbers, we get different results.

Just for any other MacOS user like me coming to this thread to find out why reusePort is not working. It seems this feature works different on older FreeBSD and Darwin than Linux according to this StackOverflow thread. On MacOS the SO_REUSEPORT implementation does not round robin the TCP requests. Instead it sends the requests to the last process to attach to the socket.

Not sure if Darwin will ever implement this or not. So for the time being MacOS users will need to put a proxy in front of their Bun servers to support multiple process request handling.

[Bytedefined]

Would be great to see support sometime soon!

[elliotthill]

For a lot of us running regular servers, like ec2 instances, we cannot run Bun in production due to this.

Yes its possible to rewrite your app to use bun.serve() or rearchitect it to run on single threaded containers, but thats not really a drop in replacement as is claimed.

Given its the 12th most thumbs up'd issue out of ~2k, can it be added to the roadmap?

[mrtcmn]

This is power of the nodejs. I'll waiting node:cluster implemetation for migrate my all servers to bun.

[lodembeep]

What if you need clustering for tasks other than serving http requests ?

Clustering is important for non web tasks, i usually spawn workers on server side for CPU intensive tasks, nothing related to serving at all.

[Jarred-Sumner]

We are planning to implement node:cluster towards the end of this month.

[lodembeep]

We are planning to implement node:cluster towards the end of this month.

Awesome !!! I will spread the word :)

[haimev]

cool! great news

[liudonghua123]

I tried the latest bun 1.1.9. And it seemed not support currently. Hope next release could support it.

 > bun server.js
Primary 11780 is running
14 |   warned.add(feature), console.warn(new NotImplementedError(feature, issue));
15 | }, $;
16 |
17 | class NotImplementedError extends Error {
18 |   code;
19 |   constructor(feature, issue) {
                               ^
NotImplementedError: node:cluster is not yet implemented in Bun. Track the status & thumbs up the issue: https://github.com/oven-sh/bun/issues/2428
 code: "ERR_NOT_IMPLEMENTED"

      at new NotImplementedError (internal:shared:19:27)
      at internal:shared:2:69
      at fork (node:cluster:25:13)
      at D:\code\node\playground\cluster\server.js:13:5

Bun v1.1.9 (Windows x64)

 >

[i8ramin]

Hello has anyone successfully been able to use pm2 with bun compiled node binary? i've tried many different commands and variations but cannot figure it out. any help would be greatly appreciated.

[techsin]

is it out?

[dienstbereit]

Hello has anyone successfully been able to use pm2 with bun compiled node binary? i've tried many different commands and variations but cannot figure it out. any help would be greatly appreciated.

I assume that as soon as node:cluster is implemented, bun will also run with pm2 in cluster mode out-of-the-box. At least I hope so ;-)

[ajimix]

We are planning to implement node:cluster towards the end of this month.

+1 if you are here in June or later, waiting for it to be implemented 😄

[Jarred-Sumner]

@nektro is working on this, please follow along in #11492

[lortmorris]

I need using clusters for s42-core.
When you think guys this features will be available?

[Jarred-Sumner]

Initial support for the cluster module will be part of the Bun v1.1.25 release, thanks to @nektro!

[dienstbereit]

Magnificent… thanks to @nektro 👏 Now I can finally go into production with my project 🤩

[censujiang]

Thank you bro!

[kravetsone]

It's happening! Thanks!

It would be nice if you provide somewhere in Documentation prons and cons of usage node:cluster or reusePort: true

[kravetsone]

It's happening! Thanks!

It would be nice if you provide somewhere in Documentation prons and cons of usage node:cluster or reusePort: true

I guess also provide benchmarks of normal mode, cluster, reusePort will be great)

[rgillan]

Not sure it's up to the Bun team to recommend one way or another (cluster vs reusePort) but happy to share our experience. We have been running two nodejs applications in production that rely upon the cluster functionality to spin up numbers of workers, each with high volume websocket servers (using the websocket module, not ws for performance). One of these applications also serves http/s content. These applications would happily service 16k concurrent connections and traffic per worker on nodejs.
One of the major attractions of Bun to us was the native integration of uWebsockets into the run time which itself is very high performance, so we no longer needed to use third party modules and the refactoring to Bun.serve was quite trivial and straightforward. There's no additional layer of clustering code needed, and the sharing of the port is done at very low level in the OS not higher up in the node stack in my understanding. We are achieving over 100k connections per worker with a very simple refactoring so Bun is a gamechanger for us.
Obviously the node:cluster module does more than just the http/websocket ports and is more generic, so the benefits will be down to how and what you need to run multiple workers for.

[Jarred-Sumner]

prons and cons of usage node:cluster or reusePort: true

Internally, cluster relies on reusePort: true, so it's mostly helpful for automatic restarts and integration with existing codebases relying on cluster