Monorepo support

[wbcs]

Updated issue

Bun supports "workspaces" in package.json, "workspace:* (protocol) in dependencies, and many more features related to workspaces. See the docs page on workspaces.

You can also use bun run --filter="*/*lib" to run multiple package.json scripts from different packages within a workspace simultaneously and view tail the logs - see the docs.

We're keeping this issue open for now, but it's not entirely clear what's left to close it.

Original issue below

---

What is the problem this feature will solve?

bun not support monorepo(root package.json workspaces field, workspace:* schema, etc.)

What is the feature you are proposing to solve the problem?

The download speed of monorepo will be faster

What alternatives have you considered?

No response

[Jarred-Sumner]

yes, but probably not for a little while

[FinnRG]

Related: The workspace: field progress is tracked in #83.

[davidmeirlevy]

can the implementation of workspaces include:

1. the ability to have separated node_modulues on each workspace.
2. the equivalent of lerna exec to execute any command inside the workspaces, even if it's not inside the package.json.
   for example, I want to run bun dev on each workspace, but the dev itself probably won't be a script inside the package.json of each workspace. the same for tests...

[factoidforrest]

I just want to say that I haven't missed lerna exec at all after ditching lerna for pure yarn workspaces. Things like that are certainly nice-to-haves but I don't think going any further than the support provided by yarn or NPM is necessary as a first-pass.

[davidmeirlevy]

@factoidforrest yarn has yarn exec that can work on workspaces, so it's equivalent.

[unional]

Maybe consider monorepo support similar to turbo and rush. They are both very efficient.

[wbcs]

nsider monorepo support similar to turbo and rush. They are both very efficie

I prefer pnpm

[unional]

I prefer pnpm

npm/yarn/pnpm are orthogonal to lerna/turbo/rush. One are package managers, the others are monorepo tools.

[davidmeirlevy]

I prefer pnpm

npm/yarn/pnpm are orthogonal to lerna/turbo/rush. One are package managers, the others are monorepo tools.

He probably meant to pnpm workspaces.
I also prefer this way, because of the full scope isolation for each package.

[ozyman42]

I prefer pnpm

npm/yarn/pnpm are orthogonal to lerna/turbo/rush. One are package managers, the others are monorepo tools.

He probably meant to pnpm workspaces. I also prefer this way, because of the full scope isolation for each package.

Yarn 2+ Plug n Play also has full scope isolation for each package. A main reason the pnpm resolution strategy might be preferable to Plug n Play is because notable projects such as React Native are currently incompatible with Plug n Play. On the flip side, Plug n Play might be considered preferable because it may solve the doppleganger problem differently than pnpm (someone ought to investigate this comparison in detail). Personally I find the "non-single singletons" doppleganger consequence the most important one because it's the root cause behind monorepo bugs involving popular libraries such as graphql-js.

Overall though I think Bun should make an authoritative choice for workspaces/monorepo support rather than letting this be highly configurable. Too much configurability leads to too much project complexity both in implementation and from a client's point of view w/ too many options to choose from & too much boilerplate. Bun needs to have a clear story for workspace/monorepos before it can be seriously considered for adoption by medium to large projects.

[unional]

Agree. IMO pnpm doppleganger solution is also the way to go.

The point I want to make by bring yo turbo and rush is the ability to efficiently running multiple scripts with dependencies.

For example:
https://turborepo.org/docs/core-concepts/pipelines
https://rushjs.io/pages/advanced/incremental_builds/
https://rushjs.io/pages/developer/selecting_subsets

[wbcs]

Agree. IMO pnpm doppleganger solution is also the way to go.

The point I want to make by bring yo turbo and rush is the ability to efficiently running multiple scripts with dependencies.

For example: https://turborepo.org/docs/core-concepts/pipelines https://rushjs.io/pages/advanced/incremental_builds/ https://rushjs.io/pages/developer/selecting_subsets

turborepo is greate.
but how rush run custom npm scripts in subsets? It run reserved keyword instruction only.

[unional]

but how rush run custom npm scripts in subsets? It run reserved keyword instruction only.

It does not. rush has it's own way to add commands to the system.
(to run custom npm scripts in one package, use rushx run ...)
I personally like turbo better too. The rush system sounds too heavy and rigid to me (I tried both).
Also mentioning rush here for completeness and also may able to draw an idea or two from it's selective script execution (those from/to/since things).

[melMass]

Hi,

IMHO the only responsibility from Bun regarding workspaces is dependency resolution, the rest should be handled by a monorepo tool like Turbo or NX.

[o-alexandrov]

Is npm run SCRIPT_NAME --workspaces part of this issue, or does it need a separate one?

---

Related to previous comments: #533 (comment), #533 (comment)

[evelant]

Per #83 it looks like some support for workspace: protocol was added but it doesn't appear to work with pnpm workspaces. Did I miss something or are there still additional changes needed to allow bun to work with a pnpm monorepo?

[Bessonov]

Probably, related: #2517

[jtlapp]

Speed and disk space are the reasons pnpm is best for monorepos. Whenever multiple workspaces within a monorepo depend on the same npm package, npm and yarn download and install a separate copy of the npm package for each workspace, whereas pnpm symbolically links them all to the same download of the package. Yarn's PnP does muddy the waters a bit, but its problems keep me from using it. See the benchmarks.

[scinos]

Whenever multiple workspaces within a monorepo depend on the same npm package, npm and yarn download and install a separate copy of the npm package for each workspace

For yarn, it's configurable: https://yarnpkg.com/configuration/yarnrc#nmMode

[jakeleventhal]

Will migrate to bun when this works

[Bessonov]

I am not sure now, it isn't supported and described here? https://bun.sh/docs/install/workspaces Do I miss something or can this issue be closed as completed?

EDIT: I see it miss support for script execution inside monorepo, ex. pnpm run --filter mypackage start. Probably, should be tracked as a separate issue.

[Hebilicious]

@Bessonov I'm not sure it should be considered completed so soon, there's several workspace related bugs and missing features (like filtering that you mentioned)

I went through the open issues, it looks like there's a bunch of duplicate, but we could use this issue as a tracking one.

• bun i in workspaces doesn't symlink local dependencies #4830
• bun install in pnpm/yarn workspaces #2517
• bun install doesn't work with workspaces #4628
• monorepo package dependencies are usable across packages, also allows use of phantom dependencies #4274
• Support bun run --workspace  #2232
• SvelteKit not working with workspaces #4174
• Bun install does not install dependencies between packages in the same monorepo #4650
• Support workspaces without workspace: protocol #4676
• bun install workspace package parse error #3582
• Workspace-References are not used in bun install #4366
• in workspaces you can not use a ! to exclude and errors out on install #4621
• bun add fails in a sub-package of a monorepo workspace #3502
• bun install not working for workspace projects from root #3806
• workspace packages list gets stale if new workspace packages are depended on #3686
• Implement workspace:^version package.json dependency version specifier #3677
• workspace member package can't depend on workspace root #2773
• Wrong package.json workspace: prefix behavior #2938
• postinstall scripts in monorepos aren't run if there's errors with the installation #4447

[Bessonov]

@Hebilicious Thank you very much for the excellent overview! I am still of the opinion that documented basic use cases should work. So, the binary answer to this issue is "yes, a monorepo is supported". I am not opposed to leaving this issue open, but, in my opinion, it distracts from addressing actual problems. Alternatively, @wbcs could add your list to the top of this issue, making it a meta-issue for outstanding work.

[james-elicx]

I feel it's relevant to share this tweet that I saw from Jarred a couple of months ago. He mentioned wanting to have another go at implementing workspaces as he didn't think the support was that good, although I'm not sure if they ever got around to redoing it.

https://twitter.com/jarredsumner/status/1682860099528978432

[jakeleventhal]

Additionally plugging this issue: #4844

[itpropro]

@Hebilicious Thank you very much for the excellent overview! I am still of the opinion that documented basic use cases should work. So, the binary answer to this issue is "yes, a monorepo is supported". I am not opposed to leaving this issue open, but, in my opinion, it distracts from addressing actual problems. Alternatively, @wbcs could add your list to the top of this issue, making it a meta-issue for outstanding work.

It's hard to argue that bun has monorepo support in terms of a drop in replacement for pnpm. Replacing pnpm works for the most part, but as soon as you come to monorepos (which is very common in OSS and other projects these days), you have to redesign your whole CI/CD pipelines or have challenges that can be very hard to solve.
You can't just run bun run --filter "@scope/package1" build instead of pnpm run --filter "@scope/package1" build etc.. There is still a long way to go and I am willing to go it with bun, but it should be communicated transparently that there is no real monorepo support as people expect when coming from pnpm or yarn.

[jakeboone02]

@itpropro I think (1) "doesn't fail when used in a monorepo", (2) "supports monorepos", and (3) "includes tools that make working with monorepos easy" are all valid but very different definitions of "monorepo support". Bun currently meets the criteria for (1), sort of meets the criteria for (2), but definitely does not meet the criteria for (3).

As evidenced by @Hebilicious's list (#533 (comment)) and other comments here, there's a lot of work to do before that last criteria is met. Documentation should probably reflect that.

[danieldunderfelt]

@itpropro I think (1) "doesn't fail when used in a monorepo", (2) "supports monorepos", and (3) "includes tools that make working with monorepos easy" are all valid but very different definitions of "monorepo support". Bun currently meets the criteria for (1), sort of meets the criteria for (2), but definitely does not meet the criteria for (3).

As evidenced by @Hebilicious's list (#533 (comment)) and other comments here, there's a lot of work to do before that last criteria is met. Documentation should probably reflect that.

Criteria 1 is not met yet, Bun 1.0 still fails for me when adding packages as described in #3502.

[hansoksendahl]

@Hebilicious thanks for the follow up.

A hybrid approach is interesting and could work. It just seems that it might be hard to evangelize across a development team. An errant bun add [package] on the part of a developer could lead to a lot of head scratching.

[ozyman42]

Currently I have to use pnpm alongside all my bun projects since bun doesn't actually support monorepos yet.
Would be great to at some point only need bun.

1. Hoisting must be disabled. Not allowed. Hoisting breaks monorepos
2. sym / hard linking should be used similar to pnpm. That's the industry standard now in mid 2024

would be nice to see someone given a mandate to go implement this.

[81reap]

I'm a bit new to the front end world but is this what y'all are talking about?

• https://bun.sh/guides/install/workspaces
• https://bun.sh/docs/install/workspaces

[birkskyum]

Found I can use bun workspaces, but when i build a package i have to execute the build with node, because bun is hoisting all dependencies to root level node_modules, and only node will be able to find those from a /packages/package1/build/app.js by properly checking parent directories in module resolution. Bun seem to only check inside the package which is not where bun install puts the node_module.

[ozyman42]

@81reap as I mentioned in my comment and as is mentioned in those workspaces docs, Bun does hoisting but that is an inadequate way to support monorepos because it permits/encourages the phantom dependencies bug class.

[birkskyum]

With pnpm I sometimes have to set i.e. shamefully-hoist=true in the .npmrc, but that taps into another highly upvoted issue

• Support reading from .npmrc #643

[metaspartan]

yes, but probably not for a little while

Any updates on this? Confirmed monorepos not working still as of bun v1.1.4 on any OS

[wesleycoder]

@metaspartan This whole issue is huge and there's definitely work being done.
I've just seen some fixes being merged related to --filter which is a step closer to having a monorepo worflow.
To help with anxiety I keep an eye on the issue list starting with these that @Hebilicious was kind to filter.

No need to put more pressure on this great group of people doing the hard work.
When it's done it will definitely be broadly announced and comemorated.
For now we can keep an eye on the issueas and changelogs and experiment on the canary builds daily, giving feedback on the proper issues.

[hansoksendahl]

It looks like the features to support monorepos are getting close to being ready. Any ETA or an RC build that includes these features?

[wesleycoder]

Everything merged to the main branch is be available on the canary builds.

[c0ncentus]

When i try switch npm to bun in windows I hoping to see if monorepo will be there .
With npm there are lerna, nx, mooncli and now crap stuff works (manual script on all 20 packages T^T).

Hope that bunjs make it simple and easy like bun -w build && bun -w publish && del **/**/dist

[wesleycoder]

Nothing stops anyone from using third party monorepo tools with bun.
I've been using turborepo with bun for a while, and while we have to make some concessions and adjust some things we can make monorepos work if we are willing to make those.

The contributors at oven/bun are doing a great work, let's not turn this issue into a long discussion with lots of people just asking for the same thing they are already working on.

Progress will be reported when it is relevant, as every open source project do, and bun hasn't let us down.
Let them work in peace and keep the anxiety in control.

[hansoksendahl]

@wesleycoder I think a lot of us are just excited. We would like to simplify our install, unit-testing, browser testing, build, and runner down to a single tool. So no anxiety needed.

I did some refactoring of my codebase and the amount of packages I was able to rip out of my projects when using bun was just... chef's kiss.

That is until I realized the monorepo support and specificity that I got with tools like pnpm was just not there due to hoisting (the node_modules strategy that bun currently uses).

Another part of what has some of us excited is that installing, building, testing, running with bun would cut down on compute time and result in real cost savings for production apps.

[birkskyum]

With latest bun I was able to finally migrate from pnpm to bun workspaces (have 4 packages that rely on one another), without any of the issues with hoisting i mentioned prior - thanks!

[kaminskypavel]

With latest bun I was able to finally migrate from pnpm to bun workspaces (have 4 packages that rely on one another), without any of the issues with hoisting i mentioned prior - thanks!

what happens if you update the packages? does the --watch flag picks it up?
#9547

[birkskyum]

It's been 2 months since i migrated from pnpm to bun workspaces, and my experience is unfortunately that pnpm still is far superior to bun workspaces in some important ways. The killer feature(s) that I find is sorely missing and hope will land soon is:

Bun needs a way to do recursive interactive bun update (missing) and have it write to a plaintext lockfile that diffs well (also missing).

[gitblit]

@birkskyum I agree 💯. This comment on #1223 issue suggests using Taze. I've had good luck with that as an intermediate solution. It will detect & use the appropriate package manager. For my bun monorepo it is successfully using bun install. I haven't gotten bun update to work so I disable updates. ymmv.

bunx --bun taze --interactive --recursive --install --update false --write

[birkskyum]

@gitblit , that worked surprisingly well, thanks! With a third-party solution there, I guess a plaintext lockfile is all i need.

Especially the text lockfile is important for CI tools like renovate/dependabot to avoid conflicts all over the place, and to be able to inspect diffs. I don't think that dependabot even works for regular repos with bun yet though, and it's not clear to me if support for regular repos will also bring support for monorepos - we'll see i guess:

• add support for bun dependabot/dependabot-core#6528

[hansoksendahl]

So long as bun continues to hoist packages up to node_modules/ in the root of the repository it cannot truly support monorepos. If package a has a dependency on c then package b should not be able to access c unless it also has c in its dependencies.

This is the ONLY issue preventing me from using bun in production monorepos.

Have there been any developments with bun's handling of monorepos that prevent hoisting?

[tunnckoCore]

@hansoksendahl If package a has a dependency on c then package b should not be able to access c unless it also has c in its dependencies.

Why?! Can you elaborate?

[hansoksendahl]

@tunnckoCore

TL;DR Hoisting breaks module boundaries.

Hoisting of dependencies enables any module managed in a monorepo to access any dependency installed by another module. Deliberately or accidentally leveraging these "ghost dependencies" can lead to unexpected failures when bundling, running code in CI/CD, or deployment. Breaking of module boundaries is such a common source of errors that most monorepo tools have aligned on explicit declarations of dependencies and deliberate separation of dependencies per module.

[hansoksendahl]

I know talk is cheap. Here is a minimum reproduction repository for the issues being discussed.

https://github.com/hansoksendahl/min-repro-bun-ws-hoist

[Scalahansolo]

I just wanted to hopefully add some weight to this conversation. Here at Motion I put in a ton of effort to try and migrate from yarn to bun for installs, but the hoisting issue(s) prevented us from migrating. React Native operates best with its deps living at the same location as the app in a monorepo, and we could find a clean migration path without the ability to limit hoisting.

I really want to continue to push to use bun across our stack, but this issue continues to not get support in favor of other newer features. I'm starting to lose Hope that bun is focused on the correct stuff, especially as it relates to supporting larger companies wanting to use bun.

[Jarred-Sumner]

@Scalahansolo we will fix it

[jzila]

I concur with the general hoisting conversation here, though there is also an existing issue where bun update inside a package does not properly respect workspaces. It attempts to create a node_modules inside the workspace while adding the "updated" module in the root package.json, leaving the package's module version unchanged. Using bun add for upgrades works as expected.