Merging multiple levels of source maps

[IceCreamYou]

Following on from #211, I have a build system that looks like this:

1. TypeScript -> one .js file per .ts[x] source file
2. Custom bundling logic -> several bundle .js files across all source .js files
3. Minifying bundles -> one .min.js file per .js bundle

So the source map chain goes:

[bundle].min.js -> [bundle].min.js.map -> [bundle].js -> [bundle].js.map -> [source].js -> [source].js.map -> [source].ts

I am experimenting with using esbuild at step # 3. (I would prefer to use it for step # 2 as well, but our code splitting is more complicated than what esbuild currently supports, so that will have to be a separate effort.)

I have found that esbuild will produce a [bundle].min.js.map that reads [bundle].js.map so that the minified output maps to the [source].js files. However it doesn't then go one level further down the chain to map all the way to [source].ts files. I am currently using merge-source-maps to work around this but of course I'd prefer to have fewer (and faster) dependencies.

Hence my request: could esbuild add support for reading more than one level of existing source maps?

[evanw]

I'm not sure I understand. Shouldn't whatever tool is being used in step 2 be responsible for integrating the source maps of the bundled code together? Then when esbuild minifies the bundle in step 3, the source map is only one level deep. What tool is being used for step 2?

[IceCreamYou]

Thanks for the prompt response.

Shouldn't whatever tool is being used in step 2 be responsible for integrating the source maps of the bundled code together?

Yes, that would absolutely make sense. I was hoping that adding this functionality to esbuild would be a small tweak for a tool that already has existing source map merging capabilities, since the effort for me to add support for source map merging to a different, mostly-internal tool that doesn't have it is considerable. (I mean, I sort of did, I'd just prefer to get rid of the dependencies I used to do it, ideally without rewriting them.)

What tool is being used for step 2?

It's built using SystemJS Builder, but I've built quite a bit of custom stuff on top of it (and SystemJS Builder is now archived).

This isn't really related to this issue, but for your interest, my code splitting requirements are:

• Developers should never have to manually configure which files end up in which bundles
• There should be a bundle per entrypoint
• No files should appear in more than one bundle
• There should be a bundle for each set of files used by X common entrypoints so that there are not orders of magnitude more bundles than entrypoints
• The bundle process should output some metadata that allows each entrypoint to preload bundles that contain shared dependencies
• No files should be loaded by an entrypoint (or its preloading) that are not dependencies of that entrypoint

Most bundlers, including esbuild as far as I can tell, offer one or more of these options for code splitting:

• One bundle per entrypoint with no shared bundles - causes problems with running shared modules multiple times if multiple entrypoints are loaded on the same page
• One bundle for everything shared - the common bundle ends up enormous
• One bundle for the shared dependencies of every combination of entrypoints - quickly becomes untenable as the number of entrypoints grows
• Escape hatch to allow manually specifying which files go in which bundles - a lot more work for developers on large codebases

The reason I started with SystemJS builder is that it had a unique "bundle arithmetic" feature where you could tell it something like "create a bundle with all the dependencies from entrypoint A and B, except the dependencies from entrypoint C." I wrote a layer on top of that that produces a bundle for each "page" in our app, and each combination of pages with shared dependencies up to a limited depth, and everything else goes in a common bundle.

Webpack's SplitChunksPlugin tries to do something sort-of smart like this based on limiting the number of chunks that'd have to be preloaded per entrypoint. At one point Rollup experimented with combining small chunks but it had various issues that resulted in its removal in the 2.0.0 release. I'm not aware of any other attempts at something like this.

[evanw]

Closing because I believe this is out of scope for esbuild. Checking for these additional source maps would slow esbuild down and make esbuild unnecessarily more complex, and this limitation is caused by other tools not handling source maps correctly instead of being caused by a bug in esbuild. It would of course be nice for someone to port merge-source-maps to a faster language, but that's not something I am interested in doing with this project.