Deno bundling v3

[kitsonk]

A lot has changed since #2475, so creating a new list:

• Support dynamic imports in bundles ([bug] dynamic import not working in bundles #4062)
• Redirects on main module should be supported (Redirect on main module can't be bundled #3082)
• Support generating types for bundles (Generate types for bundles #3385)
• Handle export * as properly (Deno.bundle() does not handle export * as (export-star-as) correctly #4542) This works now, believe it was an issue with TypeScript which was resolved in 3.9.
• Handle export * from properly (Deno.bundle causes Assertion error in the TS compiler #7518) (pr open at deno bundle supports export * exports #7607)
• Handle JavaScript imports that have a type directive (deno bundle <JavaScript file with triple-slash directive> #4539)
• Don't squash certain diagnostics ([Discussion] Best way to provide JavaScript builds with Deno #4298 (comment)) not really valid now
• Support tsconfig.json with deno bundle (deno bundle does not support tsconfig.json #3793)
• Support import-maps with deno bundle (Support --importmap with deno bundle #4379)
• Support source maps (first needs Consume source maps from JavaScript files #4499)
• The bundle loader code is written in esnext, when the target for the bundle isn't esnext, the bundle loader should be transpiled by TypeScript as well before being injected into the bundle code. (Support other targets with Deno.bundle #6297)
• The main module at bundle time has an import.meta.main === true, but in reality, the bundle loader should only set that to true when it itself is true. (Bundled import.meta.main should reflect loaded state. #6333)
• Supporting inlining WASM (Requires Bring back .wasm imports #5609 first) not really addressable
• Automatically format bundle outputs (format "deno bundle" output with dprint #5003)
• Detect other bundles and just import instead of inlining (see discussion in Add option to code split dynamic imports in bundles #5562) not applicable anymore
• Support dependency analysis of multiple "root files" (see discussion in Cargo test failed - latest master code. #5552) not applicable anymore
• Support lock files (Lock file not supported by deno bundle #6491)
• Support detecting that a module is part of a bundle (Unexpected import.meta.url value when bundling #6344)

Edit All this being said, we are looking at moving this over to swc, so this is a list of all the challenges we have that we shouldn't regress from or try to solve with the new approach.

[mangs]

Nice list. Considering CSS imports are fairly common (e.g. https://webpack.js.org/loaders/style-loader/), and Webpack, Rollup, and Parcel (and maybe others) support them, does Deno have any plans to do so? I realize Deno's a JS runtime, but that would probably get a lot of people to consider leaving Webpack behind, myself included. Regardless, very much looking forward to v1.0 in May! (fingers crossed)

[frank-dspeed]

Rollup does not support CSS Imports rollup is a esmodule bundler everything else is using the plugin API to turn it into something different.

[mangs]

@frank-dspeed True. Just curious if Deno has plans to do the same. If not, no worries. Still a wonderful project.

[kitsonk]

deno bundle will always be focused on creating workloads for Deno. The side-effect that some of those workloads can be run in a browser is "nice". Deno.bundle() is what is intended for building something for another runtime. It takes the input of sources, which means that if any source material can be wrapped in JavaScript/TypeScript, Deno.bundle() would be able to consume it and inline it into a bundle. It is just the matter of the community to come up with a solution.

[mudgen]

What is the status of supporting source maps? I need that to port PostCSS to deno.

[kitsonk]

The status is that if it isn't checked above, it isn't available. It isn't checked above.

[frank-dspeed]

• The bundle loader code is written in esnext, when the target for the bundle isn't esnext, the bundle loader should be transpiled by TypeScript as well before being injected into the bundle code.

@kitsonk is there a reason why we should even support other bundles? i think we should go for esnext only?

I think we should not even consider supporting CJS and the node-resolve algo that would push pressure on lib and framework developers to produce esnext output.

We on the runtime side could this way clean up the mess in the package ecosystem. we could write tutorials for codemods and stuff like lebab which can convert cjs to esnext.

[kitsonk]

@frank-dspeed it isn't referring to module format, but syntax. This would be only for Deno.bundle() when the target is not esnext. For example lots of browsers don't natively support nullish collescing, though Deno does.

[frank-dspeed]

@kitsonk deno should not produce some other bundels then esnext then the user needs to throw that bundle into something that transpils down for example babel and browsers.

Deno should only care for it self that would be great.

[matthewharwood]

Thanks for all the hard work @kitsonk. I'm a complete noob with bundling so pardon my ignorance. That said, I use this microbundle node package pretty frequently https://github.com/developit/microbundle#all-cli-options

Two of the features that it offers are

1. --external none
   External will remove all imports and pull in libraries directly into the bundle

2. -f modern
   Will bundle web components properly

These are two features I"m looking for so I can ditch node all together. Are these features going to be possible with deno.bundle or are there alternative solutions that I just don't know yet?

[kitsonk]

1. --external none
   External will remove all imports and pull in libraries directly into the bundle

It effectively already does this. This is the whole purpose of the bundler. The only thing that would be included are dynamic imports that cannot be statically determined at bundle time.

2. -f modern
   Will bundle web components properly

Not sure what this does, or what "bundling web components properly" means in this context. deno bundle and Deno.bundle() emit a ES Module. deno bundle will always emit with the latest ECMAScript/JavaScript syntax that Deno supports, while Deno.bundle() can be changed to transpile syntax to earlier versions of ECMAScript/JavaScript.

[matthewharwood]

@kitsonk Does --external work w/o any flags? if so, does it with the new import-maps feature. I couldn't get it to work... I was getting output that didn't seemingly match up? If you say so tho, I'll give it another shot on my repo: https://github.com/matthewharwood/deno-preact

Docs on Modern:
https://github.com/developit/microbundle#new-modern-js

[ozyman42]

One thing I would add to the list is tree-shaking from the bundle (maybe not as part of V3, but it's a feature that could be prioritized in a later version). The advantage of using a TypeScript AST for tree-shaking is you have all the static analysis foundations necessary to eliminate dead functions, which is harder to do when only analyzing plain JS modules.

[wongjiahau]

@kitsonk deno should not produce some other bundels then esnext then the user needs to throw that bundle into something that transpils down for example babel and browsers.

Deno should only care for it self that would be great.

I'm doing this now exactly, and I have to tell you it's very painful to get this working.

[ahuigo]

Nice list. Considering CSS imports are fairly common (e.g. https://webpack.js.org/loaders/style-loader/), and Webpack, Rollup, and Parcel (and maybe others) support them, does Deno have any plans to do so? I realize Deno's a JS runtime, but that would probably get a lot of people to consider leaving Webpack behind, myself included. Regardless, very much looking forward to v1.0 in May! (fingers crossed)

https://github.com/pikapkg/snowpack is more modernize than webpack and rollup

[brandonkal]

@ahuigo Snowpack IS rollup under the hood.

[newtack]

esbuild is by far the fastest bundler. It's implemented in go.

I am trying to better understand what the bundler in Deno actually is. Is it just Typescript outfile compiler option with some extras around it implemented in Typescript? Or is it using swc and Rust based libraries?

My ideal situation would be if I could have a very fast bundler in Deno itself that would include minification, tree shaking and source maps.

[David-Else]

Sorry if I am misunderstanding how this works, but is it planned in bundling v4 to be able to output an ECMAScript Module bundle without all the crazy boilerplate?

[kitsonk]

@David-Else there is extensive conversation about this in #2475 (comment). At the moment there is no plan to move away from the current bundle format, as it is currently the easiest way to ensure that the bundling is right.

Why does the internal of the bundle concern you?

[David-Else]

@kitsonk Thanks for the link, I read it and now understand the decision to include this boilerplate.

Why does the internal of the bundle concern you?

Because the results are vastly inferior to other solutions and I would like Deno to be the best solution to bundling my code. Users will not care about how Deno works inside, just that they have the best bundle possible. In 2020, it should be a standard ESM for browsers and Deno to enjoy consuming.

I just bundled some of my own code and had a look at it, the results are awful. Unreadable, wasteful cruft. The best bundler will reduce the size of the bundle without any minification using analysis, at the moment Deno adds size! I know that the source file is the 'real' code, but still many people could end up reading the deployed code on the website. Even when minified, dev-tools can 'prettify' it and users are still able to follow along.

99% of things in Deno are massively better than the competition, but unfortunately bundling is massively worse at this point. I hope things will change in the future, I could not ship this bundle to a client. We all spend countless hours trying to make our code as high quality as possible, so mutilating and bloating it at the final stage is not really an option.

I do love the philosophy of Deno regarding minimizing dependencies and doing everything in Rust where possible but not to the point of providing sub-standard results. Rollup is battle-tested and free and open source. The work is done, why not just lift the internals from that and output the best results possible? Maybe TypeScript will bundle ESM properly in the future and then Deno can go back to using TypeScript, or maybe some amazing Rust solution will appear. Until then, personally I think this needs attention.

[kitsonk]

In 2020, it should be a standard ESM for browsers and Deno to enjoy consuming.

It is a standard ES module.

If anything, blame that standards committee for choosing a module format that isn't concatenable.

The work is done, why not just lift the internals from that and output the best results possible?

Because it isn't easy to lift the internals out, and on top of it, as stated in the other issue, it does not use the TypeScript AST which means that we would have to introduce a whole set of other dependencies to get it to work.

Maybe TypeScript will bundle ESM properly

There is no such thing as "properly" bundling ESM. In fact SystemJS was/is a concurrent attempt to make ES modules down emittable. It is also the module format that jspm used up until June of this year. They moved to a situation where they don't bundle anymore, they simply just transpile CJS to ESM.

Until then, personally I think this needs attention.

I personally don't. There are a lot of other issues around usability of Deno (and bundles) that are far more important than worrying about the internals of a bundle. Like getting source maps properly working (which is something I am currently working on). At that point, with source maps, people can even worry less about what is in the bundle because they will be able to link back to the source code when debugging. Also, I am in the processes of working on moving the bundling out to Rust, which in turn will improve the speed an accuracy of bundling and make it easier to move it to swc. This will also unlock no-check for bundling, which will be yet another speed improvement. Once we have a path where we are doing all the transpiling with swc in Rust, it might be the time to look at seeing how to flatten ES modules, but right now, SystemJS is the safest, most effective way, to ensure that code can be transpiled into a concatenatable format to emit a single file bundle.

[davidcallanan]

The emitted bundle does not work well with tools that perform static analysis because all the exports come from __exp which is only computed at runtime.

[kitsonk]

The vast majority of the issues here are solved or are no longer applicable. Closing.