[WIP] BREAKING: interop, take 2

[Rich-Harris]

This supersedes #91. Summary:

• Sourcemaps are no longer generated for CommonJS modules (explanation below)
• Transformed modules are smaller, largely because namespaces are no longer needlessly generated
• The default export of a transformed CommonJS module is module.exports, or, if exports.__esModule === true, exports.default (previously it would always use .default if it was present)
• If a CommonJS module imports another CommonJS module, .default interop no longer takes place – this fixes bugs like React ApolloProvider component is undefined rollup#866

I won't lie: the approach here feels a little bit crazy. I think that's just the price of having good (and efficient) interop.

Before the explanation, a couple of comparisons:

Basic CommonJS importing

Source

// main.js
var foo = require( './foo' );
module.exports = foo * 2;

// foo.js
module.exports = 21;

Before

'use strict';

function interopDefault(ex) {
  return ex && typeof ex === 'object' && 'default' in ex ? ex['default'] : ex;
}

function createCommonjsModule(fn, module) {
  return module = { exports: {} }, fn(module, module.exports), module.exports;
}

var foo = createCommonjsModule(function (module) {
module.exports = 21;
});

var foo$1 = interopDefault(foo);


var require$$0 = Object.freeze({
  default: foo$1
});

var main = createCommonjsModule(function (module) {
var foo = interopDefault(require$$0);
module.exports = foo * 2;
});

var main$1 = interopDefault(main);

module.exports = main$1;

After

'use strict';

function createCommonjsModule(fn, module) {
  return module = { exports: {} }, fn(module, module.exports), module.exports;
}

var require$$0 = createCommonjsModule(function (module) {
module.exports = 21;
});

var main = createCommonjsModule(function (module) {
var foo = require$$0;
module.exports = foo * 2;
});

module.exports = main;

Inline require statements

Source

// main.js
module.exports = function () {
  return require( './multiply' )( 2, require( './foo' ) );
};

// multiply.js
module.exports = function ( a, b ) {
  return a * b;
};

// foo.js
module.exports = 1;

Before

'use strict';

function interopDefault(ex) {
  return ex && typeof ex === 'object' && 'default' in ex ? ex['default'] : ex;
}

function createCommonjsModule(fn, module) {
  return module = { exports: {} }, fn(module, module.exports), module.exports;
}

var multiply = createCommonjsModule(function (module) {
module.exports = function ( a, b ) {
  return a * b;
};
});

var multiply$1 = interopDefault(multiply);


var require$$1 = Object.freeze({
  default: multiply$1
});

var foo = createCommonjsModule(function (module) {
module.exports = 1;
});

var foo$1 = interopDefault(foo);


var require$$0 = Object.freeze({
  default: foo$1
});

var main = createCommonjsModule(function (module) {
module.exports = function () {
  return interopDefault(require$$1)( 2, interopDefault(require$$0) );
};
});

var main$1 = interopDefault(main);

module.exports = main$1;

After

'use strict';

function createCommonjsModule(fn, module) {
  return module = { exports: {} }, fn(module, module.exports), module.exports;
}

var require$$1 = createCommonjsModule(function (module) {
module.exports = function ( a, b ) {
  return a * b;
};
});

var require$$0 = createCommonjsModule(function (module) {
module.exports = 1;
});

var main = createCommonjsModule(function (module) {
module.exports = function () {
  return require$$1( 2, require$$0 );
};
});

module.exports = main;

How it works

Basically, when you import a CommonJS module from an ES module, you're no longer importing the module itself but a proxy, which imports the actual CommonJS module but with a prefixed ID (\0commonjs-required:/path/to/cjs-module.js). The proxy handles the ES <-> CJS interop – deciding whether module.exports or exports.default should be the default export, and adding named exports – while the module itself (with the prefixed ID) always just has a single default export which is module.exports. Its dependencies are re-declared as prefixed imports.

Meanwhile, if a module with a prefixed ID (i.e., imported by a proxy or a CommonJS module) isn't a CommonJS module, we need a different kind of interop layer, which imports the underlying ES module and exports-as-default either its default export (if there is one) or the entire namespace. That way, we don't need to reify namespaces for everything that a CommonJS module imports, only ES modules.

An unfortunate side-effect of this is that sourcemaps no longer work for CommonJS modules, because the source code lives in virtual modules, which get excluded. I haven't been able to think of a good solution to this. Honestly, I think it's a small trade-off for more reliable interop and more efficient code, given that sourcemaps are mostly useful when you're debugging your own code.

I think this covers all the bases. @rollup/collaborators and others – would welcome any feedback on this before we commit to this road! Sorry for the rambly and complex explanation.

[Rich-Harris]

@TrySound don't suppose you have any insight into why this is failing on Windows?!

[Rich-Harris]

Ah, shit. This won't work in its current form. We can't use 'virtual modules' because they get disregarded by other plugins – so code isn't transformed etc. Back to the drawing board

(╯°□°）╯︵ ┻━┻

[Rich-Harris]

Okay, back in business. I've flipped things around such that the 'real module' contains the actual module contents, while the 'virtual module' (\0commonjs-proxy:/path/to/module.js) is the proxy. The CommonJS module handles its own default interop and all the named exports, but also exports __moduleExports, which proxies (which are now just very thin wrappers) use to pass module.exports through to CommonJS consumers unmolested.

We have to jump through some slightly ugly hoops in order to prevent the entry module (if it's CommonJS) exporting __moduleExports, since it's an internal thing that you don't want messing up the bundle exports – an edge case, but a significant one. Those hoops involve hijacking all the other resolvers to determine whether we're looking at the entry module or not.

This now plays nicely with other plugins, and no longer breaks sourcemap support.

Now if we could just figure out why this isn't building on Windows...

[TrySound]

I'll check it out home.

[calvinmetcalf]

idea: only wrap the commonjs modules if there is a top level return

[TrySound]

Is top level return valid for commonjs modules?

[Rich-Harris]

@calvinmetcalf yeah, I've been thinking about this sort of thing – ideally we would be able to turn this...

module.exports = 42;

...into this...

export default 42;

..and this...

exports.foo = 'bar';

...into this...

export var foo = 'bar';

...but that's separate to this issue, which is purely about interop. We'll need to revisit it separately. (Also, I don't want to spend too much time reducing the incentives for CommonJS holdouts to join us in 2016 already...)

[calvinmetcalf]

ah I was more thinking if your doing a big rewrite, now would be the time
to throw that in (and my main goal is for using older libraries with ES6
goals)

On Wed, Aug 31, 2016 at 11:26 AM Rich Harris notifications@github.com
wrote:

@calvinmetcalf https://github.com/calvinmetcalf yeah, I've been
thinking about this sort of thing – ideally we would be able to turn this...

module.exports = 42;

...into this...

export default 42;

..and this...

exports.foo = 'bar';

...into this...

export var foo = 'bar';

...but that's separate to this issue, which is purely about interop. We'll
need to revisit it separately. (Also, I don't want to spend too much time
reducing the incentives for CommonJS holdouts to join us in 2016 already...)

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#92 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABE4n2WUszhfOyThoftQZAXGC5SGvG6pks5qlZ0kgaJpZM4Jw3l1
.

[calvinmetcalf]

yes

On Wed, Aug 31, 2016 at 11:56 AM Bogdan Chadkin notifications@github.com
wrote:

Is top level return valid for commonjs modules?

—
You are receiving this because you commented.

Reply to this email directly, view it on GitHub
#92 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABE4nxREJz2WLLMZQJTDKHW1GUfqh41-ks5qlZdEgaJpZM4Jw3l1
.

[TrySound]

Looks like dead and expensive code.

[Rich-Harris]

Awesome, thanks @TrySound. Have released this is 4.0.0.

@calvinmetcalf

ah I was more thinking if your doing a big rewrite, now would be the time
to throw that in

in my experience a big rewrite is exactly the wrong time to change the behaviour 😀 Better to lock in the bug fixes and modernise the codebase etc then tackle the wishlist. It should actually be a little bit easier to get this stuff working with the current design – I'm thinking we try to convert it the nice way, then bug out and fall back to the current createCommonjsModule approach if we can't (because of early return, assigning exports to some other variable, passing exports to a function, etc...)

[piuccio]

Just FYI. I'm using this on a project with both react and preact. Simply upgrading this plugin made the gzip bundle smaller.

68B saved when using preact, 6kB when using react. The reason why there's a big saving with react is because some useless (hope so) exported modules disappeared.

Thanks

[Rich-Harris]

@piuccio awesome! thanks for sharing those numbers. Should fall even further once we can transform CJS modules without wrapping them in createCommonjsModule(...) – have made a decent start on this but it turns out there are some changes needed in Rollup core (which I'm working on now) to fully support it

[piuccio]

Glad to know that more bytes can be saved. For correctness I should mention that the savings on react are due mostly to #93, while the savings on preact can certainly be attributed to this PR. Thanks for the great work.