Packages: Replace usage of lodash with lodash-es

[gziolo]

Description

This PR replaces all usage of lodash with lodash-es in app code. It still uses lodash for tools which can't use ES modules.

This aims to improve the size of published packages by using the version of lodash which can be optimized with tools like Webpack using tree shaking.

How has this been tested?

npm test
npm run dev

Types of changes

Refactoring.

Checklist:

• My code is tested.
• My code follows the WordPress code style.
• My code follows the accessibility standards.
• My code has proper inline documentation.

[youknowriad]

Should we update the webpack config to define lodash-es as an external? This ensures we use the lodash script defined in WordPress in WP context and keepp lodash-es for package consumption.

[youknowriad]

I'm wondering about the CHANGELOG.md. Should we mention this there as an enhancement or something?

[gziolo]

Should we update the webpack config to define lodash-es as an external? This ensures we use the lodash script defined in WordPress in WP context and keepp lodash-es for package consumption.

I think we have it covered:
https://github.com/WordPress/gutenberg/blob/master/webpack.config.js#L122-L123
https://github.com/WordPress/gutenberg/blob/master/webpack.config.js#L163

I'm wondering about the CHANGELOG.md. Should we mention this there as an enhancement or something?

Yeah, good question :)

[youknowriad]

Why do we need both?

[gziolo]

Node 8 doesn't have ES modules enabled natively so we need to use lodash in all CLI tools.

[youknowriad]

Looks like there's some breaking in e2e tests, I guess it's related?

[gziolo]

e2e tests fail because of the missing lodash-es to lodash mapping ... That's another reason why we need both.

Now that I think about it, we probably should have lodash-es for build-module folder and lodash for build ...

[youknowriad]

Now that I think about it, we probably should have lodash-es for build-module folder and lodash for build ...

I don't think build is targetted for node specifically while build-module is for browsers. I think both are targeted for both environments depending on whether you need to tree shake or not. But that does raise the question about node consumption of our packages. Maybe it's fine to delay this PR until node supports es modules by default?

[gziolo]

Now that I think about it, we probably should have lodash-es for build-module folder and lodash for build ...

I don't think build is targetted for node specifically while build-module is for browsers. I think both are targeted for both environments depending on whether you need to tree shake or not. But that does raise the question about node consumption of our packages. Maybe it's fine to delay this PR until node supports es modules by default?

build targets ES5 or node? I'm not that sure what is what now :) Anyways, ES5 doesn't use import & export but lodash-es does: https://unpkg.com/lodash-es@4.17.10/lodash.js.

[aduth]

I'd wondered if we could just use an alias such that a developer could import as though from lodash proper, but at build-time it's interpreted to lodash-es if applicable. This could minimize the amount of changes necessary, and avoids confusion around "What is lodash-es?" or cases where we might actually need to use the lodash module proper (maybe related to failures we've observed).

[aduth]

Related: https://www.npmjs.com/package/babel-plugin-module-resolver

[gziolo]

Yes, I’m coming to the same conclusion after seeing what kind of issues the current proposal creates.

[gziolo]

As discussed during the weekly JS chat yesterday, it won't work as intended in the current form. Ideally we should wait for lodash v5 which would ship lodash being the main entry point in package.json and lodash-es being the module.

[aduth]

I experimented with babel-plugin-module-resolver. It works as expected.

The main awkwardness is in how a package would need to define its dependency as lodash-es, while using it as import { foo } from 'lodash';.

diff --git a/packages/babel-preset-default/index.js b/packages/babel-preset-default/index.js
index d62a0c02d..63793941d 100644
--- a/packages/babel-preset-default/index.js
+++ b/packages/babel-preset-default/index.js
@@ -21,6 +21,11 @@ module.exports = function( api ) {
 			} ],
 			'@babel/plugin-proposal-async-generator-functions',
 			! isTestEnv && [ '@babel/plugin-transform-runtime', { corejs: 2 } ],
+			[ 'module-resolver', {
+				alias: {
+					lodash: 'lodash-es',
+				},
+			} ],
 		].filter( Boolean ),
 	};
 };

[aduth]

Should we close this pull request?

[gziolo]

Yes, it will have to list both lodash versions as dependencies and lodash-es will only be used after being published to npm 😃

[ktmn]

Did something to do with lodash or underscores make it to 3.7?

My custom JS file had some const thing = _.uniq(...); defined but after 3.7 _ was no longer defined in the window at the time, but seems to be made available later.

I fixed it with some webpack config (to use window.lodash as _) but still odd that it stopped working in the first place, since the order in which I enqueue my JS didn't change.

[youknowriad]

If you use lodash or underscore in your custom JS, you have to make sure it's a dependency of your custom script.

Before 3.7 Gutenberg was using wp-api which itself was using underscore but now Gutenberg don't need these scripts anymore so we removed them.

So each time you use an external script, don't forget to add it as a dependency to your script to avoid breakage when we update Gutenberg's code.

[ktmn]

Gotcha.

These are still reliable to use though?

https://github.com/WordPress/gutenberg/blob/master/webpack.config.js#L116-L124

[youknowriad]

yes, don't forget the dependencies in your wp script definition, that's it.

[ocean90]

Is there any chance to revisit the closing of this PR?
In the past Gutenberg has replaced dependencies due to their size, so let's be good people and do the same for our packages which may not be used in a WordPress context. For example when using @wordpress/data lodash's overhead is +24KB.

[gziolo]

As far as I remember, the issue with lodash-es was that it doesn't work in Node context because it uses import/export syntax. We are waiting for Lodash 5 which should fix all the issues:

As discussed during the weekly JS chat yesterday, it won't work as intended in the current form. Ideally, we should wait for lodash v5 which would ship lodash being the main entry point in package.json and lodash-es being the module.

By the way, npm in the latest 6.9.0 release introduced aliases (https://npm.community/t/release-npm-6-9-0/5911):

Add support for package aliases. This allows packages to be installed under a different directory than the package name listed in package.json, and adds a new dependency type to allow this to be done for registry dependencies.

Examples from the test suite in the commit (npm/cli@b7b54f2):

npm install bar@npm:foo@1.2.3

[ocean90]

Thanks for the response @gziolo!
I tried npm install lodash@npm:lodash-es and directly ran into the issue with the Node context. My project has another dev dependency that depends on lodash and only runs in Node context. Due to the alias it's now also using lodash-es which obviously doesn't work. 😞

[youknowriad]

@ocean90 maybe you can install both dependencies and only alias it in the webpack config.