[WIP] Try caching compiled JavaScript when building packages

[noisysocks]

Continuing on from yesterday's experimentation (#13103) into improving some of our build times.

This changes npm run build:packages to cache the compiled JavaScript that Babel generates. Doing so makes npm install, npm run dev, and npm run build run significantly quicker.

To test, run npm run build:packages. The first run will take a while but subsequent runs will be quicker.

$ time npm run build:packages
$ time npm run build:packages

[noisysocks]

@youknowriad: Do you think this is worth continuing with? I added caching to npm run build:packages in a way that lets us configure Travis CI to cache the cache directory.

Pros:

• npm run build, npm run dev and npm install are all faster. npm run build:packages now takes ~7 seconds, down from ~26 seconds.
• Travis CI jobs are all faster. When combined with Improve Travis CI build times #13103 it means that our JS unit tests job runs in less than 4 minutes.

Cons:

• It makes the packages build script more complex.
• It dumps build artefacts onto people's machines (~/.cache/wordpress-packages).

A compromise could be to only enable caching on Travis CI.

[noisysocks]

cc. @gziolo as well 🙂

[gziolo]

There is also a concurrent effort started by @aduth where packages are built async. #8093 was merged and reverted not that long ago. I would like to learn what went wrong, so we could better evaluate the proposed changes. Also this from @talldan:

I had a similar attempt at this over christmas (didn't realise this PR existed). I also didn't see any real improvement in build times, so didn't bother tidying things up:
#13104

It looks like you all have more expertise in this area than I do. I'm fine with all proposals which makes builds faster. With this PR in place we wouldn't have to see packages to be rebuilt every time you run npm run build when nothing has really changed. It's really annoying at the moment.

[noisysocks]

I've refreshed this this PR. I think it's worth continuing with.

The only real concern I have is that it means that we're dumping build artefacts into a hidden directory (~/.cache/wordpress-packages) on people's machines.

(Mind you, running ls -a | grep "^\." is pretty revealing: we certainly wouldn't be the only script that does this!)

[gziolo]

See https://github.com/WordPress/gutenberg/blob/master/.travis.yml#L15-L19

This is what it is, I wouldn't worry about it :)

[gziolo]

@omarreiss is it something that was ever discussed in the context of WordPress core? To cache build artifacts? Well, it might be specific to packages only...

[aduth]

What's the status of this pull request? Will you plan to revisit it?

For what it's worth, I've created a separate tracking issue for general Travis build performance ideas at #15159 . There were similar optimizations at #14860 which leverage Babel cache which seems to be similar in impact to what's been proposed here.

[omarreiss]

@gziolo I somehow missed your question. We didn't consider this yet for WordPress core, but definitely seems like we should consolidate solutions like this one to WordPress core. The optimizations for core so far focused primarily on improving the efficiency of rebuilds on grunt watch. The setup there is still quite behind on the Gutenberg one. cc @herregroen.

[noisysocks]

What's the status of this pull request? Will you plan to revisit it?

This was a quick proof of concept for an idea that I think still has merit. I'll close the PR though as I am not able to give it the time it deserves.