[feature request] only keep last n built assets

[plrthink]

First of all, I love this gem, it helps me to integrate webpack with rails without pain. With a little customization it works so great in both environment.

One thing I think it's missing is that it can't preserve last n builds between deployments like sprockets-rails does. I think it's essential because we not only want to keep assets during deployment, but remove older ones to not eat up our disk.

[mipearson]

As in keep the manifests as well as the compiled assets? There's nothing in the compile step that would remove the old assets, but the manifest is overwritten.

Thank you for your kind words regarding the gem :)

[plrthink]

I might describe the feature in the wrong way. I mean only keep last n time builds and remove the older ones. sprockets-rails has this feature as describes here. If we don't remove older ones, the size of assets folder will get larger and larger.

[mipearson]

Oh, right!

In our case the assets we compile with webpack are quite small - JS only, no images or CSS - so it hasn't hit us yet.

I would need to look at how sprockets does it, but it'd be a good thing to support.

[plrthink]

Yeah, I would also want to figure out and contribute to this gem.

[plrthink]

I have some time to search on sprockets codebase. I find this method, it describes the algorithm about cleaning older versions of assets. I think we can use it.

However, there's one different thing. Webpack seems like don't output the information about when the asset file is built like Sprockets does. But it's fine, we can check the mtime on the file itself.

[plrthink]

Oh, I miss that mainfest.json emitted by sprockets also have info about every versions of assets like below:

"workspace-901e9d31f7757378882ea086cf3c65a7.js": {
  "logical_path": "workspace.js",
  "mtime": "2016-06-17T21:01:07+08:00",
  "size": 892279,
  "digest": "901e9d31f7757378882ea086cf3c65a7"
},
"workspace-408babeced685f1dc6d1ec68806cca81.css": {
  "logical_path": "workspace.css",
  "mtime": "2016-07-27T11:41:27+08:00",
  "size": 195125,
  "digest": "408babeced685f1dc6d1ec68806cca81"
},
"workspace-16ed83e3e9615b664a8d8113dc54222d.js": {
  "logical_path": "workspace.js",
  "mtime": "2016-08-07T15:32:02+08:00",
  "size": 916000,
  "digest": "16ed83e3e9615b664a8d8113dc54222d"
},
"workspace-1eef72f05291133f14d3322772d2db0b.css": {
  "logical_path": "workspace.css",
  "mtime": "2016-08-07T15:13:32+08:00",
  "size": 195260,
  "digest": "1eef72f05291133f14d3322772d2db0b"
}

It helps to find all previous version of files. So I think instead of overwriting manifest.json every time, we should mutate that to include new assets'info and also remove older ones like sprockets does.

[mipearson]

Harder than it sounds, unfortunately - dig in to how webpack is generating the manifest and you'll see what I mean.

[plrthink]

sounds like we need help here

[mipearson]

I would recommend a post-build step that parses the manifest.json that is created and stores a separate history file, which can then be read.

[focused]

Any progress on this? Very useful gem and I need this feature too.

Update: compiling with npm on server takes a lot of RAM and CPU, so i compile with npm locally to "public/webpack-x", then after deploy finishes in "after :finishing, :compile_webpack" task I execute cp -r <current_path>/public/webpack-x/* <shared_path>/public/webpack

[ujh]

Wouldn't it be enough to delete the old version after the deployment, i.e. remove all files not listed in the manifest? Obviously it would have to be a separate task that's executed one the server is restarted.

[focused]

Yes, it should be performed after cleaning old releases, also it would be great to compile webpack assets locally (otherwise server cpu load increased dramatically).

[ujh]

Those seem like two different tasks to me. If we have a task that removes the old data and you execute it after you copy your files to the server then your use case should be covered as well, shouldn't it?

[focused]

Yes, these are different tasks. If you keep two or more releases, you can remove old data safely, but if you keep only one release, you'll probably see your web application without css/js for a second while puma/unicorn restarts. I think it's safer to remove old assets just after restart event.

[plrthink]

I'm kind of lost in there. What's the purpose of keeping last n releases? I thought it is to ensure user won't get pages without css/js while deployments, but after reading @focused 's comments I think I'm misunderstanding this.