Parallel actions that affect package.json

[joshwcomeau]

The problem

Installing or updating dependencies is quite slow: npm install --save some-dep might take 30-40+ seconds, even on a really fast computer.

Worse, they can't easily be parallelized. For example, you can't run npm install --save some-dep and then, 2 seconds later in another terminal tab, run npm uninstall some-other-dep.

The reason for this is that both of these commands modify package.json, and NPM is smart enough to "lock" write access to the file while operations are in progress (otherwise, uninstalling the dependency might accidentally undo the first command).

In a terminal, this can be mitigated by batching commands, either by installing multiple things at once (npm install --save some-dep some-new-dep), or by chaining in the same command (npm install --save some-dep && npm uninstall some-other-dep).

In Guppy, every dependency has to be installed one at a time, and there's no way to "queue" actions. When you install a dependency right now, all other dependency actions are greyed-out until it completes. This means that adding 5 dependencies is super painful, and easily the worst UX I've discovered so far with Guppy.

The solution

While we can't break the rules of running multiple commands at once, we can create our own queue system. If the person wants to install 5 dependencies, they should just be able to click "add to project" to 5 dependencies in quick succession, and Guppy should have some UI that shows the current status on those actions, without blocking them from other stuff like running tasks.

I imagine having a toast in the top-right corner that shows which step it's currently on, and how many steps are left.

We could also try and optimize this queue. For example, when the person tries to install 4 dependencies, the 1st one will start right away... but we can then group dependencies 2, 3, and 4 into a single command, to speed things along.

The MVP version of this issue, though, is probably just to add functionality to the task reducer and middleware that handles this queue, and some basic UI to show that something is in-progress. If we discover that there's additional complexity with different action types, we can limit it just to adding or removing (eg. you can add as many dependencies as you want, but if you then try to remove a dependency, a prompt tells you to wait until all dependencies have been added before removing. And vice versa)

[superhawk610]

After some very superficial initial testing, I think it actually is possible to run concurrent npm installs - I could be wrong, but I don't think the package.json is locked, but instead the individual directories within node_modules and npm's cache directory, which generally prevents concurrent installs as only one install process can use it properly at a time. However, npm allows you to specify the cache location when installing, as such:

npm install --save create-react-app --cache=/tmp/install1
npm install --save redux --cache=/tmp/install2

I tried a few different installs alongside each other in two tmux panes opened to the same project directory, and did not notice any errors so long as the cache directories were separate and writable.

I do imagine there would be errors, or at the very least unintended results, when attempting to both install and remove the same package concurrently, but that shouldn't be an issue since the UI doesn't support that.

As for implementation, how do you feel about a persistent box at the bottom of the UI visible while any long-running action is in progress? There is already a reasonably large footer at the bottom of the project screen, so it shouldn't obstruct the main UI. It could either stack distinct tasks vertically and scroll, or stack horizontally and page.

I'll give the task reducer/middleware logic a go with the --cache method and report back. I'd like to see how it actually functions for real users since like I said, my initial testing is anything but comprehensive.

[j-f1]

You could also put a status bar in the header (kind of like Xcode) that can display multiple tasks running at once, allow you to see how each task is progressing, and show useful information when the project is idle.

[superhawk610]

Alright, so my initial solution to use a separate cache directory for each install is only a solution to having multiple parallel npm installs running across separate projects to avoid having their caches conflict with each other (this may actually be its own issue, if the user installs dependencies on multiple projects at once...). Back to the original proposed method of running them back to back and using a queue for speed optimization.

[joshwcomeau]

Alright, so my initial solution to use a separate cache directory for each install is only a solution to having multiple parallel npm installs running across separate projects to avoid having their caches conflict with each other (this may actually be its own issue, if the user installs dependencies on multiple projects at once...). Back to the original proposed method of running them back to back and using a queue for speed optimization.

Ahh I see, yeah. I'm pretty sure you can already install dependencies across projects (I think I use the project name as a key to determine if things should be disabled).

I think the original idea will be a good enough experience :) but yeah it's a fair amount of work.

[joshwcomeau]

This is done now :D Thanks for all the great work, @superhawk610 !