ENOSPC in docker containers?

[indexzero]

Since updating to chokidar in forever we got this report: foreversd/forever#739. I don't know all that much about Docker containers, but I wanted to see if you'd seen this before.

[es128]

ENOSPC has been observed in a few cases to indicate other system limitations, like file descriptors. Bumping up the ulimit setting might help. But there are a lot of local variables at play.

What was the watching solution before that did not have this problem?

[indexzero]

Just the watch module. @paulmillr made the PR (thanks again for that!): https://github.com/foreverjs/forever-monitor/pull/96/files

[es128]

Looks like the prior solution was watch which is based on fs.watchFile (i.e. stat polling). This is less stable and efficient than other watch methods, so it is not used by default in chokidar, but there are situations (such as certain setups with networked drives and VMs) that require it.

Chokidar can be made to rely on fs.watchFile/polling using the usePolling: true option. Looks like forever may need to be tweaked to provide users the ability to set that option.

[indexzero]

I'd prefer not to fallback to that option if we can help it. I'll see if the ulimit change fixes it.

[es128]

Some users will need polling, sooner or later. The option needs to be made available.

I'm not suggesting it should be hardcoded or the default.

[siboulet]

It doesn't seems to be a ulimit issue.

Running in containter environments is getting fairly common. Shouldn't chokidar default to usePolling=true when it detects the underlying filesystem doesn't support fs.watch properly (just like it does when it detects it running on OS X)?

Anyway I tested fs.watch inside docker and it seems to work just fine:

var fs = require('fs');
fs.watch('/usr/src/app', function() {
  console.log(arguments);
})

I tried running chockibar tests inside docker but I'm getting several errors, I'm not sure where to go from there.

docker run -t -i --rm node bash
git clone https://github.com/paulmillr/chokidar
cd chokidar
npm install
npm test

Result: http://pastebin.com/edWzQsHL

[indexzero]

I'm curious how to detect that. If it's deterministic, I'd be fine with adding the detection to forever.

[es128]

Shouldn't chokidar default to usePolling=true when it detects the underlying filesystem doesn't support fs.watch properly (just like it does when it detects it running on OS X)?

Sure, once we work out a solid cross-platform way to detect that: #242

I tested fs.watch inside docker and it seems to work just fine

This isn't an equivalent test, it isn't watching recursively. Your run of the chokidar tests show that fs.watch is not flat-out failing, but is exhibiting lots of problems. But without access to your container, I can't really analyze/debug what's going on there with fs.watch (not that I really have the available time for that anyway).

The polling tests, on the other hand, seem to be doing fine. So the most straightforward solution in this particular case is going to be usePolling: true.

[siboulet]

But without access to your container, I can't really analyze/debug what's going on there with fs.watch

I've used the official node image here: https://registry.hub.docker.com/_/node/

You could just try the same tests as I did. It should pull that image for you and start a bash shell in it:

docker run -t -i --rm node bash

[es128]

Going to close this issue for now because I don't think there's anything chokidar could do differently to have an impact. The issue with fs.watch could potentially be isolated and reported to node/io.js, but it's really probably something going on at the system level.

The chokidar-level solution for this particular use-case at this point would be to enable polling (the older version of forever that does work was using a watcher that always relied on polling anyway).

[es128]

But I do intend to play with the described setup on docker at some point when I get a chance.