Flow server dies after couple of minutes

[adamTrz]

Hi. When I try to run yarn flow in our project it have difficulties to start (it needs 3-5 attempts to start correctly), and even if it does it's going dead after checking files, so with each next check I have to wait for server to start again (and maybe fail again as well).
Also, sometimes if it launches correctly, it shows errors within dependencies from node_modules despite the fact that there are excluded in .flowconfig file.
Other people working in project don't have same issues, so it's definitely not an issue with codebase or .flowconfig.
flow-bin or glow are not installed globally.

❯ cat package.json | grep flow.bin
    "flow-bin": "^0.81.0", 
❯ node -v
v9.11.2
❯ npm -v      
5.6.0
❯ yarn -v      
1.10.1

Logs from failed run:

❯ yarn flow
yarn run v1.10.1
$ flow
Launching Flow server for /Users/adamtrzcinski/Documents/GitHub/uhc-react-native
Spawned flow server (pid=90903)
Logs will go to /private/tmp/flow/zSUserszSadamtrzZcinskizSDocumentszSGitHubzSuhc-react-native.log
Monitor logs will go to /private/tmp/flow/zSUserszSadamtrzZcinskizSDocumentszSGitHubzSuhc-react-native.monitor_log
Launching Flow server for /Users/adamtrzcinski/Documents/GitHub/uhc-react-native
Spawned flow server (pid=91656)
Logs will go to /private/tmp/flow/zSUserszSadamtrzZcinskizSDocumentszSGitHubzSuhc-react-native.log
Monitor logs will go to /private/tmp/flow/zSUserszSadamtrzZcinskizSDocumentszSGitHubzSuhc-react-native.monitor_log
Launching Flow server for /Users/adamtrzcinski/Documents/GitHub/uhc-react-native
Spawned flow server (pid=92360)
Logs will go to /private/tmp/flow/zSUserszSadamtrzZcinskizSDocumentszSGitHubzSuhc-react-native.log
Monitor logs will go to /private/tmp/flow/zSUserszSadamtrzZcinskizSDocumentszSGitHubzSuhc-react-native.monitor_log
Launching Flow server for /Users/adamtrzcinski/Documents/GitHub/uhc-react-native
Spawned flow server (pid=93075)
Logs will go to /private/tmp/flow/zSUserszSadamtrzZcinskizSDocumentszSGitHubzSuhc-react-native.log
Monitor logs will go to /private/tmp/flow/zSUserszSadamtrzZcinskizSDocumentszSGitHubzSuhc-react-native.monitor_log
Lost connection to the flow server (0 retries remaining): \Out of retries, exiting!
error Command failed with exit code 7.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

Content of /private/tmp/flow:

❯ pwd               
/private/tmp/flow 
❯ ls
zSUserszSadamtrzZcinskizSDocumentszSGitHubzSuhc-react-native.lock
zSUserszSadamtrzZcinskizSDocumentszSGitHubzSuhc-react-native.log
zSUserszSadamtrzZcinskizSDocumentszSGitHubzSuhc-react-native.log.old
zSUserszSadamtrzZcinskizSDocumentszSGitHubzSuhc-react-native.monitor_log
zSUserszSadamtrzZcinskizSDocumentszSGitHubzSuhc-react-native.monitor_log.old
zSUserszSadamtrzZcinskizSDocumentszSGitHubzSuhc-react-native.pids
zSUserszSadamtrzZcinskizSDocumentszSGitHubzSuhc-react-native.sock
zSUserszSadamtrzZcinskizSDocumentszSGitHubzSuhc-react-native.sockv2
zSUserszSadamtrzZcinskizSDocumentszSGitHubzSuhc-react-native.sockv3
flowlib_2b16641
flowlib_103dc333								flowlib_2b4beca6
...and 160 more flowlib_... files

Log:

❯ cat zSUserszSadamtrzZcinskizSDocumentszSGitHubzSuhc-react-native.log
[2018-10-17 12:41:04.271] argv=/Users/adamtrzcinski/Documents/GitHub/uhc-react-native/node_modules/flow-bin/flow-osx-v0.81.0/flow start --flowconfig-name .flowconfig --temp-dir /tmp/flow /Users/adamtrzcinski/Documents/GitHub/uhc-react-native
[2018-10-17 12:41:04.284] Initializing Server (This might take some time)
[2018-10-17 12:41:04.284] executable=/Users/adamtrzcinski/Documents/GitHub/uhc-react-native/node_modules/flow-bin/flow-osx-v0.81.0/flow
[2018-10-17 12:41:04.284] version=0.81.0
[2018-10-17 12:41:04.285] Parsing
[2018-10-17 12:41:10.977] Building package heap
[2018-10-17 12:41:11.174] Loading libraries
[2018-10-17 12:41:12.319] Resolving dependencies
[2018-10-17 12:41:14.721] to_merge: Focused: 867, Dependents: 0, Dependencies: 254
[2018-10-17 12:41:14.721] Calculating dependencies
[2018-10-17 12:41:14.726] Merging
[2018-10-17 12:41:23.712] Merge skipped 0 of 1121 modules
[2018-10-17 12:41:23.722] Done
[2018-10-17 12:41:23.722] Checked set: Focused: 867, Dependents: 0, Dependencies: 254
[2018-10-17 12:41:23.725] Server is READY
[2018-10-17 12:41:23.725] Took 19.440583 seconds to initialize.
[2018-10-17 12:41:23.726] Status: Error
[2018-10-17 12:41:23.726] /Users/adamtrzcinski/Documents/GitHub/uhc-react-native/.flowconfig changed in an incompatible way. Exiting.
/Users/adamtrzcinski/Documents/GitHub/uhc-react-native/.flowconfig changed in an incompatible way. Exiting.

Monitor:

❯ cat zSUserszSadamtrzZcinskizSDocumentszSGitHubzSuhc-react-native.monitor_log
Oct 17 12:41:04.241 [info] argv=/Users/adamtrzcinski/Documents/GitHub/uhc-react-native/node_modules/flow-bin/flow-osx-v0.81.0/flow start --flowconfig-name .flowconfig --temp-dir /tmp/flow /Users/adamtrzcinski/Documents/GitHub/uhc-react-native
Oct 17 12:41:04.241 [info] Creating a new Flow server
Oct 17 12:41:16.207 [info] File watcher reported 74854 files changed
Unable to obtain kernel buffer: Operation not permitted
usage: sudo dmesg
Oct 17 12:41:23.767 [error] Flow server (pid 93077) exited with code Flowconfig_changed (16)
Oct 17 12:41:23.768 [info] Monitor is exiting (Dying along with server)
Oct 17 12:41:23.768 [info] Broadcasting to threads and waiting 1 second for them to exit

(Both logs indicate that flowconfig changed, but as I've mentioned, everyone else who's working on the project don't have those issues.)
Another thing is that sometimes flow creates a directory named Flow inside /Users/Library.
Edit: I've already read flow/flow-for-vscode#224, flow/flow-bin#104, #5620, #390, flow/flow-bin#135, #1428, #6025, flow/flow-bin#133, flow/flow-bin#98, #6424, #6171.
No success :(

[xcv58]

Same here.

[athomann]

Do you happen to be file syncing service like iCloud or dropbox? I was able to resolve issue once I disabled that.

[crotwell]

I believe this is caused by a change in OSX, with dmesg now requiring root.

dmesg
Unable to obtain kernel buffer: Operation not permitted
usage: sudo dmesg

I am guessing somewhere flow or a dependency is calling dmesg and failing. This line is the only mention of dmesg in the src, so might be it:

line 522 of src/monitor/flowServerMonitorServer.ml:
if Sys.unix && try Sys_utils.check_dmesg_for_oom pid "flow" with _ -> false

There are some security concerns with non-root use of dmesg, which is I presume why Apple disabled it.

https://googleprojectzero.blogspot.com/2018/09/a-cache-invalidation-bug-in-linux.html

[cglacet]

Any solution to this issue?

[dominicfraser]

After seeing the same error for

[error] Connection 'server #1' was closed from the other side
Unable to obtain kernel buffer: Operation not permitted
usage: sudo dmesg

looking in the log folder linked by the npm run flow output showed:

version=0.141.0
No saved state available
Parsing
[flow serverWorker 4] File node_modules/map-or-similar/map-or-similar.min.gzip.js is malformed
[flow serverWorker 5] File node_modules/memoizerific/memoizerific.min.gzip.js is malformed

doing an npm ls on both of those packages showed them to be used by Storybook packages. Reinstalling or rolling back did not seem to help.

Based on @crotwell's post I figured this was a system problem.

Adding

.*/node_modules/@storybook.*

to my .flowconfig worked around the problem but is obviously not ideal.

[SamChou19815]

The original issue doesn't produce much info for repro, and the issue is likely fixed.