node --watch restarts without waiting for pending I/O

[laci4321]

Version

20.1.0

Platform

Linux ##hostname## 5.15.0-25-generic #25-Ubuntu SMP Wed Mar 30 15:54:22 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux, Ubuntu 22.04

Subsystem

No response

What steps will reproduce the bug?

When I create a change in one of my watched file (by node itself with node --watc flag), the node server will restart. During this process, one of my .json file sometimes losing it's complete content and becomes an empty file. This crashes my server since this .json file has important informations to be able tu run properly.

I have this line of code, which is overwrite a content of a .json file in the server, when I load the main page of my website.

await fs.promises.writeFile('./serverPrivateOptions.json', JSON.stringify(serverPrivateOptions, null, 4));

I think this is happening because the node do not waiting until the promise of await fs.promises.writeFile() method resolves. Immediatelly restarts the server after the writeFile method deletes the contents of the file, but before ables to write the new content into it.

How often does it reproduce? Is there a required condition?

This is not a common problem, but reproducable with trying in a few minutes. It occurs more often, when I try to reload the webpage as fast as I can after a changing in one of my server files. (so during the server restart process).

What is the expected behavior? Why is that the expected behavior?

Node waits with restart until the end of the fileSystem methods.

What do you see instead?

I think this is happening because the node do not waiting until the promise of await fs.promises.writeFile() method resolves. Immediatelly restarts the server after the writeFile method deletes the contents of the file, but before ables to write the new content into it.

Additional information

I can not see the logs, because the --watch flag clears the console on every restart. (BTW Why?) So I put 2 extra line of code to be able to save log.json files outside of the directory of my app, so this is not watched by the node itself. Like this:

await fs.promises.writeFile('../logBefore.json', JSON.stringify(`Trying to save serverPrivateOptions.json ${new Date()}`, null, 4));        //this is not watched by node
await fs.promises.writeFile('./serverPrivateOptions.json', JSON.stringify(serverPrivateOptions, null, 4));          //this is watched by node
await fs.promises.writeFile('../logAfter.json', JSON.stringify(`Refreshing of serverPrivateOptions.json was successful! ${new Date()}`, null, 4));        //this is not watched by node

The external log files indicates that the server crashes between this 2 lines of codes. You can see that the logBefore contains a later time value, compared to the logAfter. (logAfter has a value because I tried to reproduce the problem multiple times.)
Content of logBefore.json: "Trying to save serverPrivateOptions.json Sat May 13 2023 19:41:01 GMT+0200 (Central European Summer Time)"
Content of logAfter.json: "Refreshing of serverPrivateOptions.json was successful! Sat May 13 2023 19:39:53 GMT+0200 (Central European Summer Time)"

[bnoordhuis]

I think this is a misunderstanding on your part, not a bug, but I'll grant that you can't really divine it from reading the documentation for the --watch switch.

Process restart sends a SIGTERM signal to the node process. Node terminates immediately unless you listen for that signal with process.on('SIGTERM', () => { /* ... */ }). It won't wait for pending promises or other work to finish.

[tniessen]

What is the expected behavior? Why is that the expected behavior?

Node waits with restart until the end of the fileSystem methods.

The current implementation of --watch is very simple: the actual application is started as a child process, so the parent process doesn't know what's going on inside the application. When a relevant change is detected, the parent process sends SIGTERM to the child process, killing it immediately unless a signal handler has been set up within the application process (as @bnoordhuis described).

--watch is mostly meant as a small development aid and not as a general-purpose automatic restart mechanism. If you need to restart your application due to a configuration change, it's better to implement that logic yourself than to rely on --watch.

If you really want to restart using --watch based on a file that your own application process is modifying, you can do one of the following:

1. Catch SIGTERM as described above.
2. Make the event atomic from the watcher's perspective. Write the file contents to an unwatched path within the same file system, then use rename() to move it to the intended path. The watcher will only observe the rename() operation, which is considered atomic in this case.

[gowthm]

The Node --watch Starts Node.js in watch mode. When in watch mode, changes in the watched files cause the Node.js process to restart. By default, watch mode will watch the entry point and any required or imported module. Use --watch-path to specify what paths to watch.
node --watch index.js
https://nodejs.org/api/cli.html#--watch

[tniessen]

Closing as answered. Please comment, reopen, or open a new issue if that's incorrect.