Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

doc: describe when stdout/err is sync #10884

Merged
merged 1 commit into from
Feb 16, 2017

Conversation

sam-github
Copy link
Contributor

process.stdout, process.stderr, and console.log() and console.error()
which use the process streams, are usually synchronous. Warn about this,
and clearly describe the conditions under which they are synchronous.

Fix: #10617

Checklist
  • documentation is changed or added
  • commit message follows commit guidelines
Affected core subsystem(s)

doc,console,process

@sam-github sam-github added console Issues and PRs related to the console subsystem. doc Issues and PRs related to the documentations. process Issues and PRs related to the process subsystem. labels Jan 18, 2017
@nodejs-github-bot nodejs-github-bot added doc Issues and PRs related to the documentations. lts-watch-v6.x labels Jan 18, 2017
Copy link
Member

@addaleax addaleax left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for doing this, it should be really helpful.

I think we typically use doc: as the subsystem for documentation-only updates, btw.

it writes to [`process.stdout`][] and [`process.stderr`][] which differ from
other Node.js streams. See the [note on process I/O][] for more information.

[note on process I/O]: process.html#process_a_note_on_process_i_o
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you move this to the other link definitions at the bottom of the file?

To check if Node.js is being run in a TTY context, read the `isTTY` property
on `process.stderr`, `process.stdout`, or `process.stdin`:
Note: `process.stderr` differs from other Node.js streams in important ways,
see [XXX][] for more information.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this be a link to the notes section, too? (ditto for the XXX below)

2. They cannot be closed ([`end()`][] will throw).
3. They will never emit the [`'finish'`][] event.
4. Writes may be synchronous depending on the what the stream is connected to
and whether the system is Windows or Unix:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It might be good to mention that synchronous writes correspond to blocking behaviour?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note that we don't do this in any other place where we wrap system calls synchronously: https://nodejs.org/api/fs.html#fs_fs_writefilesync_file_data_options for example, not sure why its so necessary here, but I'm willing.

Whether a sync call blocks or not depends on the O/S, the write buffer sizes, whether whatever process on the other side of the pipe/pty/whatever is reading data, etc. We can't just say that sync calls block, because that isn't true.

How about

If the write blocks, the synchronous call won't return until it unblocks.

If not, perhaps you can provide some text?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@addaleax If you are commenting top-to-bottom, you wouldn't notice, but this statement exists already, just a couple lines down:

Warning: Synchronous writes block the event loop until the write has completed.

@sam-github
Copy link
Contributor Author

@addaleax PTAL

@sam-github
Copy link
Contributor Author

@Fishrock123 I'd appreciate a review by you, too, when you have a moment.

under high system load, pipes that are not being read at the receiving end, or
with slow terminals or file systems, its possible for the event loop to be
blocked often enough and long enough to have severe negative performance
impacts. This may not e a problem when writing to an interactive terminal
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

typo: may not be

blocked often enough and long enough to have severe negative performance
impacts. This may not e a problem when writing to an interactive terminal
session, but consider this particularly careful when doing production logging to
the process output streams.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This writeup is much more detailed than I hoped for, thanks!

@sam-github sam-github changed the title process,console: describe when stdout/err is sync doc: describe when stdout/err is sync Jan 19, 2017
@sam-github
Copy link
Contributor Author

@Fishrock123 ?

Copy link
Contributor

@Fishrock123 Fishrock123 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Left some comments but the overall direction is good @sam-github.

* A global `console` instance configured to write to `stdout` and `stderr`.
Because this object is global, it can be used without calling
* A global `console` instance configured to write to [`process.stdout`][] and
[`process.stderr`][]. This object is global, it can be used without calling
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

, and it ?

`require('console')`.

***Warning***: The global console object's methods may be synchronous, because
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the opposite is more true: they may be asynchronous. This differers more from Node.js streams but would better reflect reality.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, browsers are synchronous for consoles too IIRC so async is even more different.

Copy link
Contributor Author

@sam-github sam-github Feb 6, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about

Warning: The global console object's methods are neither consistently synchronous like the browser APIs they resemble, nor are they consistently asynchronous like all other Node.js streams. See the note ... etc

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Better yeah, wording reads a bit weird to me but idk how to improve it atm. Something regarding the neither.

The `process.exit()` method instructs Node.js to terminate the process as
quickly as possible with the specified exit `code`. If the `code` is omitted,
The `process.exit()` method instructs Node.js to terminate the process
synchronously with the specified exit `code`. If the `code` is omitted,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we should add something like "the exit is immediate except that synchronous code may still be run from the exit event"?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

added comment about exit event

2. They cannot be closed ([`end()`][] will throw).
3. They will never emit the [`'finish'`][] event.
4. Writes may be synchronous depending on the what the stream is connected to
and whether the system is Windows or Unix:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed

4. Writes may be synchronous depending on the what the stream is connected to
and whether the system is Windows or Unix:
- Files: *synchronous* on Windows and Linux
- Terminals/TTYs: *asynchronous* on Windows, *synchronous* on Unix
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TTYs (Terminals): might be better

The `process.stderr` and `process.stdout` streams are blocking when outputting
to TTYs (terminals) on OS X as a workaround for the operating system's small,
1kb buffer size. This is to prevent interleaving between `stdout` and `stderr`.
These behaviours are partly for historical reasons, as changing them would
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think so. They are the way they are because of factors outside the node process.

Ideally they would all be synchronous I think. That is entirely impossible to implement Windows consoles* and also has other problems with pipes. (But Windows pipes have to block regardless.)

*There is no end-of-message notification IIRC

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Fishrock123 Your statement above is exactly why the it is partly for historical reasons. In your opinion pipes would be sync on Unix, but they aren't. How would you care to explain this other than "for historical reasons... backwards compat"? Also, note that I covered your point of view in the sentence below the one you comment on: "expected by some users", and that I even elaborated on why its expected with a whole paragraph. Would you like to propose some other explaination of why unix and windows are the exact opposites of each other for consoles and pipes? If one is the right way, the other is the wrong way and we have to say something about why this is.

Copy link
Contributor

@Fishrock123 Fishrock123 Feb 6, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes. Pipes on both Windows and Unix are wrong but neither is entirely our fault.

Blocking in a pipe means the process may stall forever if the downstream consumer stalls. That is not good and why pipes are (should be) async. @piscisaureus or @bnoordhuis may have more info.

It is my impression that making pipes on windows does not work correctly. See nodejs/node-v0.x-archive#3584 and 20176a9, although after reading some of the comments I am less confident about that. Maybe we could make pipes async on windows... again?

Note of course, that being async in a pipe has most of the caveats that being async in a terminal does, so it's not really good either.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Fishrock123

How would you care to explain this other than "for historical reasons... backwards compat"?

Would you like to propose some other explaination of why unix and windows are the exact opposites of each other for consoles and pipes? If one is the right way, the other is the wrong way and we have to say something about why this is

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Again:

Maybe we could make pipes async on windows... again?

I don't actually have an answer at the current time. I suppose the wording is fine.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a doc PR, I'm not about to change how the system works in it, but I'm 💯 on aligning Windows and *nix platforms.

Synchronous writes avoid problems such as output written with `console.log()` or
`console.write()` being unexpectedly interleaved, or not written at all if
`process.exit()` is called before an asynchronous write completes. See
[`process.exit()`][] for more discussion.
Copy link
Contributor

@Fishrock123 Fishrock123 Feb 6, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

more... discussion information?

[`process.exit()`][] for more discussion.

***Warning***: Synchronous writes block the event loop until the write has
completed. This can be near instantaneous in the case of output to a file, but
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

in the case of output to a file due to write-back caching.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You think our users will know what write-back caching is? Why do you think that referencing that one kind of caching will be helpful to node users? That kind of caching is only one of the things happening deep under the hood of an I/O scheduler.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmmm well I think it would be nice to mention something at least that people can do more research into.

session, but consider this particularly careful when doing production logging to
the process output streams.

To check if a stream is connected to a [TTY][] context, check the `isTTY`
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Link to isTTY?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried, but the TTY docs don't describe it as a property, its only mentioned in the text, so that's the most specific link until the TTY docs get reworked. https://nodejs.org/api/tty.html#tty_tty

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ah, ok

@sam-github
Copy link
Contributor Author

@Fishrock123 addressed most comments, have a couple open questions for you, PTAL

Copy link
Contributor

@Fishrock123 Fishrock123 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

eh, seems good enough to me, definitely better than it was

process.stdout, process.stderr, and console.log() and console.error()
which use the process streams, are usually synchronous. Warn about this,
and clearly describe the conditions under which they are synchronous.

Fix: nodejs#10617
PR-URL: nodejs#10884
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Jeremiah Senkpiel <fishrock123@rocketmail.com>
@sam-github sam-github merged commit 5ba00af into nodejs:master Feb 16, 2017
@sam-github
Copy link
Contributor Author

Thanks for the review, @Fishrock123

@italoacasas doc-only change, would be nice to have in 7.x, but not a disaster if it is not

italoacasas pushed a commit to italoacasas/node that referenced this pull request Feb 20, 2017
process.stdout, process.stderr, and console.log() and console.error()
which use the process streams, are usually synchronous. Warn about this,
and clearly describe the conditions under which they are synchronous.

Fix: nodejs#10617
PR-URL: nodejs#10884
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Jeremiah Senkpiel <fishrock123@rocketmail.com>
italoacasas pushed a commit that referenced this pull request Feb 22, 2017
process.stdout, process.stderr, and console.log() and console.error()
which use the process streams, are usually synchronous. Warn about this,
and clearly describe the conditions under which they are synchronous.

Fix: #10617
PR-URL: #10884
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Jeremiah Senkpiel <fishrock123@rocketmail.com>
@italoacasas italoacasas mentioned this pull request Feb 25, 2017
jasnell pushed a commit that referenced this pull request Mar 7, 2017
process.stdout, process.stderr, and console.log() and console.error()
which use the process streams, are usually synchronous. Warn about this,
and clearly describe the conditions under which they are synchronous.

Fix: #10617
PR-URL: #10884
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Jeremiah Senkpiel <fishrock123@rocketmail.com>
@jasnell
Copy link
Member

jasnell commented Mar 7, 2017

needs a backport PR in order to land on v4

MylesBorins pushed a commit that referenced this pull request Mar 9, 2017
process.stdout, process.stderr, and console.log() and console.error()
which use the process streams, are usually synchronous. Warn about this,
and clearly describe the conditions under which they are synchronous.

Fix: #10617
PR-URL: #10884
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Jeremiah Senkpiel <fishrock123@rocketmail.com>
@MylesBorins MylesBorins mentioned this pull request Mar 9, 2017
@sam-github sam-github deleted the doc-stdio-syncness branch April 17, 2017 20:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
console Issues and PRs related to the console subsystem. doc Issues and PRs related to the documentations. process Issues and PRs related to the process subsystem.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants