Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[8.x backport] fs: add O_DSYNC flag (+ one other) #15653

Closed
wants to merge 114 commits into from

Conversation

tniessen
Copy link
Member

Manual backport of #15451 and #15547 (those should land together anyway).

Checklist
Affected core subsystem(s)

fs, doc, test

@tniessen tniessen added the v8.x label Sep 28, 2017
@nodejs-github-bot nodejs-github-bot added c++ Issues and PRs that require attention from people who are familiar with C++. v8.x labels Sep 28, 2017
const assert = require('assert');

const common = require('../common');
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't common be required before assert?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ah right... good catch

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

@MylesBorins
Copy link
Contributor

apapirovski and others added 22 commits September 29, 2017 16:36
Expand argument validation through compat API, adjust behaviour
of response.end to not throw if stream already closed to match
http1, adjust behaviour of writeContinue to not throw if stream
already closed and other very small tweaks. Add tests for added
and fixed behaviour. Add tests for edge case behaviours of
setTimeout, createPushResponse, destroy, end and trailers.

PR-URL: nodejs#15473
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Minwoo Jung <jmwsoft@gmail.com>
Remove unused onTimeout on Http2Session and Http2Stream because
the correct _onTimeout is already declared and in use. Expand
timeout tests to handle edge cases and additional arguments.

PR-URL: nodejs#15539
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de>
The message sent using process.send() goes through JSON
serialization and parsing, which could lead to surprising behaviors.
This commit elaborate a bit more on this and add a link to
the notes about these caveats in the ECMAScript specification.

PR-URL: nodejs#12963
Refs: nodejs#12497
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Vse Mozhet Byt <vsemozhetbyt@gmail.com>
Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de>
Handle edge case where stream pause is called between resume being
called and actually evaluated. Other minor adjustments to avoid
various edge cases around stream events. Add new tests that cover
all changes.

Fixes: nodejs#15491
PR-URL: nodejs#15503
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
PR-URL: nodejs#15583
Reviewed-By: Michaël Zasso <targos@protonmail.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Richard Lau <riclau@uk.ibm.com>
Reviewed-By: Gibson Fahnestock <gibfahn@gmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de>
Reviewed-By: James M Snell <jasnell@gmail.com>
Sections in the https://tools.ietf.org/html/rfc1122 are not linkable
anymore. Replace the possibly confusing hash with the explicit
section number in the text.

PR-URL: nodejs#15542
Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Refael Ackermann <refack@gmail.com>
PR-URL: nodejs#15568
Fixes: nodejs#15522
Reviewed-By: Vse Mozhet Byt <vsemozhetbyt@gmail.com>
Reviewed-By: Refael Ackermann <refack@gmail.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Gibson Fahnestock <gibfahn@gmail.com>
Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de>
Reviewed-By: James M Snell <jasnell@gmail.com>
PR-URL: nodejs#15590
Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de>
Reviewed-By: Gibson Fahnestock <gibfahn@gmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Michaël Zasso <targos@protonmail.com>
Reviewed-By: Rich Trott <rtrott@gmail.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Michael Dawson <michael_dawson@ca.ibm.com>
Reviewed-By: Refael Ackermann <refack@gmail.com>
PR-URL: nodejs#15621
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Yuta Hiroto <hello@about-hiroppy.com>
Reviewed-By: Rich Trott <rtrott@gmail.com>
Reviewed-By: Khaidi Chu <i@2333.moe>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
code samples of napi_create_object and napi_property_descriptor were
not updated to latest API.

PR-URL: nodejs#15581
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Fixes: https://github.com/nodejs/security/issues/147
PR-URL: https://github.com/nodejs-private/node-private/pull/94
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Doug Wilson <doug@somethingdoug.com>
Reviewed-By: Myles Borins <myles.borins@gmail.com>
Reviewed-By: Evan Lucas <evanlucas@me.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Michael Dawson <michael_dawson@ca.ibm.com>
* **crypto**
  * Support for multiple ECDH curves. [nodejs#15206](nodejs#15206)
* **dgram**
  * Added `setMulticastInterface()` API. [nodejs#7855](nodejs#7855)
  * Custom lookup functions are now supported. [nodejs#14560](nodejs#14560)
* **n-api**
  * The command-line flag is no longer required to use N-API. [nodejs#14902](nodejs#14902)
* **tls**
  * Docs-only deprecation of `parseCertString()`. [nodejs#14245](nodejs#14245)
* **New Contributors**
  * Welcome Sebastiaan Deckers (@sebdeckers) as a new Collaborator! [nodejs#15354](nodejs#15354)
regress/regress-crbug-514081 allocates a 2G block of memory
and if there  are multiple variants running at the
same time this can lead to crashes, OOM kills or
the OS failing to allocate memory.  This patch
limits us to running a single variant of the test

Fixes: nodejs#6340
Backport-PR-URL: nodejs#15393
PR-URL: nodejs#6678
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Fedor Indutny <fedor@indutny.com>
VS2013 does not support defaulting move constructor and assignment
operator. This adds explicit definitions of those methods for two
classes.
This fix is required because we still support building addons with
VS2013 and the incompatibility is in v8.h.

Fixes: nodejs/node-v8#4

Backport-PR-URL: nodejs#15393
PR-URL: nodejs#13263
Reviewed-By: Gibson Fahnestock <gibfahn@gmail.com>
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-By: Franziska Hinkelmann <franziska.hinkelmann@gmail.com>
Reviewed-By: Myles Borins <myles.borins@gmail.com>
Original commit message:

    [inspector] support for cases when embedder doesn't call contextDestroyed

    Node.js doesn't have good place to call contextDestroyed.
    We need to cleanup everything on our side to allow clients to not call
    contextDestroyed method.

    R=dgozman@chromium.org,eostroukhov@chromium.com

    Bug: none
    Change-Id: Ibe3f01fd18afbfa579e5db66ab6f174d5fad7c82
    Cq-Include-Trybots: master.tryserver.blink:linux_trusty_blink_rel;master.tryserver.chromium.linux:linux_chromium_rel_ng
    Reviewed-on: https://chromium-review.googlesource.com/575519
    Reviewed-by: Dmitry Gozman <dgozman@chromium.org>
    Commit-Queue: Aleksey Kozyatinskiy <kozyatinskiy@chromium.org>
    Cr-Original-Commit-Position: refs/heads/master@{nodejs#46849}
    Reviewed-on: https://chromium-review.googlesource.com/596549
    Cr-Commit-Position: refs/heads/master@{nodejs#47060}

Backport-PR-URL: nodejs#15393
PR-URL: nodejs#14730
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-By: Ali Ijaz Sheikh <ofrobots@google.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Original commit message:

  [heap] Move SweeperTask to CancelableTask
  This mitigates the problem of blocking on the main thread when the
  platform is unable to execute background tasks in a timely manner.

  Bug: v8:6655
  Change-Id: Icdaae744ee73146b86b9a28c8035138746721971
  Reviewed-on: https://chromium-review.googlesource.com/595467
  Commit-Queue: Michael Lippautz <mlippautz@chromium.org>
  Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
  Cr-Commit-Position: refs/heads/master@{nodejs#47036}

Backport-PR-URL: nodejs#15393
PR-URL: nodejs#14001
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-By: Tobias Nießen <tniessen@tnie.de>
Original commit message:

  Make CancelableTask ids unique
  They were only limited to 32 bit when using the internal Hashmap. Since
  this has changed alreay some time ago, we can switch to 64 bit ids and
  check that we never overflow.

  Bug:
  Change-Id: Ia6c6d02d6b5e555c6941185a79427dc4aa2a1d62
  Reviewed-on: https://chromium-review.googlesource.com/598229
  Commit-Queue: Michael Lippautz <mlippautz@chromium.org>
  Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
  Cr-Commit-Position: refs/heads/master@{nodejs#47085}

Backport-PR-URL: nodejs#15393
PR-URL: nodejs#14001
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-By: Tobias Nießen <tniessen@tnie.de>
Original commit message:

  [heap] Move UnmapFreeMemoryTask to CancelableTask
  This mitigates the problem of blocking on the main thread when the
  platform is unable to execute background tasks in a timely manner.

  Bug: v8:6671
  Change-Id: I741d4b7594e8d62721dad32cbfb19551ffacd0c3
  Reviewed-on: https://chromium-review.googlesource.com/599528
  Commit-Queue: Michael Lippautz <mlippautz@chromium.org>
  Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
  Cr-Commit-Position: refs/heads/master@{nodejs#47126}

Backport-PR-URL: nodejs#15393
PR-URL: nodejs#14001
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-By: Tobias Nießen <tniessen@tnie.de>
Original commit message:

    Work around glibc thread-local storage bug

    glibc before 2.17 has a bug that makes it impossible to execute
    binaries that have single-byte thread-local variables:

        % node --version
        node: error while loading shared libraries: cannot allocate
        memory in static TLS block

    Work around that by making the one instance in the V8 code base
    an int.

    See: https://sourceware.org/bugzilla/show_bug.cgi?id=14898
    See: nodesource/distributions#513
    See: nodejs/build#809
    Change-Id: Iefd8009100cd93e26cf8dc5dc03f2d622b423385
    Reviewed-on: https://chromium-review.googlesource.com/612351
    Commit-Queue: Ben Noordhuis <info@bnoordhuis.nl>
    Reviewed-by: Eric Holk <eholk@chromium.org>
    Cr-Commit-Position: refs/heads/master@{nodejs#47400}

Backport-PR-URL: nodejs#15393
PR-URL: nodejs#14913
Ref: nodejs/build#809
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-By: Michaël Zasso <targos@protonmail.com>
Reviewed-By: Nikolai Vavilov <vvnicholas@gmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Michael Dawson <michael_dawson@ca.ibm.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Original commit message:

    Add postmortem metadata for thin strings.

    See: nodejs/llnode#117
    Change-Id: Icc2830c8e9096610df33ffdc2f89e74cb1b35662
    Reviewed-on: https://chromium-review.googlesource.com/618986
    Reviewed-by: Michael Achenbach <machenbach@chromium.org>
    Commit-Queue: Ben Noordhuis <info@bnoordhuis.nl>
    Cr-Commit-Position: refs/heads/master@{nodejs#47778}

Backport-PR-URL: nodejs#15393
PR-URL: nodejs#15184
Ref: nodejs/llnode#117
Ref: nodejs/llnode#121
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Michaël Zasso <targos@protonmail.com>
Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>
Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de>
Reviewed-By: James M Snell <jasnell@gmail.com>
trivikr and others added 18 commits October 3, 2017 11:23
In shutdown(), shuttingDown was set to true before validating options.
If invalid options are passed, error was thrown and server remained in
shuttingDown state. This code change fixes it.

PR-URL: nodejs#15676
Fixes: nodejs#15666
Refs: nodejs#14985
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de>
Specify that `NaN`, `Infinity`, and `-Infinity` are no longer valid
values for the `atime` and `mtime` arguments.

PR-URL: nodejs#15680
Fixes: nodejs#15453
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de>
Add two checks that are there for expository reasons as much as they are
sanity checks.

PR-URL: nodejs#15681
Refs: nodejs/node-addon-api#142
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Tobias Nießen <tniessen@tnie.de>
PR-URL: nodejs#15729
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Gibson Fahnestock <gibfahn@gmail.com>
Reviewed-By: Tobias Nießen <tniessen@tnie.de>
Commit af6af08 introduced a build error in a Windows-only code path in
src/node_url.cc.

Fix it by making the code a little nicer in general: const-ify the
`input` parameter to `ToASCII()` and `ToUnicode()`.

PR-URL: nodejs#15724
Refs: nodejs#15615
Refs: nodejs#15723
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Gibson Fahnestock <gibfahn@gmail.com>
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Tobias Nießen <tniessen@tnie.de>
PR-URL: nodejs#15716
Reviewed-By: Yuta Hiroto <hello@about-hiroppy.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Evan Lucas <evanlucas@me.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
PR-URL: nodejs#15733
Fixes: nodejs#15730
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Gibson Fahnestock <gibfahn@gmail.com>
As per the example, `querystring.parse` actually does URL decoding, not
encoding.

PR-URL: nodejs#15706
Reviewed-By: Timothy Gu <timothygu99@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Yuta Hiroto <hello@about-hiroppy.com>
Move the one entry that is not in alphabetical order by GitHub username
in the TSC Emeriti list in README.md.

PR-URL: nodejs#15722
Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de>
Reviewed-By: Minwoo Jung <minwoo@nodesource.com>
Reviewed-By: Daniel Bevenius <daniel.bevenius@gmail.com>
Reviewed-By: Yuta Hiroto <hello@about-hiroppy.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Michaël Zasso <targos@protonmail.com>
Reviewed-By: Tobias Nießen <tniessen@tnie.de>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
When changing from a recent version, e.g. 8.x to an older branch like 6.x
test artifacts sometimes hang around and cause failures. Using `git clean`
will prevent that from happening.

PR-URL: nodejs#15715
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Gibson Fahnestock <gibfahn@gmail.com>
Reviewed-By: Refael Ackermann <refack@gmail.com>
PR-URL: nodejs#15738
Reviewed-By: Daniel Bevenius <daniel.bevenius@gmail.com>
Reviewed-By: Vse Mozhet Byt <vsemozhetbyt@gmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
PR-URL: nodejs#14098
Reviewed-By: Refael Ackermann <refack@gmail.com>
Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de>
Reviewed-By: James M Snell <jasnell@gmail.com>
PR-URL: nodejs#15693
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Anatoli Papirovski <apapirovski@mac.com>
PR-URL: nodejs#15693
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Anatoli Papirovski <apapirovski@mac.com>
PR-URL: nodejs#15693
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Anatoli Papirovski <apapirovski@mac.com>
PR-URL: nodejs#15693
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Anatoli Papirovski <apapirovski@mac.com>
PR-URL: nodejs#15748
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Andreas Madsen <amwebdk@gmail.com>
The existence of `obj.inspect()` for custom inspection can cause people
to unintentionally break `console.log()` and friends. This is a
documentation-only deprecation that can hopefully land in 8.x.

PR-URL: nodejs#15631
Refs: nodejs#15549
Reviewed-By: Vse Mozhet Byt <vsemozhetbyt@gmail.com>
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Yuta Hiroto <hello@about-hiroppy.com>
Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de>
Reviewed-By: Refael Ackermann <refack@gmail.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Jussi Räsänen and others added 2 commits October 3, 2017 22:08
PR-URL: nodejs#15451
Fixes: nodejs#15425
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Tobias Nießen <tniessen@tnie.de>
PR-URL: nodejs#15547
Refs: nodejs#15451
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de>
@tniessen
Copy link
Member Author

tniessen commented Oct 3, 2017

Sure, there you go: https://ci.nodejs.org/job/node-test-pull-request/10394/ (failures in the previous CI should be unrelated)

MylesBorins pushed a commit that referenced this pull request Oct 7, 2017
Backport-PR-URL: #15653
PR-URL: #15451
Fixes: #15425
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Tobias Nießen <tniessen@tnie.de>
MylesBorins pushed a commit that referenced this pull request Oct 7, 2017
Backport-PR-URL: #15653
PR-URL: #15547
Refs: #15451
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de>
@MylesBorins
Copy link
Contributor

landed in 27c4efd...dec383a

@MylesBorins MylesBorins closed this Oct 7, 2017
MylesBorins pushed a commit that referenced this pull request Oct 11, 2017
Backport-PR-URL: #15653
PR-URL: #15451
Fixes: #15425
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Tobias Nießen <tniessen@tnie.de>
MylesBorins pushed a commit that referenced this pull request Oct 11, 2017
Backport-PR-URL: #15653
PR-URL: #15547
Refs: #15451
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
c++ Issues and PRs that require attention from people who are familiar with C++.
Projects
None yet
Development

Successfully merging this pull request may close these issues.