lib,src: support values > 4GB in heap statistics #10186

bnoordhuis · 2016-12-08T23:19:42Z

bnoordhuis · 2016-12-08T23:37:03Z

Forgot to mention, I didn't add a test because I don't think I can make it reliable. It only works on 64 bits architectures and even there preallocating too much address space might lead to a run-in with the OOM killer.

evanlucas

LGTM. CI: https://ci.nodejs.org/job/node-test-pull-request/5342/

We were transporting the heap statistics as uint32 values to JS land but those wrap around for values > 4 GB. Use 64 bits floats instead, those should last us a while. Fixes: nodejs#10185 PR-URL: nodejs#10186 Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: Evan Lucas <evanlucas@me.com>

bnoordhuis · 2016-12-11T13:49:07Z

Landed in aa77b76, thanks for reviewing everyone.

We were transporting the heap statistics as uint32 values to JS land but those wrap around for values > 4 GB. Use 64 bits floats instead, those should last us a while. Fixes: nodejs#10185 PR-URL: nodejs#10186 Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: Evan Lucas <evanlucas@me.com>

tamasmahr · 2017-01-25T07:58:21Z

Hi,

This is a crucial fix for us. When is this going to make it into an LTS release?

Thank you!

Tamas

addaleax · 2017-01-25T18:48:36Z

@tamasmahr Unfortunately, this first has to live in a v7.x release for two weeks before it will be included in an LTS release, so it might still take a bit. :/

We were transporting the heap statistics as uint32 values to JS land but those wrap around for values > 4 GB. Use 64 bits floats instead, those should last us a while. Fixes: nodejs#10185 PR-URL: nodejs#10186 Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: Evan Lucas <evanlucas@me.com>

Somebi · 2017-02-15T20:57:07Z

It seems this is still not implemented or half-implemented/tested. Record Allocation Timeline inside inspector can't be displayed properly because if this or something related to this fix.

bnoordhuis · 2017-02-15T21:02:52Z

@Somebi #11401 is not related to #10185.

We were transporting the heap statistics as uint32 values to JS land but those wrap around for values > 4 GB. Use 64 bits floats instead, those should last us a while. Fixes: #10185 PR-URL: #10186 Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: Evan Lucas <evanlucas@me.com>

MylesBorins · 2017-03-07T01:49:50Z

Just landed this in v4 and v6 staging. Expect an RC with this included v. soon

We were transporting the heap statistics as uint32 values to JS land but those wrap around for values > 4 GB. Use 64 bits floats instead, those should last us a while. Fixes: #10185 PR-URL: #10186 Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: Evan Lucas <evanlucas@me.com>

Somebi · 2017-03-07T10:57:59Z

Thank you. I have noticed, that CPU profiling is not working as well. I these issues are related or i should open new issue?

We were transporting the heap statistics as uint32 values to JS land but those wrap around for values > 4 GB. Use 64 bits floats instead, those should last us a while. Fixes: #10185 PR-URL: #10186 Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: Evan Lucas <evanlucas@me.com>

Notable changes * performance: The performance of several APIs has been improved. - `Buffer.compare()` is up to 35% faster on average. (Brian White) #10927 - `buffer.toJSON()` is up to 2859% faster on average. (Brian White) #10895 - `fs.*statSync()` functions are now up to 9.3% faster on average. (Brian White) #11522 - `os.loadavg` is up to 151% faster. (Brian White) #11516 - `process.memoryUsage()` is up to 34% faster. (Brian White) #11497 - `querystring.unescape()` for `Buffer`s is 15% faster on average. (Brian White) #10837 - `querystring.stringify()` is up to 7.8% faster on average. (Brian White) #10852 - `querystring.parse()` is up to 21% faster on average. (Brian White) #10874 * IPC: - Batched writes have been enabled for process IPC on platforms that support Unix Domain Sockets. (Alexey Orlenko) #10677 - Performance gains may be up to 40% for some workloads. * child_process: - `spawnSync` now returns a null `status` when child is terminated by a signal. (cjihrig) #11288 - This fixes the behavior to act like `spawn()` does. * http: - Control characters are now always rejected when using `http.request()`. (Ben Noordhuis) #8923 - Debug messages have been added for cases when headers contain invalid values. (Evan Lucas) #9195 * node: - Heap statistics now support values larger than 4GB. (Ben Noordhuis) #10186 * timers: - Timer callbacks now always maintain order when interacting with domain error handling. (John Barboza) #10522 PR-URL: #11759

Notable Changes: * buffer: - The performance of `.toJSON()` is now up to 2859% faster on average (Brian White) #10895 * IPC: - Batched writes have been enabled for process IPC on platforms that support Unix Domain Sockets. (Alexey Orlenko) #10677 - Performance gains may be up to 40% for some workloads. * http: - Control characters are now always rejected when using `http.request()`. (Ben Noordhuis) #8923 * node: - Heap statistics now support values larger than 4GB. (Ben Noordhuis) #10186

Notable Changes: * buffer: - The performance of `.toJSON()` is now up to 2859% faster on average (Brian White) #10895 * IPC: - Batched writes have been enabled for process IPC on platforms that support Unix Domain Sockets. (Alexey Orlenko) #10677 - Performance gains may be up to 40% for some workloads. * http: - Control characters are now always rejected when using `http.request()`. (Ben Noordhuis) #8923 * node: - Heap statistics now support values larger than 4GB. (Ben Noordhuis) #10186 PR-URL: #11760

Notable changes * performance: The performance of several APIs has been improved. - `Buffer.compare()` is up to 35% faster on average. (Brian White) #10927 - `buffer.toJSON()` is up to 2859% faster on average. (Brian White) #10895 - `fs.*statSync()` functions are now up to 9.3% faster on average. (Brian White) #11522 - `os.loadavg` is up to 151% faster. (Brian White) #11516 - `process.memoryUsage()` is up to 34% faster. (Brian White) #11497 - `querystring.unescape()` for `Buffer`s is 15% faster on average. (Brian White) #10837 - `querystring.stringify()` is up to 7.8% faster on average. (Brian White) #10852 - `querystring.parse()` is up to 21% faster on average. (Brian White) #10874 * IPC: - Batched writes have been enabled for process IPC on platforms that support Unix Domain Sockets. (Alexey Orlenko) #10677 - Performance gains may be up to 40% for some workloads. * child_process: - `spawnSync` now returns a null `status` when child is terminated by a signal. (cjihrig) #11288 - This fixes the behavior to act like `spawn()` does. * http: - Control characters are now always rejected when using `http.request()`. (Ben Noordhuis) #8923 - Debug messages have been added for cases when headers contain invalid values. (Evan Lucas) #9195 * node: - Heap statistics now support values larger than 4GB. (Ben Noordhuis) #10186 * timers: - Timer callbacks now always maintain order when interacting with domain error handling. (John Barboza) #10522 PR-URL: #11759

Notable Changes: * buffer: - The performance of `.toJSON()` is now up to 2859% faster on average (Brian White) nodejs/node#10895 * IPC: - Batched writes have been enabled for process IPC on platforms that support Unix Domain Sockets. (Alexey Orlenko) nodejs/node#10677 - Performance gains may be up to 40% for some workloads. * http: - Control characters are now always rejected when using `http.request()`. (Ben Noordhuis) nodejs/node#8923 * node: - Heap statistics now support values larger than 4GB. (Ben Noordhuis) nodejs/node#10186 Signed-off-by: Ilkka Myller <ilkka.myller@nodefield.com>

Notable changes * performance: The performance of several APIs has been improved. - `Buffer.compare()` is up to 35% faster on average. (Brian White) nodejs/node#10927 - `buffer.toJSON()` is up to 2859% faster on average. (Brian White) nodejs/node#10895 - `fs.*statSync()` functions are now up to 9.3% faster on average. (Brian White) nodejs/node#11522 - `os.loadavg` is up to 151% faster. (Brian White) nodejs/node#11516 - `process.memoryUsage()` is up to 34% faster. (Brian White) nodejs/node#11497 - `querystring.unescape()` for `Buffer`s is 15% faster on average. (Brian White) nodejs/node#10837 - `querystring.stringify()` is up to 7.8% faster on average. (Brian White) nodejs/node#10852 - `querystring.parse()` is up to 21% faster on average. (Brian White) nodejs/node#10874 * IPC: - Batched writes have been enabled for process IPC on platforms that support Unix Domain Sockets. (Alexey Orlenko) nodejs/node#10677 - Performance gains may be up to 40% for some workloads. * child_process: - `spawnSync` now returns a null `status` when child is terminated by a signal. (cjihrig) nodejs/node#11288 - This fixes the behavior to act like `spawn()` does. * http: - Control characters are now always rejected when using `http.request()`. (Ben Noordhuis) nodejs/node#8923 - Debug messages have been added for cases when headers contain invalid values. (Evan Lucas) nodejs/node#9195 * node: - Heap statistics now support values larger than 4GB. (Ben Noordhuis) nodejs/node#10186 * timers: - Timer callbacks now always maintain order when interacting with domain error handling. (John Barboza) nodejs/node#10522 PR-URL: nodejs/node#11759 Signed-off-by: Ilkka Myller <ilkka.myller@nodefield.com>

Notable Changes: * buffer: - The performance of `.toJSON()` is now up to 2859% faster on average (Brian White) nodejs/node#10895 * IPC: - Batched writes have been enabled for process IPC on platforms that support Unix Domain Sockets. (Alexey Orlenko) nodejs/node#10677 - Performance gains may be up to 40% for some workloads. * http: - Control characters are now always rejected when using `http.request()`. (Ben Noordhuis) nodejs/node#8923 * node: - Heap statistics now support values larger than 4GB. (Ben Noordhuis) nodejs/node#10186 Signed-off-by: Ilkka Myller <ilkka.myller@nodefield.com>

Notable changes * performance: The performance of several APIs has been improved. - `Buffer.compare()` is up to 35% faster on average. (Brian White) nodejs/node#10927 - `buffer.toJSON()` is up to 2859% faster on average. (Brian White) nodejs/node#10895 - `fs.*statSync()` functions are now up to 9.3% faster on average. (Brian White) nodejs/node#11522 - `os.loadavg` is up to 151% faster. (Brian White) nodejs/node#11516 - `process.memoryUsage()` is up to 34% faster. (Brian White) nodejs/node#11497 - `querystring.unescape()` for `Buffer`s is 15% faster on average. (Brian White) nodejs/node#10837 - `querystring.stringify()` is up to 7.8% faster on average. (Brian White) nodejs/node#10852 - `querystring.parse()` is up to 21% faster on average. (Brian White) nodejs/node#10874 * IPC: - Batched writes have been enabled for process IPC on platforms that support Unix Domain Sockets. (Alexey Orlenko) nodejs/node#10677 - Performance gains may be up to 40% for some workloads. * child_process: - `spawnSync` now returns a null `status` when child is terminated by a signal. (cjihrig) nodejs/node#11288 - This fixes the behavior to act like `spawn()` does. * http: - Control characters are now always rejected when using `http.request()`. (Ben Noordhuis) nodejs/node#8923 - Debug messages have been added for cases when headers contain invalid values. (Evan Lucas) nodejs/node#9195 * node: - Heap statistics now support values larger than 4GB. (Ben Noordhuis) nodejs/node#10186 * timers: - Timer callbacks now always maintain order when interacting with domain error handling. (John Barboza) nodejs/node#10522 PR-URL: nodejs/node#11759 Signed-off-by: Ilkka Myller <ilkka.myller@nodefield.com>

nodejs-github-bot added c++ Issues and PRs that require attention from people who are familiar with C++. v8 engine Issues and PRs related to the V8 dependency. lts-watch-v6.x labels Dec 8, 2016

bnoordhuis mentioned this pull request Dec 8, 2016

The property heap_size_limit from getHeapStatistics is incorrect for memory over 4032MB #10185

Closed

addaleax approved these changes Dec 9, 2016

View reviewed changes

addaleax added the lts-watch-v4.x label Dec 9, 2016

cjihrig approved these changes Dec 9, 2016

View reviewed changes

evanlucas approved these changes Dec 9, 2016

View reviewed changes

bnoordhuis force-pushed the fix10185 branch from 44f1053 to aa77b76 Compare December 11, 2016 13:48

nodejs-github-bot added the dont-land-on-v7.x label Dec 11, 2016

bnoordhuis closed this Dec 11, 2016

bnoordhuis deleted the fix10185 branch December 11, 2016 13:48

bnoordhuis merged commit aa77b76 into nodejs:master Dec 11, 2016

mscdex removed the dont-land-on-v7.x label Jan 13, 2017

italoacasas mentioned this pull request Jan 25, 2017

Proposal to update v7.x-staging #10989

Closed

italoacasas mentioned this pull request Jan 29, 2017

v7.5.0 proposal #11062

Merged

algv mentioned this pull request Feb 15, 2017

inspector Record Allocation Timeline bugged #11401

Closed

MylesBorins added land-on-v4.x and removed lts-watch-v4.x labels Mar 7, 2017

MylesBorins mentioned this pull request Mar 9, 2017

v6.10.1 proposal #11759

Merged

MylesBorins mentioned this pull request Mar 9, 2017

v4.8.1 proposal #11760

Merged

bnb mentioned this pull request Mar 11, 2020

Move to Electron's docs-parser tooling for Node.js documentation #32206

Open

4 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

lib,src: support values > 4GB in heap statistics #10186

lib,src: support values > 4GB in heap statistics #10186

bnoordhuis commented Dec 8, 2016

bnoordhuis commented Dec 8, 2016

evanlucas left a comment

bnoordhuis commented Dec 11, 2016

tamasmahr commented Jan 25, 2017

addaleax commented Jan 25, 2017

Somebi commented Feb 15, 2017 •

edited

Loading

bnoordhuis commented Feb 15, 2017

MylesBorins commented Mar 7, 2017

Somebi commented Mar 7, 2017

lib,src: support values > 4GB in heap statistics #10186

lib,src: support values > 4GB in heap statistics #10186

Conversation

bnoordhuis commented Dec 8, 2016

bnoordhuis commented Dec 8, 2016

evanlucas left a comment

Choose a reason for hiding this comment

bnoordhuis commented Dec 11, 2016

tamasmahr commented Jan 25, 2017

addaleax commented Jan 25, 2017

Somebi commented Feb 15, 2017 • edited Loading

bnoordhuis commented Feb 15, 2017

MylesBorins commented Mar 7, 2017

Somebi commented Mar 7, 2017

Somebi commented Feb 15, 2017 •

edited

Loading