Skip to content

Commit

Permalink
v8: integrate node-heapdump into core
Browse files Browse the repository at this point in the history
Adds `v8.writeHeapSnapshot(filename)` with impl adapted
from the `node-heapdump` module.

Also, adds a v8.getHeapSnapshot() alternative that returns
a Readable Stream

PR-URL: #26501
Reviewed-By: Richard Lau <riclau@uk.ibm.com>
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Vse Mozhet Byt <vsemozhetbyt@gmail.com>
Reviewed-By: Sam Roberts <vieuxtech@gmail.com>
Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>
  • Loading branch information
jasnell committed Mar 13, 2019
1 parent 024842f commit 5f38797
Show file tree
Hide file tree
Showing 14 changed files with 537 additions and 15 deletions.
39 changes: 38 additions & 1 deletion LICENSE
Original file line number Diff line number Diff line change
Expand Up @@ -634,7 +634,7 @@ The externally maintained libraries used by Node.js are:

- OpenSSL, located at deps/openssl, is licensed as follows:
"""
Copyright (c) 1998-2018 The OpenSSL Project. All rights reserved.
Copyright (c) 1998-2019 The OpenSSL Project. All rights reserved.

Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
Expand Down Expand Up @@ -1445,3 +1445,40 @@ The externally maintained libraries used by Node.js are:
ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF
THE POSSIBILITY OF SUCH DAMAGE.
"""

- node-heapdump, located at src/heap_utils.cc, is licensed as follows:
"""
ISC License

Copyright (c) 2012, Ben Noordhuis <info@bnoordhuis.nl>

Permission to use, copy, modify, and/or distribute this software for any
purpose with or without fee is hereby granted, provided that the above
copyright notice and this permission notice appear in all copies.

THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.

=== src/compat.h src/compat-inl.h ===

ISC License

Copyright (c) 2014, StrongLoop Inc.

Permission to use, copy, modify, and/or distribute this software for any
purpose with or without fee is hereby granted, provided that the above
copyright notice and this permission notice appear in all copies.

THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
"""
71 changes: 71 additions & 0 deletions doc/api/v8.md
Original file line number Diff line number Diff line change
Expand Up @@ -87,6 +87,24 @@ The value returned is an array of objects containing the following properties:
]
```

## v8.getHeapSnapshot()
<!-- YAML
added: REPLACEME
-->

* Returns: {stream.Readable} A Readable Stream containing the V8 heap snapshot

Generates a snapshot of the current V8 heap and returns a Readable
Stream that may be used to read the JSON serialized representation.
This JSON stream format is intended to be used with tools such as
Chrome DevTools. The JSON schema is undocumented and specific to the
V8 engine, and may change from one version of V8 to the next.

```js
const stream = v8.getHeapSnapshot();
stream.pipe(process.stdout);
```

## v8.getHeapStatistics()
<!-- YAML
added: v1.0.0
Expand Down Expand Up @@ -159,6 +177,58 @@ v8.setFlagsFromString('--trace_gc');
setTimeout(() => { v8.setFlagsFromString('--notrace_gc'); }, 60e3);
```

## v8.writeHeapSnapshot([filename])
<!-- YAML
added: REPLACEME
-->

* `filename` {string} The file path where the V8 heap snapshot is to be
saved. If not specified, a file name with the pattern
`'Heap-${yyyymmdd}-${hhmmss}-${pid}-${thread_id}.heapsnapshot'` will be
generated, where `{pid}` will be the PID of the Node.js process,
`{thread_id}` will be `0` when `writeHeapSnapshot()` is called from
the main Node.js thread or the id of a worker thread.
* Returns: {string} The filename where the snapshot was saved.

Generates a snapshot of the current V8 heap and writes it to a JSON
file. This file is intended to be used with tools such as Chrome
DevTools. The JSON schema is undocumented and specific to the V8
engine, and may change from one version of V8 to the next.

A heap snapshot is specific to a single V8 isolate. When using
[Worker Threads][], a heap snapshot generated from the main thread will
not contain any information about the workers, and vice versa.

```js
const { writeHeapSnapshot } = require('v8');
const {
Worker,
isMainThread,
parentPort
} = require('worker_threads');

if (isMainThread) {
const worker = new Worker(__filename);

worker.once('message', (filename) => {
console.log(`worker heapdump: ${filename}`);
// Now get a heapdump for the main thread.
console.log(`main thread heapdump: ${writeHeapSnapshot()}`);
});

// Tell the worker to create a heapdump.
worker.postMessage('heapdump');
} else {
parentPort.once('message', (message) => {
if (message === 'heapdump') {
// Generate a heapdump for the worker
// and return the filename to the parent.
parentPort.postMessage(writeHeapSnapshot());
}
});
}
```

## Serialization API

> Stability: 1 - Experimental
Expand Down Expand Up @@ -417,4 +487,5 @@ A subclass of [`Deserializer`][] corresponding to the format written by
[`vm.Script`]: vm.html#vm_constructor_new_vm_script_code_options
[HTML structured clone algorithm]: https://developer.mozilla.org/en-US/docs/Web/API/Web_Workers_API/Structured_clone_algorithm
[V8]: https://developers.google.com/v8/
[Worker Threads]: worker_threads.html
[here]: https://github.com/thlorenz/v8-flags/blob/master/flags-0.11.md
11 changes: 7 additions & 4 deletions lib/internal/test/heap.js
Original file line number Diff line number Diff line change
Expand Up @@ -4,14 +4,17 @@ process.emitWarning(
'These APIs are for internal testing only. Do not use them.',
'internal/test/heap');

const { createHeapDump, buildEmbedderGraph } = internalBinding('heap_utils');
const {
createHeapSnapshot,
buildEmbedderGraph
} = internalBinding('heap_utils');
const assert = require('internal/assert');

// This is not suitable for production code. It creates a full V8 heap dump,
// parses it as JSON, and then creates complex objects from it, leading
// to significantly increased memory usage.
function createJSHeapDump() {
const dump = createHeapDump();
function createJSHeapSnapshot() {
const dump = createHeapSnapshot();
const meta = dump.snapshot.meta;

const nodes =
Expand Down Expand Up @@ -81,6 +84,6 @@ function readHeapInfo(raw, fields, types, strings) {
}

module.exports = {
createJSHeapDump,
createJSHeapSnapshot,
buildEmbedderGraph
};
60 changes: 59 additions & 1 deletion lib/v8.js
Original file line number Diff line number Diff line change
Expand Up @@ -20,9 +20,65 @@ const {
Serializer: _Serializer,
Deserializer: _Deserializer
} = internalBinding('serdes');
const assert = require('internal/assert');
const { copy } = internalBinding('buffer');
const { objectToString } = require('internal/util');
const { FastBuffer } = require('internal/buffer');
const { toPathIfFileURL } = require('internal/url');
const { validatePath } = require('internal/fs/utils');
const { toNamespacedPath } = require('path');
const {
createHeapSnapshotStream,
triggerHeapSnapshot
} = internalBinding('heap_utils');
const { Readable } = require('stream');
const { owner_symbol } = require('internal/async_hooks').symbols;
const {
kUpdateTimer,
onStreamRead,
} = require('internal/stream_base_commons');
const kHandle = Symbol('kHandle');


function writeHeapSnapshot(filename) {
if (filename !== undefined) {
filename = toPathIfFileURL(filename);
validatePath(filename);
filename = toNamespacedPath(filename);
}
return triggerHeapSnapshot(filename);
}

class HeapSnapshotStream extends Readable {
constructor(handle) {
super({ autoDestroy: true });
this[kHandle] = handle;
handle[owner_symbol] = this;
handle.onread = onStreamRead;
}

_read() {
if (this[kHandle])
this[kHandle].readStart();
}

_destroy() {
// Release the references on the handle so that
// it can be garbage collected.
this[kHandle][owner_symbol] = undefined;
this[kHandle] = undefined;
}

[kUpdateTimer]() {
// Does nothing
}
}

function getHeapSnapshot() {
const handle = createHeapSnapshotStream();
assert(handle);
return new HeapSnapshotStream(handle);
}

// Calling exposed c++ functions directly throws exception as it expected to be
// called with new operator and caused an assert to fire.
Expand Down Expand Up @@ -210,6 +266,7 @@ function deserialize(buffer) {

module.exports = {
cachedDataVersionTag,
getHeapSnapshot,
getHeapStatistics,
getHeapSpaceStatistics,
setFlagsFromString,
Expand All @@ -218,5 +275,6 @@ module.exports = {
DefaultSerializer,
DefaultDeserializer,
deserialize,
serialize
serialize,
writeHeapSnapshot
};
1 change: 1 addition & 0 deletions src/async_wrap.h
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,7 @@ namespace node {
V(FSREQPROMISE) \
V(GETADDRINFOREQWRAP) \
V(GETNAMEINFOREQWRAP) \
V(HEAPSNAPSHOT) \
V(HTTP2SESSION) \
V(HTTP2STREAM) \
V(HTTP2PING) \
Expand Down
1 change: 1 addition & 0 deletions src/env.h
Original file line number Diff line number Diff line change
Expand Up @@ -383,6 +383,7 @@ constexpr size_t kFsStatsBufferLength = kFsStatsFieldsNumber * 2;
V(script_data_constructor_function, v8::Function) \
V(secure_context_constructor_template, v8::FunctionTemplate) \
V(shutdown_wrap_template, v8::ObjectTemplate) \
V(streambaseoutputstream_constructor_template, v8::ObjectTemplate) \
V(tcp_constructor_template, v8::FunctionTemplate) \
V(tick_callback_function, v8::Function) \
V(timers_callback_function, v8::Function) \
Expand Down
Loading

0 comments on commit 5f38797

Please sign in to comment.