-
Notifications
You must be signed in to change notification settings - Fork 30k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
node crashes when v8.getHeapSnapshot is called in succession. #35559
Comments
Can you post the stack trace here? |
@joyeecheung - Thanks for the reply and here is the stack trace of the code.
|
Just noticed that I ran into this independently of this issue – https://chromium-review.googlesource.com/c/v8/v8/+/2464823 fixes this. |
@addaleax so is this a problem with the v8 GC or node? |
@RaisinTen It’s a V8 bug, yes, not caused by the GC implementation but related to GC timing. We could work around it if necessary, but usually the V8 team responds very quickly during working hours (… faster than Node.js for sure 🙂 ) |
@addaleax 🙂
Did I get it right? I don't think I understand how we can work around this though. |
@RaisinTen Not quite… the process for creating a snapshot is:
Now… that last bit is a bit tricky: V8 implements “Delete the snapshot object” in 2 different ways:
Part of the problem is the word “complete” here. Now, when running the script from the issue here …
|
@addaleax thank you for the awesome explanation! 🙂 |
Original commit message: [heap-profiler] Fix crash when a snapshot deleted while taking one Fix a crash/hang that occurred when deleting a snapshot during the GC that is part of taking another one. Specifically, when deleting the only other snapshot in such a situation, the `v8::HeapSnapshot::Delete()` method sees that there is only one (complete) snapshot at that point, and decides that it is okay to perform “delete all snapshots” instead of just deleting the requested one. That resets the internal string lookup table of the heap profiler, but the new snapshot that is currently in progress still holds references to the old string lookup table, leading to a use-after-free segfault or infinite loop. Fix this by guarding against resetting the string table while another heap snapshot is being taken, and add a test that would crash before this fix. This can be triggered in Node.js by repeatedly calling `v8.getHeapSnapshot()`, which provides heap snapshots as weakly held host objects. Change-Id: If9ac3728bf79114000982f1e7bb05e8034299e3c Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2464823 Reviewed-by: Ulan Degenbaev <ulan@chromium.org> Commit-Queue: Ulan Degenbaev <ulan@chromium.org> Cr-Commit-Position: refs/heads/master@{#70445} Refs: v8/v8@3176bfd PR-URL: #35612 Refs: #35559 Reviewed-By: Michaël Zasso <targos@protonmail.com> Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: Richard Lau <rlau@redhat.com> Reviewed-By: Jiawen Geng <technicalcute@gmail.com> Reviewed-By: Rich Trott <rtrott@gmail.com> Reviewed-By: Gireesh Punathil <gpunathi@in.ibm.com> Reviewed-By: Gerhard Stöbich <deb2001-github@yahoo.de>
Original commit message: [heap-profiler] Fix crash when a snapshot deleted while taking one Fix a crash/hang that occurred when deleting a snapshot during the GC that is part of taking another one. Specifically, when deleting the only other snapshot in such a situation, the `v8::HeapSnapshot::Delete()` method sees that there is only one (complete) snapshot at that point, and decides that it is okay to perform “delete all snapshots” instead of just deleting the requested one. That resets the internal string lookup table of the heap profiler, but the new snapshot that is currently in progress still holds references to the old string lookup table, leading to a use-after-free segfault or infinite loop. Fix this by guarding against resetting the string table while another heap snapshot is being taken, and add a test that would crash before this fix. This can be triggered in Node.js by repeatedly calling `v8.getHeapSnapshot()`, which provides heap snapshots as weakly held host objects. Change-Id: If9ac3728bf79114000982f1e7bb05e8034299e3c Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2464823 Reviewed-by: Ulan Degenbaev <ulan@chromium.org> Commit-Queue: Ulan Degenbaev <ulan@chromium.org> Cr-Commit-Position: refs/heads/master@{#70445} Refs: v8/v8@3176bfd PR-URL: #35612 Refs: #35559 Reviewed-By: Michaël Zasso <targos@protonmail.com> Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: Richard Lau <rlau@redhat.com> Reviewed-By: Jiawen Geng <technicalcute@gmail.com> Reviewed-By: Rich Trott <rtrott@gmail.com> Reviewed-By: Gireesh Punathil <gpunathi@in.ibm.com> Reviewed-By: Gerhard Stöbich <deb2001-github@yahoo.de>
Fixes: #35559 PR-URL: #35612 Reviewed-By: Michaël Zasso <targos@protonmail.com> Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: Richard Lau <rlau@redhat.com> Reviewed-By: Jiawen Geng <technicalcute@gmail.com> Reviewed-By: Rich Trott <rtrott@gmail.com> Reviewed-By: Gireesh Punathil <gpunathi@in.ibm.com> Reviewed-By: Gerhard Stöbich <deb2001-github@yahoo.de>
Original commit message: [heap-profiler] Fix crash when a snapshot deleted while taking one Fix a crash/hang that occurred when deleting a snapshot during the GC that is part of taking another one. Specifically, when deleting the only other snapshot in such a situation, the `v8::HeapSnapshot::Delete()` method sees that there is only one (complete) snapshot at that point, and decides that it is okay to perform “delete all snapshots” instead of just deleting the requested one. That resets the internal string lookup table of the heap profiler, but the new snapshot that is currently in progress still holds references to the old string lookup table, leading to a use-after-free segfault or infinite loop. Fix this by guarding against resetting the string table while another heap snapshot is being taken, and add a test that would crash before this fix. This can be triggered in Node.js by repeatedly calling `v8.getHeapSnapshot()`, which provides heap snapshots as weakly held host objects. Change-Id: If9ac3728bf79114000982f1e7bb05e8034299e3c Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2464823 Reviewed-by: Ulan Degenbaev <ulan@chromium.org> Commit-Queue: Ulan Degenbaev <ulan@chromium.org> Cr-Commit-Position: refs/heads/master@{#70445} Refs: v8/v8@3176bfd PR-URL: nodejs#35612 Refs: nodejs#35559 Reviewed-By: Michaël Zasso <targos@protonmail.com> Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: Richard Lau <rlau@redhat.com> Reviewed-By: Jiawen Geng <technicalcute@gmail.com> Reviewed-By: Rich Trott <rtrott@gmail.com> Reviewed-By: Gireesh Punathil <gpunathi@in.ibm.com> Reviewed-By: Gerhard Stöbich <deb2001-github@yahoo.de>
Original commit message: [heap-profiler] Fix crash when a snapshot deleted while taking one Fix a crash/hang that occurred when deleting a snapshot during the GC that is part of taking another one. Specifically, when deleting the only other snapshot in such a situation, the `v8::HeapSnapshot::Delete()` method sees that there is only one (complete) snapshot at that point, and decides that it is okay to perform “delete all snapshots” instead of just deleting the requested one. That resets the internal string lookup table of the heap profiler, but the new snapshot that is currently in progress still holds references to the old string lookup table, leading to a use-after-free segfault or infinite loop. Fix this by guarding against resetting the string table while another heap snapshot is being taken, and add a test that would crash before this fix. This can be triggered in Node.js by repeatedly calling `v8.getHeapSnapshot()`, which provides heap snapshots as weakly held host objects. Change-Id: If9ac3728bf79114000982f1e7bb05e8034299e3c Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2464823 Reviewed-by: Ulan Degenbaev <ulan@chromium.org> Commit-Queue: Ulan Degenbaev <ulan@chromium.org> Cr-Commit-Position: refs/heads/master@{#70445} Refs: v8/v8@3176bfd PR-URL: #35612 Refs: #35559 Reviewed-By: Michaël Zasso <targos@protonmail.com> Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: Richard Lau <rlau@redhat.com> Reviewed-By: Jiawen Geng <technicalcute@gmail.com> Reviewed-By: Rich Trott <rtrott@gmail.com> Reviewed-By: Gireesh Punathil <gpunathi@in.ibm.com> Reviewed-By: Gerhard Stöbich <deb2001-github@yahoo.de>
Original commit message: [heap-profiler] Fix crash when a snapshot deleted while taking one Fix a crash/hang that occurred when deleting a snapshot during the GC that is part of taking another one. Specifically, when deleting the only other snapshot in such a situation, the `v8::HeapSnapshot::Delete()` method sees that there is only one (complete) snapshot at that point, and decides that it is okay to perform “delete all snapshots” instead of just deleting the requested one. That resets the internal string lookup table of the heap profiler, but the new snapshot that is currently in progress still holds references to the old string lookup table, leading to a use-after-free segfault or infinite loop. Fix this by guarding against resetting the string table while another heap snapshot is being taken, and add a test that would crash before this fix. This can be triggered in Node.js by repeatedly calling `v8.getHeapSnapshot()`, which provides heap snapshots as weakly held host objects. Change-Id: If9ac3728bf79114000982f1e7bb05e8034299e3c Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2464823 Reviewed-by: Ulan Degenbaev <ulan@chromium.org> Commit-Queue: Ulan Degenbaev <ulan@chromium.org> Cr-Commit-Position: refs/heads/master@{#70445} Refs: v8/v8@3176bfd PR-URL: nodejs#35612 Refs: nodejs#35559 Reviewed-By: Michaël Zasso <targos@protonmail.com> Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: Richard Lau <rlau@redhat.com> Reviewed-By: Jiawen Geng <technicalcute@gmail.com> Reviewed-By: Rich Trott <rtrott@gmail.com> Reviewed-By: Gireesh Punathil <gpunathi@in.ibm.com> Reviewed-By: Gerhard Stöbich <deb2001-github@yahoo.de>
Original commit message: [heap-profiler] Fix crash when a snapshot deleted while taking one Fix a crash/hang that occurred when deleting a snapshot during the GC that is part of taking another one. Specifically, when deleting the only other snapshot in such a situation, the `v8::HeapSnapshot::Delete()` method sees that there is only one (complete) snapshot at that point, and decides that it is okay to perform “delete all snapshots” instead of just deleting the requested one. That resets the internal string lookup table of the heap profiler, but the new snapshot that is currently in progress still holds references to the old string lookup table, leading to a use-after-free segfault or infinite loop. Fix this by guarding against resetting the string table while another heap snapshot is being taken, and add a test that would crash before this fix. This can be triggered in Node.js by repeatedly calling `v8.getHeapSnapshot()`, which provides heap snapshots as weakly held host objects. Change-Id: If9ac3728bf79114000982f1e7bb05e8034299e3c Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2464823 Reviewed-by: Ulan Degenbaev <ulan@chromium.org> Commit-Queue: Ulan Degenbaev <ulan@chromium.org> Cr-Commit-Position: refs/heads/master@{#70445} Refs: v8/v8@3176bfd PR-URL: nodejs#35612 Refs: nodejs#35559 Reviewed-By: Michaël Zasso <targos@protonmail.com> Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: Richard Lau <rlau@redhat.com> Reviewed-By: Jiawen Geng <technicalcute@gmail.com> Reviewed-By: Rich Trott <rtrott@gmail.com> Reviewed-By: Gireesh Punathil <gpunathi@in.ibm.com> Reviewed-By: Gerhard Stöbich <deb2001-github@yahoo.de>
Original commit message: [heap-profiler] Fix crash when a snapshot deleted while taking one Fix a crash/hang that occurred when deleting a snapshot during the GC that is part of taking another one. Specifically, when deleting the only other snapshot in such a situation, the `v8::HeapSnapshot::Delete()` method sees that there is only one (complete) snapshot at that point, and decides that it is okay to perform “delete all snapshots” instead of just deleting the requested one. That resets the internal string lookup table of the heap profiler, but the new snapshot that is currently in progress still holds references to the old string lookup table, leading to a use-after-free segfault or infinite loop. Fix this by guarding against resetting the string table while another heap snapshot is being taken, and add a test that would crash before this fix. This can be triggered in Node.js by repeatedly calling `v8.getHeapSnapshot()`, which provides heap snapshots as weakly held host objects. Change-Id: If9ac3728bf79114000982f1e7bb05e8034299e3c Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2464823 Reviewed-by: Ulan Degenbaev <ulan@chromium.org> Commit-Queue: Ulan Degenbaev <ulan@chromium.org> Cr-Commit-Position: refs/heads/master@{#70445} Refs: v8/v8@3176bfd PR-URL: nodejs#35612 Refs: nodejs#35559 Reviewed-By: Michaël Zasso <targos@protonmail.com> Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: Richard Lau <rlau@redhat.com> Reviewed-By: Jiawen Geng <technicalcute@gmail.com> Reviewed-By: Rich Trott <rtrott@gmail.com> Reviewed-By: Gireesh Punathil <gpunathi@in.ibm.com> Reviewed-By: Gerhard Stöbich <deb2001-github@yahoo.de>
Original commit message: [heap-profiler] Fix crash when a snapshot deleted while taking one Fix a crash/hang that occurred when deleting a snapshot during the GC that is part of taking another one. Specifically, when deleting the only other snapshot in such a situation, the `v8::HeapSnapshot::Delete()` method sees that there is only one (complete) snapshot at that point, and decides that it is okay to perform “delete all snapshots” instead of just deleting the requested one. That resets the internal string lookup table of the heap profiler, but the new snapshot that is currently in progress still holds references to the old string lookup table, leading to a use-after-free segfault or infinite loop. Fix this by guarding against resetting the string table while another heap snapshot is being taken, and add a test that would crash before this fix. This can be triggered in Node.js by repeatedly calling `v8.getHeapSnapshot()`, which provides heap snapshots as weakly held host objects. Change-Id: If9ac3728bf79114000982f1e7bb05e8034299e3c Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2464823 Reviewed-by: Ulan Degenbaev <ulan@chromium.org> Commit-Queue: Ulan Degenbaev <ulan@chromium.org> Cr-Commit-Position: refs/heads/master@{#70445} Refs: v8/v8@3176bfd PR-URL: #35612 Refs: #35559 Reviewed-By: Michaël Zasso <targos@protonmail.com> Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: Richard Lau <rlau@redhat.com> Reviewed-By: Jiawen Geng <technicalcute@gmail.com> Reviewed-By: Rich Trott <rtrott@gmail.com> Reviewed-By: Gireesh Punathil <gpunathi@in.ibm.com> Reviewed-By: Gerhard Stöbich <deb2001-github@yahoo.de>
Original commit message: [heap-profiler] Fix crash when a snapshot deleted while taking one Fix a crash/hang that occurred when deleting a snapshot during the GC that is part of taking another one. Specifically, when deleting the only other snapshot in such a situation, the `v8::HeapSnapshot::Delete()` method sees that there is only one (complete) snapshot at that point, and decides that it is okay to perform “delete all snapshots” instead of just deleting the requested one. That resets the internal string lookup table of the heap profiler, but the new snapshot that is currently in progress still holds references to the old string lookup table, leading to a use-after-free segfault or infinite loop. Fix this by guarding against resetting the string table while another heap snapshot is being taken, and add a test that would crash before this fix. This can be triggered in Node.js by repeatedly calling `v8.getHeapSnapshot()`, which provides heap snapshots as weakly held host objects. Change-Id: If9ac3728bf79114000982f1e7bb05e8034299e3c Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2464823 Reviewed-by: Ulan Degenbaev <ulan@chromium.org> Commit-Queue: Ulan Degenbaev <ulan@chromium.org> Cr-Commit-Position: refs/heads/master@{#70445} Refs: v8/v8@3176bfd PR-URL: nodejs#35612 Refs: nodejs#35559 Reviewed-By: Michaël Zasso <targos@protonmail.com> Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: Richard Lau <rlau@redhat.com> Reviewed-By: Jiawen Geng <technicalcute@gmail.com> Reviewed-By: Rich Trott <rtrott@gmail.com> Reviewed-By: Gireesh Punathil <gpunathi@in.ibm.com> Reviewed-By: Gerhard Stöbich <deb2001-github@yahoo.de>
Fixes: nodejs#35559 PR-URL: nodejs#35612 Reviewed-By: Michaël Zasso <targos@protonmail.com> Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: Richard Lau <rlau@redhat.com> Reviewed-By: Jiawen Geng <technicalcute@gmail.com> Reviewed-By: Rich Trott <rtrott@gmail.com> Reviewed-By: Gireesh Punathil <gpunathi@in.ibm.com> Reviewed-By: Gerhard Stöbich <deb2001-github@yahoo.de>
Has this landed in a version of node yet? I'm trying to debug using chrome's "Allocation instrumentation on timeline" profile and it crashes with a segfault which looks similar to this. If it's not please let me know and I'll open my own issue. I commented here as it's the only thing I could find on google when searching for this.
|
@OmgImAlexis The stack trace looks very different so I’d open a new issue. (And yes, the fix for this has been released on the latest 15.x and 14.x branches, but it’s unrelated to the Chrome feature anyway.) |
@addaleax Hi, I met the same issue as @OmgImAlexis , can you post the link of the new issue you mentioned above? thanks |
What steps will reproduce the bug?
What is the expected behavior?
no crash, the API returns streams
Additional information
I can easily recreate it, so any more info can be collected if needed.
The text was updated successfully, but these errors were encountered: