Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

src: fix AliasedBuffer memory attribution in heap snapshots #46817

Closed
wants to merge 5 commits into from

Conversation

joyeecheung
Copy link
Member

src: make util.h self-containted

Before it depended on util-inl.h. Fix it by moving
MaybeStackBuffer::AllocateSufficientStorage() into
util-inl.h

src: move AliasedBuffer implementation to -inl.h

Drive-by: Replace the SFINAE with a static_assert because we don't
have (or need) an implementation for non-scalar AliasedBufferBase
otherwise. Add forward declarations to memory_tracker.h now that
aliased-buffer.h no longer includes util-inl.h.

src: fix AliasedBuffer memory attribution in heap snapshots

Before the AliasedBuffers were represented solely by the TypedArrays
event though they also retain additional memory themselves.
Make the accounting more accurate by implementing MemoryRetainer in
AliasedBuffer.

Before:

Screenshot 2023-02-24 at 16 06 00

After:

Screenshot 2023-02-24 at 16 05 51

Before it depended on util-inl.h. Fix it by moving
MaybeStackBuffer::AllocateSufficientStorage() into
util-inl.h
Drive-by: Replace the SFINAE with a static_assert because we don't
have (or need) an implementation for non-scalar AliasedBufferBase
otherwise. Add forward declarations to memory_tracker.h now that
aliased-buffer.h no longer includes util-inl.h.
Before the AliasedBuffers were represented solely by the TypedArrays
event though they also retain additional memory themselves.
Make the accounting more accurate by implementing MemoryRetainer in
AliasedBuffer.
@nodejs-github-bot
Copy link
Collaborator

Review requested:

  • @nodejs/gyp
  • @nodejs/http2
  • @nodejs/net

@nodejs-github-bot nodejs-github-bot added c++ Issues and PRs that require attention from people who are familiar with C++. lib / src Issues and PRs related to general changes in the lib or src directory. needs-ci PRs that need a full CI run. labels Feb 24, 2023
@nodejs-github-bot
Copy link
Collaborator

@nodejs-github-bot
Copy link
Collaborator

@nodejs-github-bot
Copy link
Collaborator

@nodejs-github-bot
Copy link
Collaborator

@joyeecheung
Copy link
Member Author

Fixed linter error

Copy link
Member

@mcollina mcollina left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@nodejs-github-bot
Copy link
Collaborator

joyeecheung added a commit that referenced this pull request Mar 1, 2023
Before it depended on util-inl.h. Fix it by moving
MaybeStackBuffer::AllocateSufficientStorage() into
util-inl.h

PR-URL: #46817
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
joyeecheung added a commit that referenced this pull request Mar 1, 2023
Drive-by: Replace the SFINAE with a static_assert because we don't
have (or need) an implementation for non-scalar AliasedBufferBase
otherwise. Add forward declarations to memory_tracker.h now that
aliased-buffer.h no longer includes util-inl.h.

PR-URL: #46817
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
joyeecheung added a commit that referenced this pull request Mar 1, 2023
Before the AliasedBuffers were represented solely by the TypedArrays
event though they also retain additional memory themselves.
Make the accounting more accurate by implementing MemoryRetainer in
AliasedBuffer.

PR-URL: #46817
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
@joyeecheung
Copy link
Member Author

Landed in d0608c2...3646a66

@joyeecheung joyeecheung closed this Mar 1, 2023
kvakil added a commit to kvakil/node that referenced this pull request Mar 13, 2023
Compiling with clang 14 on Ubuntu was failing with the following error:

```bash
$ CXX=clang++ CC=clang ./configure --ninja
$ make
<snip>
[7/14] LINK cctest
FAILED: cctest
clang++ <snip>
/usr/bin/ld: obj/src/inspector/libnode.node_string.o: in function `node::inspector::protocol::StringUtil::StringViewToUtf8[abi:cxx11](v8_inspector::StringView)':
node_string.cc:(.text+0x2ce): undefined reference to `node::MaybeStackBuffer<char, 1024ul>::AllocateSufficientStorage(unsigned long)'
/usr/bin/ld: obj/src/inspector/libnode.node_string.o: in function `node::inspector::protocol::StringUtil::fromUTF16[abi:cxx11](unsigned short const*, unsigned long)':
node_string.cc:(.text+0xad9): undefined reference to `node::MaybeStackBuffer<char, 1024ul>::AllocateSufficientStorage(unsigned long)'
clang: error: linker command failed with exit code 1 (use -v to see invocation)
```

I bisected it to the referenced PR. This change includes the header file
which defines `node::MaybeStackBuffer::AllocateSufficientStorage`.

Refs: nodejs#46817
kvakil added a commit to kvakil/node that referenced this pull request Mar 13, 2023
Compiling with clang 14 on Ubuntu was failing with a linker error that
`node::MaybeStackBuffer::AllocateSufficientStorage` was undefined in
`src/inspector/node_string.cc`. I bisected it to the referenced PR.
Include `util-inl.h` which defines
`node::MaybeStackBuffer::AllocateSufficientStorage`.

Refs: nodejs#46817
targos pushed a commit that referenced this pull request Mar 13, 2023
Before it depended on util-inl.h. Fix it by moving
MaybeStackBuffer::AllocateSufficientStorage() into
util-inl.h

PR-URL: #46817
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
targos pushed a commit that referenced this pull request Mar 13, 2023
Drive-by: Replace the SFINAE with a static_assert because we don't
have (or need) an implementation for non-scalar AliasedBufferBase
otherwise. Add forward declarations to memory_tracker.h now that
aliased-buffer.h no longer includes util-inl.h.

PR-URL: #46817
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
targos pushed a commit that referenced this pull request Mar 13, 2023
Before the AliasedBuffers were represented solely by the TypedArrays
event though they also retain additional memory themselves.
Make the accounting more accurate by implementing MemoryRetainer in
AliasedBuffer.

PR-URL: #46817
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
targos pushed a commit that referenced this pull request Mar 14, 2023
Before it depended on util-inl.h. Fix it by moving
MaybeStackBuffer::AllocateSufficientStorage() into
util-inl.h

PR-URL: #46817
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
@targos
Copy link
Member

targos commented Mar 14, 2023

Should land after the permission model to avoid conflicts

nodejs-github-bot pushed a commit that referenced this pull request Mar 15, 2023
Compiling with clang 14 on Ubuntu was failing with a linker error that
`node::MaybeStackBuffer::AllocateSufficientStorage` was undefined in
`src/inspector/node_string.cc`. I bisected it to the referenced PR.
Include `util-inl.h` which defines
`node::MaybeStackBuffer::AllocateSufficientStorage`.

Refs: #46817
PR-URL: #47057
Reviewed-By: Mohammed Keyvanzadeh <mohammadkeyvanzade94@gmail.com>
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: Darshan Sen <raisinten@gmail.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Tobias Nießen <tniessen@tnie.de>
targos pushed a commit that referenced this pull request Mar 18, 2023
Compiling with clang 14 on Ubuntu was failing with a linker error that
`node::MaybeStackBuffer::AllocateSufficientStorage` was undefined in
`src/inspector/node_string.cc`. I bisected it to the referenced PR.
Include `util-inl.h` which defines
`node::MaybeStackBuffer::AllocateSufficientStorage`.

Refs: #46817
PR-URL: #47057
Reviewed-By: Mohammed Keyvanzadeh <mohammadkeyvanzade94@gmail.com>
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: Darshan Sen <raisinten@gmail.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Tobias Nießen <tniessen@tnie.de>
danielleadams pushed a commit that referenced this pull request Apr 11, 2023
Before it depended on util-inl.h. Fix it by moving
MaybeStackBuffer::AllocateSufficientStorage() into
util-inl.h

PR-URL: #46817
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
@danielleadams danielleadams added the backport-requested-v18.x PRs awaiting manual backport to the v18.x-staging branch. label Jul 2, 2023
danielleadams pushed a commit that referenced this pull request Jul 6, 2023
Compiling with clang 14 on Ubuntu was failing with a linker error that
`node::MaybeStackBuffer::AllocateSufficientStorage` was undefined in
`src/inspector/node_string.cc`. I bisected it to the referenced PR.
Include `util-inl.h` which defines
`node::MaybeStackBuffer::AllocateSufficientStorage`.

Refs: #46817
PR-URL: #47057
Reviewed-By: Mohammed Keyvanzadeh <mohammadkeyvanzade94@gmail.com>
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: Darshan Sen <raisinten@gmail.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Tobias Nießen <tniessen@tnie.de>
targos pushed a commit that referenced this pull request Nov 10, 2023
Drive-by: Replace the SFINAE with a static_assert because we don't
have (or need) an implementation for non-scalar AliasedBufferBase
otherwise. Add forward declarations to memory_tracker.h now that
aliased-buffer.h no longer includes util-inl.h.

PR-URL: #46817
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
targos pushed a commit that referenced this pull request Nov 10, 2023
Before the AliasedBuffers were represented solely by the TypedArrays
event though they also retain additional memory themselves.
Make the accounting more accurate by implementing MemoryRetainer in
AliasedBuffer.

PR-URL: #46817
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
sercher added a commit to sercher/graaljs that referenced this pull request Apr 25, 2024
Drive-by: Replace the SFINAE with a static_assert because we don't
have (or need) an implementation for non-scalar AliasedBufferBase
otherwise. Add forward declarations to memory_tracker.h now that
aliased-buffer.h no longer includes util-inl.h.

PR-URL: nodejs/node#46817
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
sercher added a commit to sercher/graaljs that referenced this pull request Apr 25, 2024
Before the AliasedBuffers were represented solely by the TypedArrays
event though they also retain additional memory themselves.
Make the accounting more accurate by implementing MemoryRetainer in
AliasedBuffer.

PR-URL: nodejs/node#46817
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
sercher added a commit to sercher/graaljs that referenced this pull request Apr 25, 2024
Drive-by: Replace the SFINAE with a static_assert because we don't
have (or need) an implementation for non-scalar AliasedBufferBase
otherwise. Add forward declarations to memory_tracker.h now that
aliased-buffer.h no longer includes util-inl.h.

PR-URL: nodejs/node#46817
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
sercher added a commit to sercher/graaljs that referenced this pull request Apr 25, 2024
Before the AliasedBuffers were represented solely by the TypedArrays
event though they also retain additional memory themselves.
Make the accounting more accurate by implementing MemoryRetainer in
AliasedBuffer.

PR-URL: nodejs/node#46817
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport-requested-v18.x PRs awaiting manual backport to the v18.x-staging branch. c++ Issues and PRs that require attention from people who are familiar with C++. lib / src Issues and PRs related to general changes in the lib or src directory. needs-ci PRs that need a full CI run.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants