This repository has been archived by the owner on Nov 17, 2023. It is now read-only.
Fix memory leak for size-zero ndarray #14365
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
@mxnet-label-bot update [pr-work-in-progress] |
marcoabreu
added
pr-awaiting-review
wkcn
approved these changes
Mar 8, 2019
access2rohit
approved these changes
Mar 8, 2019
|
@yuxihu Can you look at the failing checks. LGTM |
|
Yes, I am looking into the test failures. |
|
Very nice catch @yuxihu ! |
yuxihu
force-pushed
the
mem_leak
branch
4 times, most recently
from
2248647
to
142ddcf
Compare
|
Left few nit picky comments |
|
@eric-haibin-lin please help review. @mxnet-label-bot update [pr-awaiting-review] |
marcoabreu
added
pr-awaiting-review
access2rohit
approved these changes
Mar 12, 2019
apeforest
reviewed
Mar 13, 2019
wkcn
added
pr-awaiting-merge
Yes. I am not sure the reason behind the logic so I do not change it in this PR. We also use aligned alloc in CPU which allocates 16/64 bytes when size is 0. |
|
@eric-haibin-lin It is ready for final review. |
yuxihu
added a commit
to yuxihu/incubator-mxnet
that referenced
this pull request
Mar 20, 2019
This reverts commit 3ab1dec.
TaoLv
pushed a commit
that referenced
this pull request
Mar 20, 2019
vdantu
pushed a commit
to vdantu/incubator-mxnet
that referenced
this pull request
Mar 31, 2019
* free memory for size zero storage handle * skip adding nullptr into GPU memory pool * set context for aux handle * set context for aux handle once it is created
ZhennanQin
pushed a commit
to ZhennanQin/incubator-mxnet
that referenced
this pull request
Apr 3, 2019
* free memory for size zero storage handle * skip adding nullptr into GPU memory pool * set context for aux handle * set context for aux handle once it is created
ZhennanQin
pushed a commit
to ZhennanQin/incubator-mxnet
that referenced
this pull request
Apr 3, 2019
nswamy
pushed a commit
that referenced
this pull request
Apr 5, 2019
* free memory for size zero storage handle * skip adding nullptr into GPU memory pool * set context for aux handle * set context for aux handle once it is created
nswamy
pushed a commit
that referenced
this pull request
Apr 5, 2019
haohuanw
pushed a commit
to haohuanw/incubator-mxnet
that referenced
this pull request
Jun 23, 2019
* free memory for size zero storage handle * skip adding nullptr into GPU memory pool * set context for aux handle * set context for aux handle once it is created
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fixes #13951 Fixes #14358
For size-zero ndarray (e.g. mx.nd.array([]), mx.nd.ones(0)), the storage handle size is 0. Currently we only free handles which size is larger than 0. This leads to memory leak for size-zero ndarray.
In this PR, we remove the check on storage handle size which was used to decide if we need to free a storage handle. After relaxing the check, we need to make sure nullptr is not reused in pooled storage manager and the context for aux handle is correctly set for sparse ndarray.
With this PR, the memory leak issues mentioned above are fixed.