Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix/add mem 4 #1965

Merged
merged 3 commits into from
Nov 19, 2015
Merged

Fix/add mem 4 #1965

merged 3 commits into from
Nov 19, 2015

Conversation

whyrusleeping
Copy link
Member

putting #1954 on top of 0.4.0

License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
@rht
Copy link
Contributor

rht commented Nov 15, 2015

(Only maximum resident size can be retrieved from time)

dev0.4.0, 1KB each
1mem

fix/add-mem-4, 1KB each
memory

@whyrusleeping whyrusleeping mentioned this pull request Nov 16, 2015
2 tasks
if err == dag.ErrNotFound {
nd, err = root.GetLinkedNode(ctx, e.src, path[0])
}

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

rm this space as it separates the err check clause

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i added the space purposely because i felt it made it more readable, do these things and then in either case check the error.

@whyrusleeping
Copy link
Member Author

@rht the memory leak was really only noticeable when you started adding hundreds of thousands of files.

@rht
Copy link
Contributor

rht commented Nov 17, 2015

@whyrusleeping a proof of your statement
no-sync + fix-add-mem-4, each 10B up to 10k files (I used no-sync because sync was too slow, and even here, it took at least 2 hrs)
fix-add-4-mem

And here is without intermediate roots
no-sync + no-intermediate-root + no-chunk-channel + no-outputdagnode, each 10B up to 10k
memory

If you subtract the 2nd graph from the 1st, you get the memory growth of creating each intermediate roots.

Of all the perf PRs, I think the most effective one is no-intermediate-root, but I encountered an issue with a hash in the travis-ci/pr test (possibly related to the final hash created by InsertNodeAtPath).

} else {
err = nil // no longer an error case
} else if err == dag.ErrNotFound {
nd, err = root.GetLinkedNode(ctx, e.src, path[0])
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe write why there's two checks here. i get it after reading but useful

@jbenet
Copy link
Member

jbenet commented Nov 17, 2015

LGTM otherwise.

License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
@whyrusleeping
Copy link
Member Author

while @rht's PR makes the performance boost from this PR negligible, its still good to have for anything else that may be using the DagEditor in the future.

@whyrusleeping
Copy link
Member Author

merging.

whyrusleeping added a commit that referenced this pull request Nov 19, 2015
@whyrusleeping whyrusleeping merged commit 5f8cfc9 into dev0.4.0 Nov 19, 2015
@jbenet jbenet removed the status/in-progress In progress label Nov 19, 2015
@rht rht mentioned this pull request Nov 24, 2015
@ghost ghost deleted the fix/add-mem-4 branch January 22, 2017 04:03
SgtPooki added a commit that referenced this pull request Dec 2, 2023
## [4.2.0](ipfs/ipfs-webui@v4.1.1...v4.2.0) (2023-12-02)


 CID `bafybeidf7cpkwsjkq6xs3r6fbbxghbugilx3jtezbza7gua3k5wjixpmba`

 --- 



### Features

* peers table can be filtered ([#2181](ipfs/ipfs-webui#2181)) ([37b5880](ipfs/ipfs-webui@37b5880))
* **pinning:** add Scaleway pinning service ([#2132](ipfs/ipfs-webui#2132)) ([0cc3b04](ipfs/ipfs-webui@0cc3b04))


### Bug Fixes

* error loading scaleway template URL ([7a3388e](ipfs/ipfs-webui@7a3388e))
* replace estuary with web3 for pinning ([#2182](ipfs/ipfs-webui#2182)) ([e2fc7c8](ipfs/ipfs-webui@e2fc7c8))
* status page supports webrtc ([#2180](ipfs/ipfs-webui#2180)) ([ff75d4e](ipfs/ipfs-webui@ff75d4e)), closes [#1965](ipfs/ipfs-webui#1965)
* switch to new dnslink updater ([#2175](ipfs/ipfs-webui#2175)) ([a63f48a](ipfs/ipfs-webui@a63f48a))


### Trivial Changes

* Pull transifex translations ([#2162](ipfs/ipfs-webui#2162)) ([c0f8f54](ipfs/ipfs-webui@c0f8f54))
* pull transifex translations ([#2177](ipfs/ipfs-webui#2177)) ([7f377ff](ipfs/ipfs-webui@7f377ff))
* update release transifex links ([74cce69](ipfs/ipfs-webui@74cce69))
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants