-
Notifications
You must be signed in to change notification settings - Fork 105
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
A memory leak in sort #116
Comments
I eliminated a leak (didn't do much testing, though) in the commit biod/BioD@5d94926 |
Thanks for the fast response, I'll do a couple of test-runs and report back. |
Unfortunately, looks like there is still an issue. Seems like it's a little slower (I may be imagining it though), but about 2 hours into the same process described above, sort is taking 35GB of RAM |
Thanks, I'll run it under Valgrind then. |
Now I did a bit of testing on a 7GB file, and the tool seems to respect the memory limit (I also checked that 1&2 commits back it didn't). |
Artem, You may want to make a new sambamba release from the latest sources, just to ensure that people have access to this fix. The current published binary (0.5.1) suffers from the memory leak, and in quite a few scenarios (exomes, whole genomes, sorting, merging) it runs out of memory on AWS instances. I've rebuilt the latest version from source and it fixes the bug. But given that Ubuntu 14 doesn't come with a package for ldc, it may not be trivial for others. Thanks again for making sambamba. |
@asimenos thanks for the reminder, I've uploaded new binaries |
I'm using 0.5.0-alpha to sort a large (100 GB) BAM file (BWA-MEM | Samblaster | Sambamba view | Sambamba sort).
There appears to be a fairly slow but steady memory leak in the sort tool. I'm running it with -t 32 -l 5 -m 28 GiB, several hours into the process I see that sort has climbed to 36.9G. (View is started with -l 0 if that is of any consequence).
The text was updated successfully, but these errors were encountered: