Segmentation Fault After Closing SequenceReader #88
Comments
|
Thanks for reporting this @jessrosenfield. What version of Atropos are you using? |
|
I'm using 1.1.21 via
https://github.com/jdidion/atropos/archive/1.1.21.tar.gz
…On Mon, Dec 23, 2019 at 7:31 PM John Didion ***@***.***> wrote:
Thanks for reporting this @jessrosenfield
<https://github.com/jessrosenfield>. What version of Atropos are you
using?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#88?email_source=notifications&email_token=AALXJPLMQDZT5QO5TIQREXTQ2F7BNA5CNFSM4J6ZA4M2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEHSMLUA#issuecomment-568640976>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AALXJPPLGRESXBUYALDEBM3Q2F7BNANCNFSM4J6ZA4MQ>
.
|
|
Thanks @jessrosenfield. Do you know if it fails on the same line of code every time? That line is simply setting a variable to My suspicion is that this is instead an out-of-memory error. Since you said you are running this in a Docker container, can you check that the Docker engine is configured to allocate a sufficient amount of memory to the container? Try 32 GB - that should be overkill, but once you figure out whether that is the fix you can start scaling back, perhaps using a profiler to determine the peak usage. |
|
Sorry for the delay, I've been on vacation. This was already run with
excessive memory (64GB) allocated on the ec2 instance and the docker
process was only using a small fraction of that memory. When I return I
will do some profiling to check that the docker container has proper access
to the memory on the instance and do additional profiling of memory used
over time.
…On Wed, Dec 25, 2019, 13:51 John Didion ***@***.***> wrote:
Thanks @jessrosenfield <https://github.com/jessrosenfield>. Do you know
if it fails on the same line of code every time? That line is simply
setting a variable to None and is only ever called by a single thread, so
I can't see how it would cause a SegmentationFault.
My suspicion is that this is instead an out-of-memory error. Since you
said you are running this in a Docker container, can you check that the
Docker engine is configured to allocate a sufficient amount of memory to
the container? Try 32 GB - that should be overkill, but once you figure out
whether that is the fix you can start scaling back, perhaps using a
profiler to determine the peak usage.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#88?email_source=notifications&email_token=AALXJPNQIY2XCGXGD5NO343Q2O2S7A5CNFSM4J6ZA4M2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEHUSBAA#issuecomment-568926336>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AALXJPKGHYOGFL7VV44RVZ3Q2O2S7ANCNFSM4J6ZA4MQ>
.
|
|
I'll also check that the trace is consistent.
On Mon, Dec 30, 2019, 12:38 Jessica Rosenfield <rosenfield.jess@gmail.com>
wrote:
… Sorry for the delay, I've been on vacation. This was already run with
excessive memory (64GB) allocated on the ec2 instance and the docker
process was only using a small fraction of that memory. When I return I
will do some profiling to check that the docker container has proper access
to the memory on the instance and do additional profiling of memory used
over time.
On Wed, Dec 25, 2019, 13:51 John Didion ***@***.***> wrote:
> Thanks @jessrosenfield <https://github.com/jessrosenfield>. Do you know
> if it fails on the same line of code every time? That line is simply
> setting a variable to None and is only ever called by a single thread,
> so I can't see how it would cause a SegmentationFault.
>
> My suspicion is that this is instead an out-of-memory error. Since you
> said you are running this in a Docker container, can you check that the
> Docker engine is configured to allocate a sufficient amount of memory to
> the container? Try 32 GB - that should be overkill, but once you figure out
> whether that is the fix you can start scaling back, perhaps using a
> profiler to determine the peak usage.
>
> —
> You are receiving this because you were mentioned.
> Reply to this email directly, view it on GitHub
> <#88?email_source=notifications&email_token=AALXJPNQIY2XCGXGD5NO343Q2O2S7A5CNFSM4J6ZA4M2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEHUSBAA#issuecomment-568926336>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AALXJPKGHYOGFL7VV44RVZ3Q2O2S7ANCNFSM4J6ZA4MQ>
> .
>
|
|
I was able to inspect the stack trace in the core dump and was able to trace it to a bug in python 3.5.2 (default on ubuntu 16.04). This bug has been fixed from python 3.5.3 onwards. This issue can be closed. |
|
Thank you @siddharthab! I will make a note of this on the readme so that others avoid 3.5.2. Starting from atropos 2.0, python 3.6 will be required at a minimum, so this won't be an issue anymore. |
I cannot reliably reproduce this error, but I have been getting segmentation faults every now and then on single-end adapter-match and paired-end insert-match trimming. Upon adding
faulthandler.enable()toatropos/commands/__init__.pyI was able to get this output after a couple reruns of the following command (using a self-built docker image w/ a ubuntu 16.04 base on an ec2 instance with 32 cpus). I hope this information is still useful to have on hand despite the absence of the docker image and input files used (I've left off the-o,-p,-pe1, and-pe2args).The trace indicates the error is occurring here: https://github.com/jdidion/atropos/blob/1.1/atropos/io/seqio.py#L93
The text was updated successfully, but these errors were encountered: