Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Bogart Failed ?? #546

Closed
caonetto opened this issue Jul 7, 2017 · 18 comments
Closed

Bogart Failed ?? #546

caonetto opened this issue Jul 7, 2017 · 18 comments

Comments

@caonetto
Copy link

caonetto commented Jul 7, 2017 •
edited by brianwalenz
Loading

Hi, I keep on getting this error when running Canu.

-- Running jobs.  First attempt out of 2.
----------------------------------------
-- Starting 'bat' concurrent execution on Fri Jul  7 04:06:48 2017 with 328.325 GB free disk space (1 processes; 3 concurrently)

    cd unitigging/4-unitigger
    ./unitigger.sh 1 > ./unitigger.000001.out 2>&1

-- Finished on Fri Jul  7 04:06:49 2017 (1 second) with 328.325 GB free disk space
----------------------------------------
--
-- Bogart failed, retry
--
--
-- Running jobs.  Second attempt out of 2.
----------------------------------------
-- Starting 'bat' concurrent execution on Fri Jul  7 04:06:49 2017 with 328.325 GB free disk space (1 processes; 3 concurrently)

    cd unitigging/4-unitigger
    ./unitigger.sh 1 > ./unitigger.000001.out 2>&1

-- Finished on Fri Jul  7 04:06:49 2017 (lickety-split) with 328.325 GB free disk space
----------------------------------------
--
-- Bogart failed, tried 2 times, giving up.
--

ABORT:
ABORT: Canu snapshot v1.5 +125 changes (r8325 10163a78fc4ff1f774e74e6e63af187b69e0ab96)
ABORT: Don't panic, but a mostly harmless error occurred and Canu stopped.
ABORT: Try restarting.  If that doesn't work, ask for help.
ABORT:

Any ideas?
@brianwalenz
Copy link
Member

Yup, do a code update. That module is under active development.

I'm somewhat interested in what the error reported in unitigger.000001.out is too, though I expect it's fixed already.

@brianwalenz brianwalenz changed the title Bugar Failed ?? Bogart Failed ?? Jul 7, 2017
@caonetto
Copy link
Author

caonetto commented Jul 7, 2017

Thanks Brian!, what do you mean by doing a code update?
This is the content of unitigger.000001.out

Running job 1 based on command line options.
./unitigger.sh: 86: ./unitigger.sh: cannot create ../canu.ctgStore/seqDB.v001.sizes.txt: Directory nonexistent

Thanks!

@brianwalenz
Copy link
Member

The version string says you're not using a release version, so 'git pull' from within the canu/ directory will update the code to the latest version. Then 'make' to compile. Or just grab the latest code directly: http://canu.readthedocs.io/en/latest/

I see I need to update how logging is saved here. The error I'm after should be in 'unitigger.err'.

@caonetto
Copy link
Author

caonetto commented Jul 7, 2017 •
edited by brianwalenz
Loading

Hi Brian, attached is the content of unitigger.err.

==> PARAMETERS.

Resources:
  Memory                15 GB
  Compute Threads       4 (command line)

Lengths:
  Minimum read          0 bases
  Minimum overlap       500 bases

Overlap Error Rates:
  Graph                 0.045 (4.500%)
  Max                   0.045 (4.500%)

Deviations:
  Graph                 6.000
  Bubble                6.000
  Repeat                3.000

Edge Confusion:
  Absolute              2100
  Percent               200.0000

Unitig Construction:
  Minimum intersection  500 bases
  Maxiumum placements   2 positions

Debugging Enabled:
  (none)

==> LOADING AND FILTERING OVERLAPS.

ReadInfo()-- Using 9636 reads, no minimum read length used.

OverlapCache()-- limited to 15360MB memory (user supplied).
OverlapCache()--       0MB for read data.
OverlapCache()--       0MB for best edges.
OverlapCache()--       0MB for unitig layouts.
OverlapCache()--       0MB for tigs.
OverlapCache()--       0MB for id maps.
OverlapCache()--       0MB for error profiles.
OverlapCache()--       0MB for overlap cache pointers.
OverlapCache()--      48MB for overlap cache initial bucket.
OverlapCache()--      64MB for overlap cache thread data.
OverlapCache()--       0MB for number of overlaps per read.
OverlapCache()--       0MB for other processes.
OverlapCache()-- ---------
OverlapCache()--     112MB for data structures (sum of above).
OverlapCache()--   15247MB available for overlaps.

OverlapCache()-- Loading number of overlaps per read.
OverlapCache()--  Retain at least 1802 overlaps/read, based on 300.46x coverage.
OverlapCache()--  Initial guess at 103699 overlaps/read (maximum 137 overlaps/read).
OverlapCache()--  103699 overlaps/read - load all for    9636 reads, some for       0 reads -       161564 overlaps to load -    2MB

OverlapCache()-- minPer           = 1802 overlaps/reads
OverlapCache()-- maxPer           = 103699 overlaps/reads
OverlapCache()-- numBelow         = 9636 reads (all overlaps loaded)
OverlapCache()-- numEqual         = 0 reads (all overlaps loaded)
OverlapCache()-- numAbove         = 0 reads (some overlaps loaded)
OverlapCache()-- totalLoad        = 161564 overlaps (100.00%)

OverlapCache()-- availForOverlaps = 15247MB
OverlapCache()-- totalMemory      = 0MB for organization
OverlapCache()-- totalMemory      = 2MB for overlaps
OverlapCache()-- totalMemory      = 2MB used

OverlapCache()-- Loading: overlaps processed           27 (000.02%) loaded           27 (000.02%) droppeddupe            0 (000.00%)
OverlapCache()-- Loading: overlaps processed       161564 (100.00%) loaded       160496 (099.34%) droppeddupe            0 (000.00%)

BestOverlapGraph()-- allocating best edges (0MB)

BestOverlapGraph()-- finding initial best edges.

BestOverlapGraph()-- filtering suspicious reads.
BestOverlapGraph()-- marked 0 reads as suspicious.

BestOverlapGraph()-- filtering high error edges.

BestOverlapGraph()-- filtering reads with lopsided best edges.

BestOverlapGraph()-- filtering spur reads.
BestOverlapGraph()-- detected 1023 spur reads.

BestOverlapGraph()-- removing best edges for contained reads.

==> BUILDING GREEDY TIGS.

breakSingletonTigs()-- Removed 2309 singleton tigs; reads are now unplaced.
optimizePositions()-- Optimizing read positions for 2416 tigs, with 4 threads.
bogart: bogart/AS_BAT_Unitig.C:195: void Unitig::optimize_initPlace(uint32, optPos*, optPos*, bool, std::set<unsigned int>&, bool): Assertion `cnt > 0' failed.

Failed with 'Aborted'; backtrace (libbacktrace):
AS_UTL/AS_UTL_stackTrace.C::102 in _Z17AS_UTL_catchCrashiP9siginfo_tPv()
(null)::0 in (null)()
(null)::0 in (null)()
(null)::0 in (null)()
(null)::0 in (null)()
(null)::0 in (null)()
bogart/AS_BAT_Unitig.C::195 in _ZN6Unitig18optimize_initPlaceEjP6optPosS1_bRSt3setIjSt4lessIjESaIjEEb()
bogart/AS_BAT_Unitig.C::262 in _ZN6Unitig8optimizeEPKcS1_()
bogart/AS_BAT_TigVector.C::169 in _ZN9TigVector17optimizePositionsEPKcS1_._omp_fn.0()
(null)::0 in (null)()
(null)::0 in (null)()
(null)::0 in (null)()
(null)::0 in (null)()
Aborted

@caonetto
Copy link
Author

caonetto commented Jul 7, 2017 •
edited by brianwalenz
Loading

Hi Brian, tried again after updating Canu, still getting the same problem. I am in a Debian 9 machine.
This is the output:

Configure OEA for 1gb memory with batches of at most 25000 reads and (unlimited) bases.
Configured 1 jobs.

OEA job   1 from read         1 to read      9636 - 28.7 bases +  0.4 adjusts +  0.3 reads +  4.9 olaps +  8.0 fseq/rseq + 32.0 fadj/radj + 32.0 work + 256.0 misc = 362.3 MB

-- Finished stage 'overlapErrorAdjustmentConfigure', reset canuIteration.
--
-- Running jobs.  First attempt out of 2.
----------------------------------------
-- Starting 'oea' concurrent execution on Fri Jul  7 07:42:46 2017 with 328.326 GB free disk space (1 processes; 12 concurrently)

    cd unitigging/3-overlapErrorAdjustment
    ./oea.sh 1 > ./oea.000001.out 2>&1

-- Finished on Fri Jul  7 07:42:48 2017 (2 seconds) with 328.325 GB free disk space
----------------------------------------
-- Found 1 overlap error adjustment output files.
-- Finished stage 'overlapErrorAdjustmentCheck', reset canuIteration.
----------------------------------------
-- Starting command on Fri Jul  7 07:42:48 2017 with 328.325 GB free disk space

    cd unitigging/3-overlapErrorAdjustment
    /mnt/jiao/nanopore/canu/Linux-amd64/bin/ovStoreBuild \
      -G ../canu.gkpStore \
      -O ../canu.ovlStore \
      -evalues \
      -L ./oea.files \
    > ./oea.apply.err 2>&1

-- Finished on Fri Jul  7 07:42:48 2017 (lickety-split) with 328.325 GB free disk space
----------------------------------------
-- No report available.
-- Finished stage 'updateOverlapStore', reset canuIteration.
-- Finished stage 'unitig', reset canuIteration.
--
-- Running jobs.  First attempt out of 2.
----------------------------------------
-- Starting 'bat' concurrent execution on Fri Jul  7 07:42:48 2017 with 328.325 GB free disk space (1 processes; 3 concurrently)

    cd unitigging/4-unitigger
    ./unitigger.sh 1 > ./unitigger.000001.out 2>&1

-- Finished on Fri Jul  7 07:42:49 2017 (1 second) with 328.324 GB free disk space
----------------------------------------
--
-- Bogart failed, retry
--
--
-- Running jobs.  Second attempt out of 2.
----------------------------------------
-- Starting 'bat' concurrent execution on Fri Jul  7 07:42:49 2017 with 328.324 GB free disk space (1 processes; 3 concurrently)

    cd unitigging/4-unitigger
    ./unitigger.sh 1 > ./unitigger.000001.out 2>&1

-- Finished on Fri Jul  7 07:42:49 2017 (lickety-split) with 328.324 GB free disk space
----------------------------------------
--
-- Bogart failed, tried 2 times, giving up.
--

ABORT:
ABORT: Canu snapshot v1.5 +125 changes (r8325 10163a78fc4ff1f774e74e6e63af187b69e0ab96)
ABORT: Don't panic, but a mostly harmless error occurred and Canu stopped.
ABORT: Try restarting.  If that doesn't work, ask for help.
ABORT:

@brianwalenz
Copy link
Member

What's the error in unitigger.err?

@caonetto
Copy link
Author

caonetto commented Jul 7, 2017 •
edited by brianwalenz
Loading

Thanks again Brian,

This is the output.

==> PARAMETERS.

Resources:
  Memory                15 GB
  Compute Threads       4 (command line)

Lengths:
  Minimum read          0 bases
  Minimum overlap       500 bases

Overlap Error Rates:
  Graph                 0.045 (4.500%)
  Max                   0.045 (4.500%)

Deviations:
  Graph                 6.000
  Bubble                6.000
  Repeat                3.000

Edge Confusion:
  Absolute              2100
  Percent               200.0000

Unitig Construction:
  Minimum intersection  500 bases
  Maxiumum placements   2 positions

Debugging Enabled:
  (none)

==> LOADING AND FILTERING OVERLAPS.

ReadInfo()-- Using 9636 reads, no minimum read length used.

OverlapCache()-- limited to 15360MB memory (user supplied).
OverlapCache()--       0MB for read data.
OverlapCache()--       0MB for best edges.
OverlapCache()--       0MB for unitig layouts.
OverlapCache()--       0MB for tigs.
OverlapCache()--       0MB for id maps.
OverlapCache()--       0MB for error profiles.
OverlapCache()--       0MB for overlap cache pointers.
OverlapCache()--      48MB for overlap cache initial bucket.
OverlapCache()--      64MB for overlap cache thread data.
OverlapCache()--       0MB for number of overlaps per read.
OverlapCache()--       0MB for other processes.
OverlapCache()-- ---------
OverlapCache()--     112MB for data structures (sum of above).
OverlapCache()--   15247MB available for overlaps.

OverlapCache()-- Loading number of overlaps per read.
OverlapCache()--  Retain at least 1802 overlaps/read, based on 300.46x coverage.
OverlapCache()--  Initial guess at 103699 overlaps/read (maximum 137 overlaps/read).
OverlapCache()--  103699 overlaps/read - load all for    9636 reads, some for       0 reads -       161564 overlaps to load -    2MB

OverlapCache()-- minPer           = 1802 overlaps/reads
OverlapCache()-- maxPer           = 103699 overlaps/reads
OverlapCache()-- numBelow         = 9636 reads (all overlaps loaded)
OverlapCache()-- numEqual         = 0 reads (all overlaps loaded)
OverlapCache()-- numAbove         = 0 reads (some overlaps loaded)
OverlapCache()-- totalLoad        = 161564 overlaps (100.00%)

OverlapCache()-- availForOverlaps = 15247MB
OverlapCache()-- totalMemory      = 0MB for organization
OverlapCache()-- totalMemory      = 2MB for overlaps
OverlapCache()-- totalMemory      = 2MB used

OverlapCache()-- Loading: overlaps processed           27 (000.02%) loaded           27 (000.02%) droppeddupe            0 (000.00%)
OverlapCache()-- Loading: overlaps processed       161564 (100.00%) loaded       160496 (099.34%) droppeddupe            0 (000.00%)

BestOverlapGraph()-- allocating best edges (0MB)

BestOverlapGraph()-- finding initial best edges.

BestOverlapGraph()-- filtering suspicious reads.
BestOverlapGraph()-- marked 0 reads as suspicious.

BestOverlapGraph()-- filtering high error edges.

BestOverlapGraph()-- filtering reads with lopsided best edges.

BestOverlapGraph()-- filtering spur reads.
BestOverlapGraph()-- detected 1023 spur reads.

BestOverlapGraph()-- removing best edges for contained reads.

==> BUILDING GREEDY TIGS.

breakSingletonTigs()-- Removed 2309 singleton tigs; reads are now unplaced.
optimizePositions()-- Optimizing read positions for 2416 tigs, with 4 threads.
bogart: bogart/AS_BAT_Unitig.C:195: void Unitig::optimize_initPlace(uint32, optPos*, optPos*, bool, std::set<unsigned int>&, bool): Assertion `cnt > 0' failed.

Failed with 'Aborted'; backtrace (libbacktrace):
AS_UTL/AS_UTL_stackTrace.C::102 in _Z17AS_UTL_catchCrashiP9siginfo_tPv()
(null)::0 in (null)()
(null)::0 in (null)()
(null)::0 in (null)()
(null)::0 in (null)()
(null)::0 in (null)()
bogart/AS_BAT_Unitig.C::195 in _ZN6Unitig18optimize_initPlaceEjP6optPosS1_bRSt3setIjSt4lessIjESaIjEEb()
bogart/AS_BAT_Unitig.C::262 in _ZN6Unitig8optimizeEPKcS1_()
bogart/AS_BAT_TigVector.C::169 in _ZN9TigVector17optimizePositionsEPKcS1_._omp_fn.0()
(null)::0 in (null)()
(null)::0 in (null)()
(null)::0 in (null)()
(null)::0 in (null)()
Aborted

@skoren skoren mentioned this issue Jul 7, 2017
Closed
@brianwalenz
Copy link
Member

One more time: give the latest code a shot.

If this doesn't work, I'm not sure what else to try. I'd like to get a copy of your gkpStore, ovlStore, and 4-unitigger directories so I can see what it's doing.

You should be at:
Building snapshot v1.5 +133 changes (r8333 bb3bdfe)

@caonetto
Copy link
Author

caonetto commented Jul 8, 2017 via email •
edited by brianwalenz
Loading

@brianwalenz
Copy link
Member

Awesome, glad to hear!

@starsyi
Copy link

starsyi commented Sep 14, 2017

Hi, I also met this error in canu 1.6, the latest version. I want to ask how do you solve it?

-- Canu 1.6
--
-- CITATIONS
--
-- Koren S, Walenz BP, Berlin K, Miller JR, Phillippy AM.
-- Canu: scalable and accurate long-read assembly via adaptive k-mer weighting and repeat separation.
-- Genome Res. 2017 May;27(5):722-736.
-- http://doi.org/10.1101/gr.215087.116
-- 
-- Read and contig alignments during correction, consensus and GFA building use:
--   Šošic M, Šikic M.
--   Edlib: a C/C ++ library for fast, exact sequence alignment using edit distance.
--   Bioinformatics. 2017 May 1;33(9):1394-1395.
--   http://doi.org/10.1093/bioinformatics/btw753
-- 
-- Overlaps are generated using:
--   Berlin K, et al.
--   Assembling large genomes with single-molecule sequencing and locality-sensitive hashing.
--   Nat Biotechnol. 2015 Jun;33(6):623-30.
--   http://doi.org/10.1038/nbt.3238
-- 
--   Myers EW, et al.
--   A Whole-Genome Assembly of Drosophila.
--   Science. 2000 Mar 24;287(5461):2196-204.
--   http://doi.org/10.1126/science.287.5461.2196
-- 
--   Li H.
--   Minimap and miniasm: fast mapping and de novo assembly for noisy long sequences.
--   Bioinformatics. 2016 Jul 15;32(14):2103-10.
--   http://doi.org/10.1093/bioinformatics/btw152
-- 
-- Corrected read consensus sequences are generated using an algorithm derived from FALCON-sense:
--   Chin CS, et al.
--   Phased diploid genome assembly with single-molecule real-time sequencing.
--   Nat Methods. 2016 Dec;13(12):1050-1054.
--   http://doi.org/10.1038/nmeth.4035
-- 
-- Contig consensus sequences are generated using an algorithm derived from pbdagcon:
--   Chin CS, et al.
--   Nonhybrid, finished microbial genome assemblies from long-read SMRT sequencing data.
--   Nat Methods. 2013 Jun;10(6):563-9
--   http://doi.org/10.1038/nmeth.2474
-- 
-- CONFIGURE CANU
--
-- Detected Java(TM) Runtime Environment '1.8.0_65' (from '/usr/bin/java').
-- Detected gnuplot version '5.0 patchlevel 6' (from '/***/***/bin/gnuplot-5.0.6/bin/gnuplot') and image format 'svg'.
-- Detected 28 CPUs and 252 gigabytes of memory.
-- Detected PBS/Torque '4.2.7' with 'pbsnodes' binary in /***/bin/pbsnodes.
-- Detecting PBS/Torque resources.
-- 
-- Found   1 host  with 128 cores and 4031 GB memory under PBS/Torque control.
-- Found  40 hosts with  28 cores and  251 GB memory under PBS/Torque control.
--
--                     (tag)Threads
--            (tag)Memory         |
--        (tag)         |         |  algorithm
--        -------  ------  --------  -----------------------------
-- Grid:  meryl    125 GB   14 CPUs  (k-mer counting)
-- Grid:  cormhap   32 GB    4 CPUs  (overlap detection with mhap)
-- Grid:  obtovl    12 GB    4 CPUs  (overlap detection)
-- Grid:  utgovl    12 GB    4 CPUs  (overlap detection)
-- Grid:  cor       32 GB    4 CPUs  (read correction)
-- Grid:  ovb        4 GB    1 CPU   (overlap store bucketizer)
-- Grid:  ovs       32 GB    1 CPU   (overlap store sorting)
-- Grid:  red        8 GB    4 CPUs  (read error detection)
-- Grid:  oea        2 GB    1 CPU   (overlap error adjustment)
-- Grid:  bat       35 GB    4 CPUs  (contig construction)
-- Grid:  cns       83 GB    8 CPUs  (consensus)
-- Grid:  gfa       16 GB    4 CPUs  (GFA alignment and processing)
--
-- Found PacBio corrected reads in 'unitigging/***.gkpStore'.
--
-- Generating assembly '***' in '/***/canu-1.6.v1'
--
-- Parameters:
--
--  genomeSize        100000000
--
--  Overlap Generation Limits:
--    corOvlErrorRate 0.2400 ( 24.00%)
--    obtOvlErrorRate 0.0450 (  4.50%)
--    utgOvlErrorRate 0.0450 (  4.50%)
--
--  Overlap Processing Limits:
--    corErrorRate    0.3000 ( 30.00%)
--    obtErrorRate    0.0450 (  4.50%)
--    utgErrorRate    0.0450 (  4.50%)
--    cnsErrorRate    0.0750 (  7.50%)
--
--
-- BEGIN ASSEMBLY
--
--
-- Bogart failed, retry
--
--
-- Running jobs.  Second attempt out of 2.
--
-- 'unitigger.jobSubmit-01.sh' -> job 74303[].admin1 task 1.

-- Finished on Thu Sep 14 17:42:57 2017 (lickety-split) with 421009.481 GB free disk space
----------------------------------------

-- BEGIN ASSEMBLY
--
--
-- Bogart failed, tried 2 times, giving up.
--

ABORT:
ABORT: Canu 1.6
ABORT: Don't panic, but a mostly harmless error occurred and Canu stopped.
ABORT: Try restarting.  If that doesn't work, ask for help.
ABORT:

@brianwalenz this error still be unresloved in canu 1.6?

@brianwalenz
Copy link
Member

There seems to be a pipeline error when the first attempt to run 'bogart' fails. Canu submits the second attempt, but then immediately says it fails -- because it hasn't run yet.

What's in unitigger/4-unitigger/unitigger.err? If it ends with "Bye." then it finished successfully and a restart of canu should resume the assembly.

Does file unitigger/*.ctgStore/seqDB.v001.sizes.txt exist?

@brianwalenz brianwalenz reopened this Sep 14, 2017
@starsyi
Copy link

starsyi commented Sep 14, 2017

thanks!
my unitigger/4-unitigger/unitigger.err file like this:

==> PARAMETERS.

Resources:
  Memory                35 GB
  Compute Threads       4 (command line)

Lengths:
  Minimum read          0 bases
  Minimum overlap       500 bases

Overlap Error Rates:
  Graph                 0.045 (4.500%)
  Max                   0.045 (4.500%)

Deviations:
  Graph                 6.000
  Bubble                6.000
  Repeat                3.000

Edge Confusion:
  Absolute              2100
  Percent               200.0000

Unitig Construction:
  Minimum intersection  500 bases
  Maxiumum placements   2 positions

Debugging Enabled:
  (none)

==> LOADING AND FILTERING OVERLAPS.

ReadInfo()-- Using 1353091 reads, no minimum read length used.

OverlapCache()-- limited to 35840MB memory (user supplied).

OverlapCache()--      10MB for read data.
OverlapCache()--      51MB for best edges.
OverlapCache()--     134MB for tigs.
OverlapCache()--      36MB for tigs - read layouts.
OverlapCache()--      51MB for tigs - error profiles.
OverlapCache()--    8960MB for tigs - error profile overlaps.
OverlapCache()--       0MB for other processes.
OverlapCache()-- ---------
OverlapCache()--    9269MB for data structures (sum of above).
OverlapCache()-- ---------
OverlapCache()--      25MB for overlap store structure.
OverlapCache()--   26544MB for overlap data.
OverlapCache()-- ---------
OverlapCache()--   35840MB allowed.
OverlapCache()--
OverlapCache()-- Retain at least 73 overlaps/read, based on 36.51x coverage.
OverlapCache()-- Initial guess at 1285 overlaps/read.
OverlapCache()--
OverlapCache()-- Adjusting for sparse overlaps.
OverlapCache()--
OverlapCache()--               reads loading olaps          olaps               memory
OverlapCache()--   olaps/read       all      some          loaded                 free
OverlapCache()--   ----------   -------   -------     ----------- -------     --------
OverlapCache()--         1285   1324682     28409       174381135  79.65%      23883 MB
OverlapCache()--        56381   1353091         0       218939420 100.00%      23203 MB
OverlapCache()--
OverlapCache()-- Loading overlaps.
OverlapCache()--
OverlapCache()--          read from store           saved in cache
OverlapCache()--   ------------ ---------   ------------ ---------
OverlapCache()--       18799619 (008.59%)       18721773 (008.55%)
OverlapCache()--       34614832 (015.81%)       34476441 (015.75%)
OverlapCache()--       51065619 (023.32%)       50862386 (023.23%)
OverlapCache()--       67490939 (030.83%)       67217875 (030.70%)
OverlapCache()--       83302523 (038.05%)       82965994 (037.89%)
OverlapCache()--       98808238 (045.13%)       98418061 (044.95%)
OverlapCache()--      114400138 (052.25%)      113946360 (052.04%)
OverlapCache()--      129036112 (058.94%)      128527860 (058.70%)
OverlapCache()--      143854989 (065.71%)      143294818 (065.45%)
OverlapCache()--      160613501 (073.36%)      159978080 (073.07%)
OverlapCache()--      176965466 (080.83%)      176262573 (080.51%)
OverlapCache()--      193250778 (088.27%)      192473208 (087.91%)
OverlapCache()--      210237947 (096.03%)      209371479 (095.63%)
OverlapCache()--   ------------ ---------   ------------ ---------
OverlapCache()--      218939420 (100.00%)      218025209 (099.58%)
OverlapCache()--
OverlapCache()-- Ignored 604010 duplicate overlaps.
OverlapCache()--
OverlapCache()-- Symmetrizing overlaps.
OverlapCache()--   Finding missing twins.
OverlapCache()--   Found 40733 missing twins in 218025209 overlaps, 977 are strong.
OverlapCache()--   Dropping weak non-twin overlaps; allocated 0 MB scratch space.
OverlapCache()--   Dropped 6796 overlaps; scratch space released.
OverlapCache()--   Adding 33937 missing twin overlaps.
OverlapCache()--   Finished.

BestOverlapGraph()-- allocating best edges (41MB)

BestOverlapGraph()-- finding initial best edges.

BestOverlapGraph()-- filtering suspicious reads.
BestOverlapGraph()-- marked 714 reads as suspicious.

BestOverlapGraph()-- filtering high error edges.

BestOverlapGraph()-- filtering reads with lopsided best edges.

BestOverlapGraph()-- filtering spur reads.
BestOverlapGraph()-- detected 4728 spur reads and 6289 singleton reads.

BestOverlapGraph()-- removing best edges for contained reads.

==> BUILDING GREEDY TIGS.

breakSingletonTigs()-- Removed 9411 singleton tigs; reads are now unplaced.
optimizePositions()-- Optimizing read positions for 1353092 reads in 10348 tigs, with 4 threads.
optimizePositions()--   Allocating scratch space for 1353092 reads (0 KB).
optimizePositions()--   Initializing positions with 4 threads.
optimizePositions()--   Recomputing positions, iteration 1, with 4 threads.
optimizePositions()--     Reset zero.
optimizePositions()--     Checking convergence.
optimizePositions()--     converged: 1330611 reads
optimizePositions()--     changed:    22481 reads
optimizePositions()--   Recomputing positions, iteration 2, with 4 threads.
optimizePositions()--     Reset zero.
optimizePositions()--     Checking convergence.
optimizePositions()--     converged: 1332860 reads
optimizePositions()--     changed:    20232 reads
optimizePositions()--   Recomputing positions, iteration 3, with 4 threads.
optimizePositions()--     Reset zero.
optimizePositions()--     Checking convergence.
optimizePositions()--     converged: 1333298 reads
optimizePositions()--     changed:    19794 reads
optimizePositions()--   Recomputing positions, iteration 4, with 4 threads.
optimizePositions()--     Reset zero.
optimizePositions()--     Checking convergence.
optimizePositions()--     converged: 1333984 reads
optimizePositions()--     changed:    19108 reads
optimizePositions()--   Recomputing positions, iteration 5, with 4 threads.
optimizePositions()--     Reset zero.
optimizePositions()--     Checking convergence.
optimizePositions()--     converged: 1334319 reads
optimizePositions()--     changed:    18773 reads
optimizePositions()--   Expanding short reads with 4 threads.
optimizePositions()--   Updating positions.
optimizePositions()--   Finished.

==> PLACE CONTAINED READS.

computeErrorProfiles()-- Computing error profiles for 10348 tigs, with 4 threads.
computeErrorProfiles()-- Finished.

placeContains()-- placing 1144359 contained and 14377 unplaced reads, with 4 threads.
placeContains()-- Placed 1107493 contained reads and 5039 unplaced reads.
placeContains()-- Failed to place 36866 contained reads (too high error suspected) and 9338 unplaced reads (lack of overlaps suspected).
optimizePositions()-- Optimizing read positions for 1353092 reads in 10348 tigs, with 4 threads.
optimizePositions()--   Allocating scratch space for 1353092 reads (0 KB).
optimizePositions()--   Initializing positions with 4 threads.
bogart: bogart/AS_BAT_OptimizePositions.C:140: void Unitig::optimize_initPlace(uint32, optPos*, optPos*, bool, std::set<unsigned int>&, bool): Assertion `cnt > 0' failed.

Failed with 'Aborted'; backtrace (libbacktrace):
AS_UTL/AS_UTL_stackTrace.C::102 in _Z17AS_UTL_catchCrashiP9siginfo_tPv()
(null)::0 in (null)()
(null)::0 in (null)()
(null)::0 in (null)()
(null)::0 in (null)()
(null)::0 in (null)()
bogart/AS_BAT_OptimizePositions.C::140 in _ZN6Unitig18optimize_initPlaceEjP6optPosS1_bRSt3setIjSt4lessIjESaIjEEb()
bogart/AS_BAT_OptimizePositions.C::391 in _ZN9TigVector17optimizePositionsEPKcS1_._omp_fn.0()
../.././libgomp/team.c::120 in gomp_thread_start()
(null)::0 in (null)()
(null)::0 in (null)()
(null)::0 in (null)()

and I can't find unitigger/*.ctgStore/seqDB.v001.sizes.txt this file.

@starsyi
Copy link

starsyi commented Sep 14, 2017

@brianwalenz
in unitigging this dir, there have 0-mercounts, 1-overlapper/, 3-overlapErrorAdjustment/, 4-unitigger/, */gkpStore/ and */ovlStore/ , total six directorys. no any *.ctgStore/ directory.

@brianwalenz
Copy link
Member

That's definitely a crash. :-(

Are you able to share the data for debug (I'd need the unitigging/*gkpStore and unitigging/*ovlStore directories, containing reads and overlaps)? If not, we can try debugging remotely, but that's slow and painful.

Upload data using http://canu.readthedocs.io/en/latest/faq.html#how-can-i-send-data-to-you

@starsyi
Copy link

starsyi commented Sep 15, 2017

@brianwalenz I'm sorry，I can't share. The data I have run successfully by canu 1.3, 1.4 and 1.5. what I can tell you is the genome size about 1Gbp , and the date about 120Gbp。

@swarris
Copy link

swarris commented Sep 18, 2017

I have the same problems. Retrying did not help. For now I'm switching back to 1.5.

unitigger.err:
computeErrorProfiles()-- Computing error profiles for 167752 tigs, with 8 threads.
computeErrorProfiles()-- Finished.

placeContains()-- placing 2274157 contained and 339641 unplaced reads, with 8 threads.
placeContains()-- Placed 1579082 contained reads and 19901 unplaced reads.
placeContains()-- Failed to place 695075 contained reads (too high error suspected) and 319740 unplaced reads (lack of overlaps suspected).
optimizePositions()-- Optimizing read positions for 167752 reads in 2827314 tigs, with 8 threads.
optimizePositions()-- Allocating scratch space for 2827314 reads (0 KB).
optimizePositions()-- Initializing positions with 8 threads.
bogart: bogart/AS_BAT_OptimizePositions.C:140: void Unitig::optimize_initPlace(uint32, optPos*, optPos*, bool, std::set&, bool): Assertion `cnt > 0' failed.

Failed with 'Aborted'; backtrace (libbacktrace):
AS_UTL/AS_UTL_stackTrace.C::102 in _Z17AS_UTL_catchCrashiP7siginfoPv()
(null)::0 in (null)()
(null)::0 in (null)()
(null)::0 in (null)()
(null)::0 in (null)()
(null)::0 in (null)()
bogart/AS_BAT_OptimizePositions.C::140 in _ZN6Unitig18optimize_initPlaceEjP6optPosS1_bRSt3setIjSt4lessIjESaIjEEb()
(null)::0 in (null)()
(null)::0 in (null)()
(null)::0 in (null)()
(null)::0 in (null)()
Aborted

Bogart log:
Command:
/home/ubuntu/bin/canu-1.6/Linux-amd64/bin/bogart
-G ../chrF_80m_150ov.gkpStore
-O ../chrF_80m_150ov.ovlStore
-o ./chrF_80m_150ov
-gs 80000000
-eg 0.075
-eM 0.075
-mo 150
-dg 6
-db 6
-dr 3
-ca 2100
-cp 200
-threads 8
-M 23
-unassembled 2 0 1.0 0.5 5

@angross angross mentioned this issue Dec 1, 2017
Closed
@brianwalenz
Copy link
Member

This should be fixed (d580c23). Note that you CANNOT upgrade to the latest code and resume an assembly started with the 1.6 release.

Duplicate of #718.

@brianwalenz brianwalenz mentioned this issue Mar 29, 2018
Closed
brianwalenz added a commit that referenced this issue Apr 16, 2018
@brianwalenz
Be more selective on which overlaps to use when optimizing positions.…
7feb0b0 
… Issues #546, #718 and #844 (at least).
@Bessem06 Bessem06 mentioned this issue Apr 18, 2018
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@swarris @brianwalenz @starsyi @caonetto