Code

Reference-less contig stitching#1221

Code

Draft

Donaim wants to merge 156 commits intomastercfe-lab/MiCall:masterfrom contig-mergercfe-lab/MiCall:contig-mergerCopy head branch name to clipboard

+2,057-889

Commits

Commits on Dec 24, 2024

• Start implementation of merge_contigs

  Donaim

  committed
  0beecb6

  Copy full SHA for 0beecb6

• Add tests for find_maximum_overlap

  Donaim

  committed
  9ee3136

  Copy full SHA for 9ee3136

• Add more test cases for find_maximum_overlap

  Donaim

  committed
  6beb4d9

  Copy full SHA for 6beb4d9

• Rename merge_contigs into find_maximum_overlap

  Donaim

  committed
  2b77b1a

  Copy full SHA for 2b77b1a

• Implement show_maximum_overlap

  Donaim

  committed
  a063ac6

  Copy full SHA for a063ac6

• Add main entry to find_maximum_overlap script

  Donaim

  committed
  4708aab

  Copy full SHA for 4708aab

• Fix no overlap case in find_maximum_overlap

  Donaim

  committed
  c1af31d

  Copy full SHA for c1af31d

• Fix formatting error in tests

  Donaim

  committed
  f104e84

  Copy full SHA for f104e84

• Change tie behaviour of find_maximum_overlap

  Donaim

  committed
  aaccdc2

  Copy full SHA for aaccdc2

Commits on Dec 25, 2024

• Add TODO comment for find_maximum_overlap

  Donaim

  committed
  73c7ed5

  Copy full SHA for 73c7ed5

Commits on Dec 31, 2024

• Improve code in tests for find_maximum_overlap

  Donaim

  committed
  eb22286

  Copy full SHA for eb22286

• Initial implementation of referenceless contig stitcher

  Donaim

  committed
  ec4647b

  Copy full SHA for ec4647b

• Prepare to integrate referenceless contig stitcher with CLI

  Donaim

  committed
  3943d72

  Copy full SHA for 3943d72

• Split contig stitcher reading based on reference avail

  Donaim

  committed
  f4a6179

  Copy full SHA for f4a6179

• Factor out referencefull_contig_stitcher

  Donaim

  committed
  d286cc6

  Copy full SHA for d286cc6

• Fix warning in test_contig_stitcher

  Donaim

  committed
  931388e

  Copy full SHA for 931388e

• Improve typing in referencefull_contig_stitcher

  Donaim

  committed
  ca8248b

  Copy full SHA for ca8248b

• Finish initial implementation of referenceless_contig_stitcher

  Donaim

  committed
  6ec43ca

  Copy full SHA for 6ec43ca

• Add referenceless stitcher to contig_stitcher.py

  Donaim

  committed
  a240d7d

  Copy full SHA for a240d7d

Commits on Jan 2, 2025

• Change operating format of referenceless stitcher to FASTA

  Donaim

  committed
  47065c9

  Copy full SHA for 47065c9

• Improve contig_stitcher.py CLI parsing

  Donaim

  committed
  1db40d9

  Copy full SHA for 1db40d9

• Correct handling of FASTA extension in sample.py

  Donaim

  committed
  2b95042

  Copy full SHA for 2b95042

• Add more test cases for referenceless stitcher

  Donaim

  committed
  01c6c17

  Copy full SHA for 01c6c17

• Merge remote-tracking branch 'origin/master' into contig-merger

  Donaim

  committed
  eeccb44

  Copy full SHA for eeccb44

• Fix writing in referenceless_contig_stitcher

  Donaim

  committed
  9ec9792

  Copy full SHA for 9ec9792

• Perform referenceless contig stitching in sample.py

  Donaim

  committed
  e25717a

  Copy full SHA for e25717a

• Add Config.empty static method

  Donaim

  committed
  09972c0

  Copy full SHA for 09972c0

• Fix style error in stitcher_contigs.py

  Donaim

  committed
  6b4c16a

  Copy full SHA for 6b4c16a

• Make sure that empty contings are not causing a crash

  Donaim

  committed
  acba19b

  Copy full SHA for acba19b

• Improve combination in referenceless contig stitcher

  Donaim

  committed
  f6d9c2a

  Copy full SHA for f6d9c2a

Commits on Jan 3, 2025

• Fix type error in referenceless contig_stitcher

  Donaim

  committed
  6589510

  Copy full SHA for 6589510

• Improve probability calculation in referenceless contig_stitcher

  Donaim

  committed
  42fcc9e

  Copy full SHA for 42fcc9e

• Run referenceless contig stitcher in sample.py

  Donaim

  committed
  0fd9979

  Copy full SHA for 0fd9979

• Tweak ACCEPTABLE_STITCHING_PROB in referenceless contig_stitcher

  Donaim

  committed
  e23898d

  Copy full SHA for e23898d

• Improve code readability in referencless contig stitcher

  Donaim

  committed
  96281b8

  Copy full SHA for 96281b8

• Improve overlap combination in referenceless contig stitcher

  Donaim

  committed
  ed4cc84

  Copy full SHA for ed4cc84

• Factor out overlap stitcher

  Donaim

  committed
  b988304

  Copy full SHA for b988304

• Fix typing in contig_stitcher

  Donaim

  committed
  88dc5ba

  Copy full SHA for 88dc5ba

• Factor out sort_concordance_indexes into overlap_stitcher

  Donaim

  committed
  60a45b7

  Copy full SHA for 60a45b7

• Merge remote-tracking branch 'origin/master' into HEAD

  Donaim

  committed
  ae8a6b7

  Copy full SHA for ae8a6b7

• Improve overlap combination in referenceless_contig_stitcher

  Donaim

  committed
  695e5b8

  Copy full SHA for 695e5b8

• Implement a more accurate pvalue calculation for overlaps

  Donaim

  committed
  cb797f7

  Copy full SHA for cb797f7

• Remove a finished FIXME comment

  Donaim

  committed
  4952a4d

  Copy full SHA for 4952a4d

• Fix type error in contig_stitcher.py

  Donaim

  committed
  ecb81c3

  Copy full SHA for ecb81c3

• Memoize calls to calc_overlap_pvalue

  Donaim

  committed
  0f95ed7

  Copy full SHA for 0f95ed7

• Do not use lists in refereceless_contig_stitcher

  Donaim

  committed
  9020979

  Copy full SHA for 9020979

• Fix logging in contig stitchers

  Donaim

  committed
  d38ca14

  Copy full SHA for d38ca14

Commits on Jan 4, 2025

• Add csv_to_fasta utility

  Donaim

  committed
  31a8d3b

  Copy full SHA for 31a8d3b

• Add tests for csv_to_fasta

  Donaim

  committed
  9411fa2

  Copy full SHA for 9411fa2

• Add some logging to referenceless contig stitcher

  Donaim

  committed
  8243579

  Copy full SHA for 8243579

• Improve CLI entry of contig_stitcher.py

  Donaim

  committed
  a041fc7

  Copy full SHA for a041fc7

• Fix covering behaviour of referenceless contig stitcher

  Donaim

  committed
  582fc82

  Copy full SHA for 582fc82

• Add one more test to referenceless contig_stitcher

  Donaim

  committed
  d50a36c

  Copy full SHA for d50a36c

• Fix probability calculations for covered contigs in referenceless stitcher

  Donaim

  committed
  fe6fbc9

  Copy full SHA for fe6fbc9

• Fix formatting error

  Donaim

  committed
  059b4a3

  Copy full SHA for 059b4a3

• Small optimize alignments

  Donaim

  committed
  b36afe4

  Copy full SHA for b36afe4

• Factor out score methods in ContigsPaths

  Donaim

  committed
  8ed3d95

  Copy full SHA for 8ed3d95

Commits on Jan 5, 2025

• Simplify control flow in extend_by_1

  Donaim

  committed
  c290422

  Copy full SHA for c290422

• Further optimize referenceless version

  Donaim

  committed
  961bd53

  Copy full SHA for 961bd53

• Introduce pessimisstic probability

  Donaim

  committed
  c0e4da1

  Copy full SHA for c0e4da1

• Increase threshold for stitching in referenceless

  Donaim

  committed
  b6dd4a9

  Copy full SHA for b6dd4a9

• Further increase threshold for stitching in referenceless

  Donaim

  committed
  27a392f

  Copy full SHA for 27a392f

• Adjust MAX_ALTERNATIVES value in referenceless

  Donaim

  committed
  1938719

  Copy full SHA for 1938719

Commits on Jan 7, 2025

• Better error handling in csv_to_fasta

  Donaim

  committed
  041a5be

  Copy full SHA for 041a5be

• Stitch merged_contigs with referenceless stitcher

  Donaim

  committed
  60330e6

  Copy full SHA for 60330e6

• Do not accept merged_contigs in fasta_to_csv

  Donaim

  committed
  06644a0

  Copy full SHA for 06644a0

Commits on Jan 8, 2025

• Remove pessimisstic_probability from referenceless

  Donaim

  committed
  cfcaad5

  Copy full SHA for cfcaad5

• Revert "Remove pessimisstic_probability from referenceless"

  Show description for d7082c3
  Donaim

  committed
  d7082c3

  Copy full SHA for d7082c3

• Fix pessimisstic probability calculation in referenceless

  Donaim

  committed
  f656023

  Copy full SHA for f656023

• Fix fasta_to_csv CLI interface

  Donaim

  committed
  3f0f93b

  Copy full SHA for 3f0f93b

Commits on Jan 9, 2025

• Simplify calc_overlap_pvalue calculation

  Donaim

  committed
  b10b826

  Copy full SHA for b10b826

• Small improvements to overlap finder

  Donaim

  committed
  3c713d2

  Copy full SHA for 3c713d2

• Optimize find_maximum_overlap

  Donaim

  committed
  08e33ec

  Copy full SHA for 08e33ec

• Use optimized find_maximum_overlap in referenceless

  Donaim

  committed
  131acc7

  Copy full SHA for 131acc7

• Simplify code in find_maximum_overlap

  Donaim

  committed
  35c7907

  Copy full SHA for 35c7907

• Optimize overlap extension in referenceless

  Donaim

  committed
  f591a65

  Copy full SHA for f591a65

Commits on Jan 10, 2025

• Optimize referenceless by applying an heuristics

  Donaim

  committed
  af7312b

  Copy full SHA for af7312b

• Optimize referenceless by improving max overlap heuristics

  Donaim

  committed
  fc24bf5

  Copy full SHA for fc24bf5

• Improve caching in referenceless

  Donaim

  committed
  2d0c4fa

  Copy full SHA for 2d0c4fa

• Fix typing in referenceless stitcher

  Donaim

  committed
  f13bdb4

  Copy full SHA for f13bdb4

• Enable find_maximum_overlap caching

  Donaim

  committed
  617bf27

  Copy full SHA for 617bf27

• Tweak minimap parameters in referenceless

  Show description for eb87412
  Donaim

  committed
  eb87412

  Copy full SHA for eb87412

• Cache cutoff determination in referenceless

  Donaim

  committed
  4fb6167

  Copy full SHA for 4fb6167

• Reduce amount of logging in referenceless

  Donaim

  committed
  23d30c6

  Copy full SHA for 23d30c6

• Reduce amout of logging in referenceless further

  Donaim

  committed
  4b4778f

  Copy full SHA for 4b4778f

• Sort contigs by size in referenceless

  Donaim

  committed
  ba5daf1

  Copy full SHA for ba5daf1

• Add size field to Overlap struct of referenceless

  Donaim

  committed
  c97a211

  Copy full SHA for c97a211

• Return number of matches from find_maximum_overlap

  Donaim

  committed
  c2b3ae3

  Copy full SHA for c2b3ae3

• Fix probability heuristic in referenceless

  Donaim

  committed
  a4db363

  Copy full SHA for a4db363

• Remove redundant calculation in referenceless

  Donaim

  committed
  33288ca

  Copy full SHA for 33288ca

• Improve scoring in referenceless

  Donaim

  committed
  4ea1fd5

  Copy full SHA for 4ea1fd5

• Fix scoring in referenceless

  Donaim

  committed
  a159da1

  Copy full SHA for a159da1

• Revert to non-adaptable scoring in referenceless

  Donaim

  committed
  40f3c88

  Copy full SHA for 40f3c88

• Revert to before Score class in referenceless

  Donaim

  committed
  26ef042

  Copy full SHA for 26ef042

• Remove redundant comments in referenceless

  Donaim

  committed
  8aea4be

  Copy full SHA for 8aea4be

• Wrap max_acceptable_prob in referenceless

  Donaim

  committed
  393180c

  Copy full SHA for 393180c

Commits on Jan 11, 2025

• Limit number of contigs in referenceless

  Donaim

  committed
  865bc32

  Copy full SHA for 865bc32

• Improve MaximumAcceptableProbability calculation in referenceless

  Donaim

  committed
  e63d9e4

  Copy full SHA for e63d9e4

• Do not clear MaximumAcceptableProbability pool in referenceless

  Donaim

  committed
  cdf16f2

  Copy full SHA for cdf16f2

• Revert "Do not clear MaximumAcceptableProbability pool in referenceless"

  Show description for c93be63
  Donaim

  committed
  c93be63

  Copy full SHA for c93be63

Commits on Jan 13, 2025

• Add a final loop in referenceless to collect covered contigs

  Donaim

  committed
  d60e474

  Copy full SHA for d60e474

• Add another heuristic to referenceless stitcher

  Donaim

  committed
  e040cb6

  Copy full SHA for e040cb6

• Rename classes in referenceless

  Donaim

  committed
  60e1cb7

  Copy full SHA for 60e1cb7

• Small improvements to pool structure in referenceless

  Donaim

  committed
  45e50dc

  Copy full SHA for 45e50dc

• Reduce logging in referenceless

  Donaim

  committed
  a1a4eca

  Copy full SHA for a1a4eca

• Further improvements to the pool structure of referenceless

  Donaim

  committed
  d846b08

  Copy full SHA for d846b08

• Replace Fractions by ints in referenceless

  Donaim

  committed
  24825e0

  Copy full SHA for 24825e0

• Use correlate instead of convolve in find_maximum_overlap

  Donaim

  committed
  bbf87b0

  Copy full SHA for bbf87b0

• Use oaconvolve in find_maximum_overlap

  Donaim

  committed
  1225d33

  Copy full SHA for 1225d33

• Use np.convolve in find_maxmimum_overlap

  Donaim

  committed
  1639dad

  Copy full SHA for 1639dad

• Do not use scipy in find_maximum_overlap

  Donaim

  committed
  2e03d9d

  Copy full SHA for 2e03d9d

• Use faster method for conversion of strings into np.arrays

  Donaim

  committed
  ce5971e

  Copy full SHA for ce5971e

Commits on Jan 14, 2025

• Improve concordance calculations in contig stitcher

  Show description for 94017ad
  Donaim

  committed
  94017ad

  Copy full SHA for 94017ad

• Make probability the only score in referenceless

  Donaim

  committed
  4de1569

  Copy full SHA for 4de1569

• Remove redundant method in referenceless

  Donaim

  committed
  d058a17

  Copy full SHA for d058a17

• Do not use len(SortedList)

  Donaim

  committed
  8483bbe

  Copy full SHA for 8483bbe

• Fix initial filtering in referenceless

  Donaim

  committed
  56ff67e

  Copy full SHA for 56ff67e

• Improve logging in referenceless

  Donaim

  committed
  95d85b1

  Copy full SHA for 95d85b1

• Shortcut referenceless final laps

  Donaim

  committed
  bc49847

  Copy full SHA for bc49847

• Fix punctuation in referenceless logging

  Donaim

  committed
  be0090e

  Copy full SHA for be0090e

• Do not print average concordance in contig stitcher

  Donaim

  committed
  0260a02

  Copy full SHA for 0260a02

• Optimize smallest score access in referenceless

  Donaim

  committed
  983cd8e

  Copy full SHA for 983cd8e

• Fix caching in referenceless

  Donaim

  committed
  9a8c6e5

  Copy full SHA for 9a8c6e5

• Fix some logging in referenceless

  Donaim

  committed
  569e3c5

  Copy full SHA for 569e3c5

• Optimize conversion into numpy arrays in find_maximum_overlap

  Donaim

  committed
  1e3ccbf

  Copy full SHA for 1e3ccbf

• Reduce amount of caching in referenceless

  Donaim

  committed
  f1204b9

  Copy full SHA for f1204b9

• Remove CombineCache from referenceless completely

  Donaim

  committed
  abe0be2

  Copy full SHA for abe0be2

• Implement a seed idea for referenceless

  Donaim

  committed
  e6bd7e8

  Copy full SHA for e6bd7e8

• Add singleton method to ContigsPath in referenceless

  Donaim

  committed
  1dee289

  Copy full SHA for 1dee289

• Bring back combine_cache to referenceless

  Donaim

  committed
  85c5d78

  Copy full SHA for 85c5d78

• Factor out cutoffs handling in referenceless

  Donaim

  committed
  262ba71

  Copy full SHA for 262ba71

• Make cutoffs cache global again in referenceless

  Donaim

  committed
  6262410

  Copy full SHA for 6262410

• Make OverlapFinder global in referenceless

  Donaim

  committed
  fe6c8d7

  Copy full SHA for fe6c8d7

• Introduce ALIGN_CACHE into referenceless

  Donaim

  committed
  5b53ccc

  Copy full SHA for 5b53ccc

• Improve heuristics that skips overlap search in referenceless

  Donaim

  committed
  06d37f4

  Copy full SHA for 06d37f4

• Remove useless checks in referenceless

  Donaim

  committed
  6d071f6

  Copy full SHA for 6d071f6

• Reorder heuristics calculations in referenceless

  Donaim

  committed
  3e7f20c

  Copy full SHA for 3e7f20c

Commits on Jan 16, 2025

• Make sure to start with longer seeds first in referenceless

  Donaim

  committed
  0b40c4b

  Copy full SHA for 0b40c4b

• Add cat utility

  Donaim

  committed
  98a2807

  Copy full SHA for 98a2807

• Use cat in sample.py

  Donaim

  committed
  8fb4f7d

  Copy full SHA for 8fb4f7d

• Remove all options from cat utility

  Donaim

  committed
  4951ac4

  Copy full SHA for 4951ac4

Commits on Jan 20, 2025

• Factor out code related to concordance calculations

  Donaim

  committed
  66acb77

  Copy full SHA for 66acb77

Commits on Jan 21, 2025

• Merge remote-tracking branch 'origin/master' into contig-merger

  Donaim

  committed
  83463ca

  Copy full SHA for 83463ca

• Simplify exp_accumulate_array code

  Donaim

  committed
  4b0352c

  Copy full SHA for 4b0352c

• Make sure that exp_accumulate_array receives a binary array

  Donaim

  committed
  237bf45

  Copy full SHA for 237bf45

• Improve typing in exp_accumulate_array

  Donaim

  committed
  92a4d28

  Copy full SHA for 92a4d28

• Rewrite concordance calculations in numpy

  Donaim

  committed
  466e473

  Copy full SHA for 466e473

• Factor out exp_accumulate_array in overlap_stitcher

  Donaim

  committed
  6c74dd0

  Copy full SHA for 6c74dd0

• Optimize numpy.fromiter calls

  Donaim

  committed
  751d20f

  Copy full SHA for 751d20f

Commits on Jan 22, 2025

• Fix cat.py exceptional behaviour

  Donaim

  committed
  cce2d95

  Copy full SHA for cce2d95

• Allow to accept 0 files in cat.py

  Donaim

  committed
  b8f1d9e

  Copy full SHA for b8f1d9e

Commits on Jan 23, 2025

• Uncomment perfectly good test cases

  Donaim

  committed
  d7e6d83

  Copy full SHA for d7e6d83

• Move overlap stitcher tests into a separate file

  Donaim

  committed
  e84baf7

  Copy full SHA for e84baf7

• Fix logical error in overlap_stitcher

  Donaim

  committed
  f710943

  Copy full SHA for f710943

Commits on Jan 24, 2025

• Fall back to scipy.signal.convolve for very big arrays

  Donaim

  committed
  64f418c

  Copy full SHA for 64f418c

Commits on Jan 31, 2025

• Make version check more reliable in MiCall

  Donaim

  committed
  a2ae0be

  Copy full SHA for a2ae0be