forked from BenLangmead/bowtie2
-
Notifications
You must be signed in to change notification settings - Fork 0
/
MANUAL
2060 lines (1490 loc) · 84.4 KB
/
MANUAL
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
INTRODUCTION
What is Bowtie 2?
Bowtie 2 is an ultrafast and memory-efficient tool for aligning
sequencing reads to long reference sequences. It is particularly good at
aligning reads of about 50 up to 100s or 1,000s of characters to
relatively long (e.g. mammalian) genomes. Bowtie 2 indexes the genome
with an FM Index (based on the Burrows-Wheeler Transform or BWT) to keep
its memory footprint small: for the human genome, its memory footprint
is typically around 3.2 gigabytes of RAM. Bowtie 2 supports gapped,
local, and paired-end alignment modes. Multiple processors can be used
simultaneously to achieve greater alignment speed. Cufflinks: a tool for
transcriptome assembly and isoform quantitiation from Bowtie 2 outputs
alignments in SAM format, enabling interoperation with a large number of
other tools (e.g. SAMtools, GATK) that use SAM. Bowtie 2 is distributed
under the GPLv3 license, and it runs on the command line under Windows,
Mac OS X and Linux.
Bowtie 2 is often the first step in pipelines for comparative genomics,
including for variation calling, ChIP-seq, RNA-seq, BS-seq. Bowtie 2 and
Bowtie (also called “Bowtie 1” here) are also tightly integrated into
some tools, including TopHat: a fast splice junction mapper for RNA-seq
reads, RNA-seq reads, Crossbow: a cloud-enabled software tool for
analyzing resequencing data, and Myrna: a cloud-enabled software tool
for aligning RNA-seq reads and measuring differential gene expression.
If you use Bowtie 2 for your published research, please cite the Bowtie
paper. Thank you!
How is Bowtie 2 different from Bowtie 1?
Bowtie 1 was released in 2009 and was geared toward aligning the
relatively short sequencing reads (up to 25-50 nucleotides) prevalent at
the time. Since then, technology has improved both sequencing throughput
(more nucleotides produced per sequencer per day) and read length (more
nucleotides per read).
The chief differences between Bowtie 1 and Bowtie 2 are:
1. For reads longer than about 50 bp Bowtie 2 is generally faster, more
sensitive, and uses less memory than Bowtie 1. For relatively short
reads ( e.g. less than 50 bp) Bowtie 1 is sometimes faster and/or
more sensitive.
2. Bowtie 2 supports gapped alignment with affine gap penalties. Number
of gaps and gap lengths are not restricted, except by way of the
configurable scoring scheme. Bowtie 1 finds just ungapped
alignments.
3. Bowtie 2 supports local alignment, which doesn’t require reads to
align end-to-end. Local alignments might be “trimmed” (“soft
clipped”) at one or both extremes in a way that optimizes alignment
score. Bowtie 2 also supports end-to-end alignment which, like
Bowtie 1, requires that the read align entirely.
4. There is no upper limit on read length in Bowtie 2. Bowtie 1 had an
upper limit of around 1000 bp.
5. Bowtie 2 allows alignments to overlap ambiguous characters (e.g. Ns)
in the reference. Bowtie 1 does not.
6. Bowtie 2 does away with Bowtie 1’s notion of alignment “stratum”,
and its distinction between “Maq-like” and “end-to-end” modes. In
Bowtie 2 all alignments lie along a continuous spectrum of alignment
scores where the scoring scheme, similar to Needleman-Wunsch and
Smith-Waterman.
7. Bowtie 2’s paired-end alignment is more flexible. E.g. for pairs
that do not align in a paired fashion, Bowtie 2 attempts to find
unpaired alignments for each mate.
8. Bowtie 2 reports a spectrum of mapping qualities, in contrast for
Bowtie 1 which reports either 0 or high.
9. Bowtie 2 does not align colorspace reads.
Bowtie 2 is not a “drop-in” replacement for Bowtie 1. Bowtie 2’s
command-line arguments and genome index format are both different from
Bowtie 1’s.
What isn’t Bowtie 2?
Bowtie 1 and Bowtie 2 are not general-purpose alignment tools like
MUMmer, BLAST or Vmatch. Bowtie 2 works best when aligning to large
genomes, though it supports arbitrarily small reference sequences
(e.g. amplicons). It handles very long reads (i.e. upwards of 10s or
100s of kilobases), but it is optimized for the read lengths and error
modes yielded by recent sequencers, such as the Illumina HiSeq 2000,
Roche 454, and Ion Torrent instruments.
If your goal is to align two very large sequences (e.g. two genomes),
consider using MUMmer. If your goal is very sensitive alignment to a
relatively short reference sequence (e.g. a bacterial genome), this can
be done with Bowtie 2 but you may want to consider using tools like
NUCmer, BLAT, or BLAST. These tools can be extremely slow when the
reference genome is long, but are often adequate when the reference is
short.
Bowtie 2 does not support alignment of colorspace reads.
What does it mean that some older Bowtie 2 versions are “beta”?
We said those Bowtie 2 versions were in “beta” to convey that it was not
as polished as a tool that had been around for a while, and was still in
flux. Since version 2.0.1, we declared Bowtie 2 was no longer “beta”.
OBTAINING BOWTIE 2
Download Bowtie 2 sources and binaries from the Download section of the
Sourceforge site. Binaries are available for the Intel x86_64
architecture running Linux, Mac OS X, and Windows. If you plan to
compile Bowtie 2 yourself, make sure to get the source package, i.e.,
the filename that ends in “-source.zip”.
Building from source
Building Bowtie 2 from source requires a GNU-like environment with GCC,
GNU Make and other basics. It should be possible to build Bowtie 2 on
most vanilla Linux installations or on a Mac installation with Xcode
installed. (But see note about the TBB library below). Bowtie 2 can also
be built on Windows using a 64-bit MinGW distribution and MSYS. In order
to simplify the MinGW setup it might be worth investigating popular
MinGW personal builds since these are coming already prepared with most
of the toolchains needed.
First, download the source package from the sourceforge site. Make sure
you’re getting the source package; the file downloaded should end in
-source.zip. Unzip the file, change to the unzipped directory, and build
the Bowtie 2 tools by running GNU make (usually with the command make,
but sometimes with gmake) with no arguments. If building with MinGW, run
make from the MSYS environment.
Bowtie 2 can be run on many threads. By default, Bowtie 2 uses the
Threading Building Blocks library (TBB) for this. If TBB is not
available on your system (e.g. make prints an error like
tbb/mutex.h: No such file or directory), you can install it yourself
(see Threading Building Blocks library) or build Bowtie 2 with
make NO_TBB=1 to use pthreads or Windows native multithreading instead.
Adding to PATH
By adding your new Bowtie 2 directory to your PATH environment variable,
you ensure that whenever you run bowtie2, bowtie2-build or
bowtie2-inspect from the command line, you will get the version you just
installed without having to specify the entire path. This is recommended
for most users. To do this, follow your operating system’s instructions
for adding the directory to your PATH.
If you would like to install Bowtie 2 by copying the Bowtie 2 executable
files to an existing directory in your PATH, make sure that you copy all
the executables, including bowtie2, bowtie2-align-s, bowtie2-align-l,
bowtie2-build, bowtie2-build-s, bowtie2-build-l, bowtie2-inspect,
bowtie2-inspect-s and bowtie2-inspect-l.
THE bowtie2 ALIGNER
bowtie2 takes a Bowtie 2 index and a set of sequencing read files and
outputs a set of alignments in SAM format.
“Alignment” is the process by which we discover how and where the read
sequences are similar to the reference sequence. An “alignment” is a
result from this process, specifically: an alignment is a way of “lining
up” some or all of the characters in the read with some characters from
the reference in a way that reveals how they’re similar. For example:
Read: GACTGGGCGATCTCGACTTCG
||||| |||||||||| |||
Reference: GACTG--CGATCTCGACATCG
Where dash symbols represent gaps and vertical bars show where aligned
characters match.
We use alignment to make an educated guess as to where a read originated
with respect to the reference genome. It’s not always possible to
determine this with certainty. For instance, if the reference genome
contains several long stretches of As (AAAAAAAAA etc.) and the read
sequence is a short stretch of As (AAAAAAA), we cannot know for certain
exactly where in the sea of As the read originated.
End-to-end alignment versus local alignment
By default, Bowtie 2 performs end-to-end read alignment. That is, it
searches for alignments involving all of the read characters. This is
also called an “untrimmed” or “unclipped” alignment.
When the –local option is specified, Bowtie 2 performs local read
alignment. In this mode, Bowtie 2 might “trim” or “clip” some read
characters from one or both ends of the alignment if doing so maximizes
the alignment score.
End-to-end alignment example
The following is an “end-to-end” alignment because it involves all the
characters in the read. Such an alignment can be produced by Bowtie 2 in
either end-to-end mode or in local mode.
Read: GACTGGGCGATCTCGACTTCG
Reference: GACTGCGATCTCGACATCG
Alignment:
Read: GACTGGGCGATCTCGACTTCG
||||| |||||||||| |||
Reference: GACTG--CGATCTCGACATCG
Local alignment example
The following is a “local” alignment because some of the characters at
the ends of the read do not participate. In this case, 4 characters are
omitted (or “soft trimmed” or “soft clipped”) from the beginning and 3
characters are omitted from the end. This sort of alignment can be
produced by Bowtie 2 only in local mode.
Read: ACGGTTGCGTTAATCCGCCACG
Reference: TAACTTGCGTTAAATCCGCCTGG
Alignment:
Read: ACGGTTGCGTTAA-TCCGCCACG
||||||||| ||||||
Reference: TAACTTGCGTTAAATCCGCCTGG
Scores: higher = more similar
An alignment score quantifies how similar the read sequence is to the
reference sequence aligned to. The higher the score, the more similar
they are. A score is calculated by subtracting penalties for each
difference (mismatch, gap, etc.) and, in local alignment mode, adding
bonuses for each match.
The scores can be configured with the --ma (match bonus), --mp (mismatch
penalty), --np (penalty for having an N in either the read or the
reference), --rdg (affine read gap penalty) and --rfg (affine reference
gap penalty) options.
End-to-end alignment score example
A mismatched base at a high-quality position in the read receives a
penalty of -6 by default. A length-2 read gap receives a penalty of -11
by default (-5 for the gap open, -3 for the first extension, -3 for the
second extension). Thus, in end-to-end alignment mode, if the read is 50
bp long and it matches the reference exactly except for one mismatch at
a high-quality position and one length-2 read gap, then the overall
score is -(6 + 11) = -17.
The best possible alignment score in end-to-end mode is 0, which happens
when there are no differences between the read and the reference.
Local alignment score example
A mismatched base at a high-quality position in the read receives a
penalty of -6 by default. A length-2 read gap receives a penalty of -11
by default (-5 for the gap open, -3 for the first extension, -3 for the
second extension). A base that matches receives a bonus of +2 be
default. Thus, in local alignment mode, if the read is 50 bp long and it
matches the reference exactly except for one mismatch at a high-quality
position and one length-2 read gap, then the overall score equals the
total bonus, 2 * 49, minus the total penalty, 6 + 11, = 81.
The best possible score in local mode equals the match bonus times the
length of the read. This happens when there are no differences between
the read and the reference.
Valid alignments meet or exceed the minimum score threshold
For an alignment to be considered “valid” (i.e. “good enough”) by Bowtie
2, it must have an alignment score no less than the minimum score
threshold. The threshold is configurable and is expressed as a function
of the read length. In end-to-end alignment mode, the default minimum
score threshold is -0.6 + -0.6 * L, where L is the read length. In local
alignment mode, the default minimum score threshold is 20 + 8.0 * ln(L),
where L is the read length. This can be configured with the --score-min
option. For details on how to set options like --score-min that
correspond to functions, see the section on setting function options.
Mapping quality: higher = more unique
The aligner cannot always assign a read to its point of origin with high
confidence. For instance, a read that originated inside a repeat element
might align equally well to many occurrences of the element throughout
the genome, leaving the aligner with no basis for preferring one over
the others.
Aligners characterize their degree of confidence in the point of origin
by reporting a mapping quality: a non-negative integer Q = -10 log10 p,
where p is an estimate of the probability that the alignment does not
correspond to the read’s true point of origin. Mapping quality is
sometimes abbreviated MAPQ, and is recorded in the SAM MAPQ field.
Mapping quality is related to “uniqueness.” We say an alignment is
unique if it has a much higher alignment score than all the other
possible alignments. The bigger the gap between the best alignment’s
score and the second-best alignment’s score, the more unique the best
alignment, and the higher its mapping quality should be.
Accurate mapping qualities are useful for downstream tools like variant
callers. For instance, a variant caller might choose to ignore evidence
from alignments with mapping quality less than, say, 10. A mapping
quality of 10 or less indicates that there is at least a 1 in 10 chance
that the read truly originated elsewhere.
Aligning pairs
A “paired-end” or “mate-pair” read consists of pair of mates, called
mate 1 and mate 2. Pairs come with a prior expectation about (a) the
relative orientation of the mates, and (b) the distance separating them
on the original DNA molecule. Exactly what expectations hold for a given
dataset depends on the lab procedures used to generate the data. For
example, a common lab procedure for producing pairs is Illumina’s
Paired-end Sequencing Assay, which yields pairs with a relative
orientation of FR (“forward, reverse”) meaning that if mate 1 came from
the Watson strand, mate 2 very likely came from the Crick strand and
vice versa. Also, this protocol yields pairs where the expected genomic
distance from end to end is about 200-500 base pairs.
For simplicity, this manual uses the term “paired-end” to refer to any
pair of reads with some expected relative orientation and distance.
Depending on the protocol, these might actually be referred to as
“paired-end” or “mate-paired.” Also, we always refer to the individual
sequences making up the pair as “mates.”
Paired inputs
Pairs are often stored in a pair of files, one file containing the mate
1s and the other containing the mates 2s. The first mate in the file for
mate 1 forms a pair with the first mate in the file for mate 2, the
second with the second, and so on. When aligning pairs with Bowtie 2,
specify the file with the mate 1s mates using the -1 argument and the
file with the mate 2s using the -2 argument. This causes Bowtie 2 to
take the paired nature of the reads into account when aligning them.
Paired SAM output
When Bowtie 2 prints a SAM alignment for a pair, it prints two records
(i.e. two lines of output), one for each mate. The first record
describes the alignment for mate 1 and the second record describes the
alignment for mate 2. In both records, some of the fields of the SAM
record describe various properties of the alignment; for instance, the
7th and 8th fields (RNEXT and PNEXT respectively) indicate the reference
name and position where the other mate aligned, and the 9th field
indicates the inferred length of the DNA fragment from which the two
mates were sequenced. See the SAM specification for more details
regarding these fields.
Concordant pairs match pair expectations, discordant pairs don’t
A pair that aligns with the expected relative mate orientation and with
the expected range of distances between mates is said to align
“concordantly”. If both mates have unique alignments, but the alignments
do not match paired-end expectations (i.e. the mates aren’t in the
expected relative orientation, or aren’t within the expected distance
range, or both), the pair is said to align “discordantly”. Discordant
alignments may be of particular interest, for instance, when seeking
structural variants.
The expected relative orientation of the mates is set using the --ff,
--fr, or --rf options. The expected range of inter-mates distances (as
measured from the furthest extremes of the mates; also called “outer
distance”) is set with the -I and -X options. Note that setting -I and
-X far apart makes Bowtie 2 slower. See documentation for -I and -X.
To declare that a pair aligns discordantly, Bowtie 2 requires that both
mates align uniquely. This is a conservative threshold, but this is
often desirable when seeking structural variants.
By default, Bowtie 2 searches for both concordant and discordant
alignments, though searching for discordant alignments can be disabled
with the --no-discordant option.
Mixed mode: paired where possible, unpaired otherwise
If Bowtie 2 cannot find a paired-end alignment for a pair, by default it
will go on to look for unpaired alignments for the constituent mates.
This is called “mixed mode.” To disable mixed mode, set the --no-mixed
option.
Bowtie 2 runs a little faster in --no-mixed mode, but will only consider
alignment status of pairs per se, not individual mates.
Some SAM FLAGS describe paired-end properties
The SAM FLAGS field, the second field in a SAM record, has multiple bits
that describe the paired-end nature of the read and alignment. The first
(least significant) bit (1 in decimal, 0x1 in hexadecimal) is set if the
read is part of a pair. The second bit (2 in decimal, 0x2 in
hexadecimal) is set if the read is part of a pair that aligned in a
paired-end fashion. The fourth bit (8 in decimal, 0x8 in hexadecimal) is
set if the read is part of a pair and the other mate in the pair had at
least one valid alignment. The sixth bit (32 in decimal, 0x20 in
hexadecimal) is set if the read is part of a pair and the other mate in
the pair aligned to the Crick strand (or, equivalently, if the reverse
complement of the other mate aligned to the Watson strand). The seventh
bit (64 in decimal, 0x40 in hexadecimal) is set if the read is mate 1 in
a pair. The eighth bit (128 in decimal, 0x80 in hexadecimal) is set if
the read is mate 2 in a pair. See the SAM specification for a more
detailed description of the FLAGS field.
Some SAM optional fields describe more paired-end properties
The last several fields of each SAM record usually contain SAM optional
fields, which are simply tab-separated strings conveying additional
information about the reads and alignments. A SAM optional field is
formatted like this: “XP:i:1” where “XP” is the TAG, “i” is the TYPE
(“integer” in this case), and “1” is the VALUE. See the SAM
specification for details regarding SAM optional fields.
Mates can overlap, contain, or dovetail each other
The fragment and read lengths might be such that alignments for the two
mates from a pair overlap each other. Consider this example:
(For these examples, assume we expect mate 1 to align to the left of
mate 2.)
Mate 1: GCAGATTATATGAGTCAGCTACGATATTGTT
Mate 2: TGTTTGGGGTGACACATTACGCGTCTTTGAC
Reference: GCAGATTATATGAGTCAGCTACGATATTGTTTGGGGTGACACATTACGCGTCTTTGAC
It’s also possible, though unusual, for one mate alignment to contain
the other, as in these examples:
Mate 1: GCAGATTATATGAGTCAGCTACGATATTGTTTGGGGTGACACATTACGC
Mate 2: TGTTTGGGGTGACACATTACGC
Reference: GCAGATTATATGAGTCAGCTACGATATTGTTTGGGGTGACACATTACGCGTCTTTGAC
Mate 1: CAGCTACGATATTGTTTGGGGTGACACATTACGC
Mate 2: CTACGATATTGTTTGGGGTGAC
Reference: GCAGATTATATGAGTCAGCTACGATATTGTTTGGGGTGACACATTACGCGTCTTTGAC
And it’s also possible, though unusual, for the mates to “dovetail”,
with the mates seemingly extending “past” each other as in this example:
Mate 1: GTCAGCTACGATATTGTTTGGGGTGACACATTACGC
Mate 2: TATGAGTCAGCTACGATATTGTTTGGGGTGACACAT
Reference: GCAGATTATATGAGTCAGCTACGATATTGTTTGGGGTGACACATTACGCGTCTTTGAC
In some situations, it’s desirable for the aligner to consider all these
cases as “concordant” as long as other paired-end constraints are not
violated. Bowtie 2’s default behavior is to consider overlapping and
containing as being consistent with concordant alignment. By default,
dovetailing is considered inconsistent with concordant alignment.
These defaults can be overridden. Setting --no-overlap causes Bowtie 2
to consider overlapping mates as non-concordant. Setting --no-contain
causes Bowtie 2 to consider cases where one mate alignment contains the
other as non-concordant. Setting --dovetail causes Bowtie 2 to consider
cases where the mate alignments dovetail as concordant.
Reporting
The reporting mode governs how many alignments Bowtie 2 looks for, and
how to report them. Bowtie 2 has three distinct reporting modes. The
default reporting mode is similar to the default reporting mode of many
other read alignment tools, including BWA. It is also similar to Bowtie
1’s -M alignment mode.
In general, when we say that a read has an alignment, we mean that it
has a valid alignment. When we say that a read has multiple alignments,
we mean that it has multiple alignments that are valid and distinct from
one another.
Distinct alignments map a read to different places
Two alignments for the same individual read are “distinct” if they map
the same read to different places. Specifically, we say that two
alignments are distinct if there are no alignment positions where a
particular read offset is aligned opposite a particular reference offset
in both alignments with the same orientation. E.g. if the first
alignment is in the forward orientation and aligns the read character at
read offset 10 to the reference character at chromosome 3, offset
3,445,245, and the second alignment is also in the forward orientation
and also aligns the read character at read offset 10 to the reference
character at chromosome 3, offset 3,445,245, they are not distinct
alignments.
Two alignments for the same pair are distinct if either the mate 1s in
the two paired-end alignments are distinct or the mate 2s in the two
alignments are distinct or both.
Default mode: search for multiple alignments, report the best one
By default, Bowtie 2 searches for distinct, valid alignments for each
read. When it finds a valid alignment, it generally will continue to
look for alignments that are nearly as good or better. It will
eventually stop looking, either because it exceeded a limit placed on
search effort (see -D and -R) or because it already knows all it needs
to know to report an alignment. Information from the best alignments are
used to estimate mapping quality (the MAPQ SAM field) and to set SAM
optional fields, such as AS:i and XS:i. Bowtie 2 does not guarantee that
the alignment reported is the best possible in terms of alignment score.
See also: -D, which puts an upper limit on the number of dynamic
programming problems (i.e. seed extensions) that can “fail” in a row
before Bowtie 2 stops searching. Increasing -D makes Bowtie 2 slower,
but increases the likelihood that it will report the correct alignment
for a read that aligns many places.
See also: -R, which sets the maximum number of times Bowtie 2 will
“re-seed” when attempting to align a read with repetitive seeds.
Increasing -R makes Bowtie 2 slower, but increases the likelihood that
it will report the correct alignment for a read that aligns many places.
-k mode: search for one or more alignments, report each
In -k mode, Bowtie 2 searches for up to N distinct, valid alignments for
each read, where N equals the integer specified with the -k parameter.
That is, if -k 2 is specified, Bowtie 2 will search for at most 2
distinct alignments. It reports all alignments found, in descending
order by alignment score. The alignment score for a paired-end alignment
equals the sum of the alignment scores of the individual mates. Each
reported read or pair alignment beyond the first has the SAM ‘secondary’
bit (which equals 256) set in its FLAGS field. See the SAM specification
for details.
Bowtie 2 does not “find” alignments in any specific order, so for reads
that have more than N distinct, valid alignments, Bowtie 2 does not
guarantee that the N alignments reported are the best possible in terms
of alignment score. Still, this mode can be effective and fast in
situations where the user cares more about whether a read aligns (or
aligns a certain number of times) than where exactly it originated.
-a mode: search for and report all alignments
-a mode is similar to -k mode except that there is no upper limit on the
number of alignments Bowtie 2 should report. Alignments are reported in
descending order by alignment score. The alignment score for a
paired-end alignment equals the sum of the alignment scores of the
individual mates. Each reported read or pair alignment beyond the first
has the SAM ‘secondary’ bit (which equals 256) set in its FLAGS field.
See the SAM specification for details.
Some tools are designed with this reporting mode in mind. Bowtie 2 is
not! For very large genomes, this mode is very slow.
Randomness in Bowtie 2
Bowtie 2’s search for alignments for a given read is “randomized.” That
is, when Bowtie 2 encounters a set of equally-good choices, it uses a
pseudo-random number to choose. For example, if Bowtie 2 discovers a set
of 3 equally-good alignments and wants to decide which to report, it
picks a pseudo-random integer 0, 1 or 2 and reports the corresponding
alignment. Arbitrary choices can crop up at various points during
alignment.
The pseudo-random number generator is re-initialized for every read, and
the seed used to initialize it is a function of the read name,
nucleotide string, quality string, and the value specified with --seed.
If you run the same version of Bowtie 2 on two reads with identical
names, nucleotide strings, and quality strings, and if --seed is set the
same for both runs, Bowtie 2 will produce the same output; i.e., it will
align the read to the same place, even if there are multiple equally
good alignments. This is intuitive and desirable in most cases. Most
users expect Bowtie to produce the same output when run twice on the
same input.
However, when the user specifies the --non-deterministic option, Bowtie
2 will use the current time to re-initialize the pseudo-random number
generator. When this is specified, Bowtie 2 might report different
alignments for identical reads. This is counter-intuitive for some
users, but might be more appropriate in situations where the input
consists of many identical reads.
Multiseed heuristic
To rapidly narrow the number of possible alignments that must be
considered, Bowtie 2 begins by extracting substrings (“seeds”) from the
read and its reverse complement and aligning them in an ungapped fashion
with the help of the FM Index. This is “multiseed alignment” and it is
similar to what Bowtie 1 does, except Bowtie 1 attempts to align the
entire read this way.
This initial step makes Bowtie 2 much faster than it would be without
such a filter, but at the expense of missing some valid alignments. For
instance, it is possible for a read to have a valid overall alignment
but to have no valid seed alignments because each potential seed
alignment is interrupted by too many mismatches or gaps.
The trade-off between speed and sensitivity/accuracy can be adjusted by
setting the seed length (-L), the interval between extracted seeds (-i),
and the number of mismatches permitted per seed (-N). For more sensitive
alignment, set these parameters to (a) make the seeds closer together,
(b) make the seeds shorter, and/or (c) allow more mismatches. You can
adjust these options one-by-one, though Bowtie 2 comes with some useful
combinations of options prepackaged as “preset options.”
-D and -R are also options that adjust the trade-off between speed and
sensitivity/accuracy.
FM Index memory footprint
Bowtie 2 uses the FM Index to find ungapped alignments for seeds. This
step accounts for the bulk of Bowtie 2’s memory footprint, as the FM
Index itself is typically the largest data structure used. For instance,
the memory footprint of the FM Index for the human genome is about 3.2
gigabytes of RAM.
Ambiguous characters
Non-whitespace characters besides A, C, G or T are considered
“ambiguous.” N is a common ambiguous character that appears in reference
sequences. Bowtie 2 considers all ambiguous characters in the reference
(including IUPAC nucleotide codes) to be Ns.
Bowtie 2 allows alignments to overlap ambiguous characters in the
reference. An alignment position that contains an ambiguous character in
the read, reference, or both, is penalized according to --np. --n-ceil
sets an upper limit on the number of positions that may contain
ambiguous reference characters in a valid alignment. The optional field
XN:i reports the number of ambiguous reference characters overlapped by
an alignment.
Note that the multiseed heuristic cannot find _seed_ alignments that
overlap ambiguous reference characters. For an alignment overlapping an
ambiguous reference character to be found, it must have one or more seed
alignments that do not overlap ambiguous reference characters.
Presets: setting many settings at once
Bowtie 2 comes with some useful combinations of parameters packaged into
shorter “preset” parameters. For example, running Bowtie 2 with the
--very-sensitive option is the same as running with options:
-D 20 -R 3 -N 0 -L 20 -i S,1,0.50. The preset options that come with
Bowtie 2 are designed to cover a wide area of the
speed/sensitivity/accuracy trade-off space, with the presets ending in
fast generally being faster but less sensitive and less accurate, and
the presets ending in sensitive generally being slower but more
sensitive and more accurate. See the documentation for the preset
options for details.
Filtering
Some reads are skipped or “filtered out” by Bowtie 2. For example, reads
may be filtered out because they are extremely short or have a high
proportion of ambiguous nucleotides. Bowtie 2 will still print a SAM
record for such a read, but no alignment will be reported and the YF:i
SAM optional field will be set to indicate the reason the read was
filtered.
- YF:Z:LN: the read was filtered because it had length less than or
equal to the number of seed mismatches set with the -N option.
- YF:Z:NS: the read was filtered because it contains a number of
ambiguous characters (usually N or .) greater than the ceiling
specified with --n-ceil.
- YF:Z:SC: the read was filtered because the read length and the match
bonus (set with --ma) are such that the read can’t possibly earn an
alignment score greater than or equal to the threshold set with
--score-min
- YF:Z:QC: the read was filtered because it was marked as failing
quality control and the user specified the --qc-filter option. This
only happens when the input is in Illumina’s QSEQ format (i.e. when
--qseq is specified) and the last (11th) field of the read’s QSEQ
record contains 1.
If a read could be filtered for more than one reason, the value YF:Z
flag will reflect only one of those reasons.
Alignment summary
When Bowtie 2 finishes running, it prints messages summarizing what
happened. These messages are printed to the “standard error” (“stderr”)
filehandle. For datasets consisting of unpaired reads, the summary might
look like this:
20000 reads; of these:
20000 (100.00%) were unpaired; of these:
1247 (6.24%) aligned 0 times
18739 (93.69%) aligned exactly 1 time
14 (0.07%) aligned >1 times
93.77% overall alignment rate
For datasets consisting of pairs, the summary might look like this:
10000 reads; of these:
10000 (100.00%) were paired; of these:
650 (6.50%) aligned concordantly 0 times
8823 (88.23%) aligned concordantly exactly 1 time
527 (5.27%) aligned concordantly >1 times
----
650 pairs aligned concordantly 0 times; of these:
34 (5.23%) aligned discordantly 1 time
----
616 pairs aligned 0 times concordantly or discordantly; of these:
1232 mates make up the pairs; of these:
660 (53.57%) aligned 0 times
571 (46.35%) aligned exactly 1 time
1 (0.08%) aligned >1 times
96.70% overall alignment rate
The indentation indicates how subtotals relate to totals.
Wrapper scripts
The bowtie2, bowtie2-build and bowtie2-inspect executables are actually
wrapper scripts that call binary programs as appropriate. The wrappers
shield users from having to distinguish between “small” and “large”
index formats, discussed briefly in the following section. Also, the
bowtie2 wrapper provides some key functionality, like the ability to
handle compressed inputs, and the functionality for --un, --al and
related options.
It is recommended that you always run the bowtie2 wrappers and not run
the binaries directly.
Small and large indexes
bowtie2-build can index reference genomes of any size. For genomes less
than about 4 billion nucleotides in length, bowtie2-build builds a
“small” index using 32-bit numbers in various parts of the index. When
the genome is longer, bowtie2-build builds a “large” index using 64-bit
numbers. Small indexes are stored in files with the .bt2 extension, and
large indexes are stored in files with the .bt2l extension. The user
need not worry about whether a particular index is small or large; the
wrapper scripts will automatically build and use the appropriate index.
Performance tuning
1. If your computer has multiple processors/cores, use -p
The -p option causes Bowtie 2 to launch a specified number of
parallel search threads. Each thread runs on a different
processor/core and all threads find alignments in parallel,
increasing alignment throughput by approximately a multiple of the
number of threads (though in practice, speedup is somewhat worse
than linear).
2. If reporting many alignments per read, try reducing
bowtie2-build --offrate
If you are using -k or -a options and Bowtie 2 is reporting many
alignments per read, using an index with a denser SA sample can
speed things up considerably. To do this, specify a
smaller-than-default -o/--offrate value when running bowtie2-build.
A denser SA sample yields a larger index, but is also particularly
effective at speeding up alignment when many alignments are reported
per read.
3. If bowtie2 “thrashes”, try increasing bowtie2-build --offrate
If bowtie2 runs very slowly on a relatively low-memory computer, try
setting -o/--offrate to a _larger_ value when building the index.
This decreases the memory footprint of the index.
Command Line
Setting function options
Some Bowtie 2 options specify a function rather than an individual
number or setting. In these cases the user specifies three parameters:
(a) a function type F, (b) a constant term B, and (c) a coefficient A.
The available function types are constant (C), linear (L), square-root
(S), and natural log (G). The parameters are specified as F,B,A - that
is, the function type, the constant term, and the coefficient are
separated by commas with no whitespace. The constant term and
coefficient may be negative and/or floating-point numbers.
For example, if the function specification is L,-0.4,-0.6, then the
function defined is:
f(x) = -0.4 + -0.6 * x
If the function specification is G,1,5.4, then the function defined is:
f(x) = 1.0 + 5.4 * ln(x)
See the documentation for the option in question to learn what the
parameter x is for. For example, in the case if the --score-min option,
the function f(x) sets the minimum alignment score necessary for an
alignment to be considered valid, and x is the read length.
Usage
bowtie2 [options]* -x <bt2-idx> {-1 <m1> -2 <m2> | -U <r> | --interleaved <i>} -S [<sam>]
Main arguments
-x <bt2-idx>
The basename of the index for the reference genome. The basename is the
name of any of the index files up to but not including the final .1.bt2
/ .rev.1.bt2 / etc. bowtie2 looks for the specified index first in the
current directory, then in the directory specified in the
BOWTIE2_INDEXES environment variable.
-1 <m1>
Comma-separated list of files containing mate 1s (filename usually
includes _1), e.g. -1 flyA_1.fq,flyB_1.fq. Sequences specified with this
option must correspond file-for-file and read-for-read with those
specified in <m2>. Reads may be a mix of different lengths. If - is
specified, bowtie2 will read the mate 1s from the “standard in” or
“stdin” filehandle.
-2 <m2>
Comma-separated list of files containing mate 2s (filename usually
includes _2), e.g. -2 flyA_2.fq,flyB_2.fq. Sequences specified with this
option must correspond file-for-file and read-for-read with those
specified in <m1>. Reads may be a mix of different lengths. If - is
specified, bowtie2 will read the mate 2s from the “standard in” or
“stdin” filehandle.
-U <r>
Comma-separated list of files containing unpaired reads to be aligned,
e.g. lane1.fq,lane2.fq,lane3.fq,lane4.fq. Reads may be a mix of
different lengths. If - is specified, bowtie2 gets the reads from the
“standard in” or “stdin” filehandle.
-S <sam>
File to write SAM alignments to. By default, alignments are written to
the “standard out” or “stdout” filehandle (i.e. the console).
Options
Input options
-q
Reads (specified with <m1>, <m2>, <s>) are FASTQ files. FASTQ files
usually have extension .fq or .fastq. FASTQ is the default format. See
also: --solexa-quals and --int-quals.
--interleaved
Reads interleaved FASTQ files where the first two records (8 lines)
represent a mate pair.
--tab5
Each read or pair is on a single line. An unpaired read line is
[name]\t[seq]\t[qual]\n. A paired-end read line is
[name]\t[seq1]\t[qual1]\t[seq2]\t[qual2]\n. An input file can be a mix
of unpaired and paired-end reads and Bowtie 2 recognizes each according
to the number of fields, handling each as it should.
--tab6
Similar to --tab5 except, for paired-end reads, the second end can have
a different name from the first:
[name1]\t[seq1]\t[qual1]\t[name2]\t[seq2]\t[qual2]\n
--qseq
Reads (specified with <m1>, <m2>, <s>) are QSEQ files. QSEQ files
usually end in _qseq.txt. See also: --solexa-quals and --int-quals.
-f
Reads (specified with <m1>, <m2>, <s>) are FASTA files. FASTA files
usually have extension .fa, .fasta, .mfa, .fna or similar. FASTA files
do not have a way of specifying quality values, so when -f is set, the
result is as if --ignore-quals is also set.
-r
Reads (specified with <m1>, <m2>, <s>) are files with one input sequence
per line, without any other information (no read names, no qualities).
When -r is set, the result is as if --ignore-quals is also set.
-F k:<int>,i:<int>
Reads are substrings (k-mers) extracted from a FASTA file <s>.
Specifically, for every reference sequence in FASTA file <s>, Bowtie 2
aligns the k-mers at offsets 1, 1+i, 1+2i, … until reaching the end of
the reference. Each k-mer is aligned as a separate read. Quality values
are set to all Is (40 on Phred scale). Each k-mer (read) is given a name
like <sequence>_<offset>, where <sequence> is the name of the FASTA
sequence it was drawn from and <offset> is its 0-based offset of origin
with respect to the sequence. Only single k-mers, i.e. unpaired reads,
can be aligned in this way.
-c
The read sequences are given on command line. I.e. <m1>, <m2> and
<singles> are comma-separated lists of reads rather than lists of read
files. There is no way to specify read names or qualities, so -c also
implies --ignore-quals.
-s/--skip <int>
Skip (i.e. do not align) the first <int> reads or pairs in the input.
-u/--qupto <int>
Align the first <int> reads or read pairs from the input (after the
-s/--skip reads or pairs have been skipped), then stop. Default: no
limit.
-5/--trim5 <int>
Trim <int> bases from 5’ (left) end of each read before alignment
(default: 0).
-3/--trim3 <int>
Trim <int> bases from 3’ (right) end of each read before alignment
(default: 0).
--phred33
Input qualities are ASCII chars equal to the Phred quality plus 33. This
is also called the “Phred+33” encoding, which is used by the very latest
Illumina pipelines.
--phred64
Input qualities are ASCII chars equal to the Phred quality plus 64. This
is also called the “Phred+64” encoding.
--solexa-quals
Convert input qualities from Solexa (which can be negative) to Phred
(which can’t). This scheme was used in older Illumina GA Pipeline
versions (prior to 1.3). Default: off.
--int-quals
Quality values are represented in the read input file as space-separated
ASCII integers, e.g., 40 40 30 40…, rather than ASCII characters, e.g.,
II?I…. Integers are treated as being on the Phred quality scale unless
--solexa-quals is also specified. Default: off.
Preset options in --end-to-end mode
--very-fast
Same as: -D 5 -R 1 -N 0 -L 22 -i S,0,2.50
--fast
Same as: -D 10 -R 2 -N 0 -L 22 -i S,0,2.50
--sensitive
Same as: -D 15 -R 2 -L 22 -i S,1,1.15 (default in --end-to-end mode)
--very-sensitive
Same as: -D 20 -R 3 -N 0 -L 20 -i S,1,0.50
Preset options in --local mode
--very-fast-local
Same as: -D 5 -R 1 -N 0 -L 25 -i S,1,2.00
--fast-local
Same as: -D 10 -R 2 -N 0 -L 22 -i S,1,1.75
--sensitive-local
Same as: -D 15 -R 2 -N 0 -L 20 -i S,1,0.75 (default in --local mode)
--very-sensitive-local
Same as: -D 20 -R 3 -N 0 -L 20 -i S,1,0.50
Alignment options
-N <int>
Sets the number of mismatches to allowed in a seed alignment during
multiseed alignment. Can be set to 0 or 1. Setting this higher makes
alignment slower (often much slower) but increases sensitivity. Default:
0.
-L <int>
Sets the length of the seed substrings to align during multiseed
alignment. Smaller values make alignment slower but more sensitive.
Default: the --sensitive preset is used by default, which sets -L to 20
both in --end-to-end mode and in --local mode.