Lathe

A tool for generating bacterial genomes from metagenomes with Nanopore long read sequencing

Installation

First, install miniconda3

Then install snakemake. This can be done with the following.

conda install snakemake
snakemake --version #please ensure this is >=5.4.3

Next, clone this github directory to some location where it can be stored permanently. Remember to keep it updated with git pull.

git clone https://github.com/elimoss/lathe.git

Instructions to enable cluster execution with SLURM can be found at https://github.com/bhattlab/slurm .

Typical installation time: 5-10 minutes.

Change as of 2021-02-03

Lathe has been adapted to run on multiple samples simultaneously, instead of one sample per snakemake command. The pipeline can now take in either .fast5 raw data from a nanopore run, or basecalled fastq files. The config file has been changed to reflect this. You now provide sample information and datasets in a tab-delimited file, and indicate this file in the config.yaml. Provide this file to the file_names_txt argument in the configfile.

file_names_txt is a 2 or 3 column tsv with the following columns
SAMPLE_NAME     FAST5/FASTQ_READS      SHORT_READS_1,SHORT_READS_2
sample name in the first column will be used to name ouptut
the second column can be a directory containing fast5 files (the output of a nanopore run)
  -OR- a single fastq file containing basecalled data 
Optionally, a short read sequencing dataset can be provided in the third column, 
  with pairs separated by a comma. If this option is selected, short read
  polishing will be turned on. 

Inputs

Alter config.yaml to provide the following:

• file_names_txt: Tab delimited file describing sample names and input datasets. See config.yaml for a description.

• flowcell: flowcell code, e.g. FLO-MIN106, passed to basecaller

• kit: kit code, e.g. SQK-LSK109, passed to basecaller

• genome_size: Estimated genome size, e.g. 50m, passed to Canu.

• singularity: location (including on the internet) of a singularity image to be used for the workflow. Don't change this.

• use_grid: should Canu execute in distributed mode on a cluster?

• grid_options: Extra options for execution on a cluster

• canu_args: Extra options for Canu

• skip_circularization: Should circularization be omitted from the workflow?

Lathe uses the Flye assembler by default. For Canu, please specify 'canu' for the assembler parameter in the config. For cluster Canu execution, please note: if set to True, you will need to install Canu, e.g. conda install -c conda-forge -c bioconda Canu=1.8 as well as provide any additional required parameters for your job scheduler in the config.yaml file. Please see the example config file. When executing on a cluster, Canu will appear to fail, as the first process does not produce an assembly and instead spawns subsequent jobs on the cluster. Don't worry, just re-run Lathe when the assembly completes.

To execute please run the following. Please note, you must substitute a parent directory containing all of your data and working directories for /labs/.

snakemake --use-singularity --singularity-args '--bind /labs/,/scg/,/home/ ' -s /path/to/lathe/Snakefile \
--configfile path/to/modified_config.yaml --restart-times 0 --keep-going --latency-wait 30
# --profile scg #enable cluster support, highly recommended.  See above.

Outputs

The outputs generated by this workflow will look like the following:

samplename/
├── 0.basecall
│   ├── samplename.fq
│   └── nanoplots
├── 1.assemble
│   ├── samplename_merged.fasta
│   ├── samplename_raw_assembly.fa
│   ├── samplename_raw_assembly.fa.amb
│   ├── samplename_raw_assembly.fa.ann
│   ├── samplename_raw_assembly.fa.bwt
│   ├── samplename_raw_assembly.fa.fai
│   ├── samplename_raw_assembly.fa.pac
│   ├── samplename_raw_assembly.fa.paf
│   ├── samplename_raw_assembly.fa.sa
│   ├── assemble_100m (if specified)
│   └── assemble_250m (if specified)
├── 2.polish
│   ├── samplename_polished.corrected.fasta
│   ├── samplename_polished.fasta
│   ├── samplename_polished.fasta.bam
│   ├── samplename_polished.fasta.bam.bai
│   ├── samplename_polished.fasta.fai
│   ├── samplename_polished.fasta.misassemblies.tsv
│   ├── medaka (if specified)
│   ├── pilon (if specified)
│   └── racon (if specified)
├── 3.circularization
│   ├── 1.candidate_genomes
│   ├── 2.circularization
│   ├── 3.circular_sequences #circularized genomes
│   ├── 4.samplename_circularized.corrected.fasta
│   ├── 4.samplename_circularized.fasta
│   ├── 4.samplename_circularized.fasta.bam
│   ├── 4.samplename_circularized.fasta.bam.bai
│   ├── 4.samplename_circularized.fasta.fai
│   └── 4.samplename_circularized.fasta.misassemblies.tsv
└── 5.final
    ├── samplename_final.fa
    └── samplename_final.fa.fai

Tutorial

The tutorial can be run using the provided config file and input data within the tutorial folder. This tutorial uses pre-basecalled long read data (to reduce total file sizes) and performs assembly with Flye and short read polishing. To reduce runtime, this tutorial skips basecalling, long read polishing, and circularization steps. With cluster execution enabled, this tutorial should be completed in under 6 hours. Successful completion will be indicated by the presence of a atcc_tutorial_final.fa file in the 5.final directory. To run the tutorial:

• unzip the short read (tutorial/inputdata/atcc_100000_sr.fastq.gz) and long read (tutorial/atcc_tutorial/0.basecall/atcc_tutorial.fq.gz) data

• edit the config file to provide the absolute path to the short read input data (atcc_100000_sr.fastq)

• run Lathe using the command:

snakemake --use-singularity --singularity-args '--bind /yourrootdirectories/ ' -s /path/to/lathe/Snakefile \
--configfile path/to/config_nobasecalling.yaml --restart-times 0 --keep-going --latency-wait 30
# --profile clusterconfiguration #enable cluster support, highly recommended.  See above.