Integration (or not?) with dahak-bespin

[charlesreid1]

dahak-taco: https://github.com/dahak-metagenomics/dahak-taco

dahak-bespin: https://github.com/dahak-metagenomics/dahak-bespin

What is the difference between dahak-taco and dahak-bespin, and how do they relate?

• dahak-taco is a command line tool that wraps snakemake workflows, providing users with a simple interface for running complex workflows, while not taking away control over the workflow configuration and parameters.
• dahak-bespin is a command line tool that spins up a cluster on AWS. this requests N nodes of type X, builds a VPC, hooks them all up to the VPC, and adds a spy node for monitoring the status of the N workflow nodes. it basically turns setting up an entire AWS cluster into a single bash command.

These two tools are totally separate from each other - dahak-bespin could potentially have other uses, and dahak-taco does not always expect an AWS cluster set up by dahak-bespin (it's just one possibility).

However, they do complement each other - dahak-bespin can be used to create a cluster, dahak-taco can be used to wrap a Snakemake workflow, and by passing the right configuration file Snakemake will use the cluster created by dahak-bespin to run jobs.

An alternative mash-up would be to use dahak-taco with a kubernetes cluster. Snakemake has built-in support for submitting jobs via kubernetes cluster, so taco would just have to wrap that functionality to allow the user to use dahak-taco with a kubernetes cluster.

The end result: user can run one dahak-bespin command to spin up a cluster in the cloud (or, you know, run Q commands to spin up a kubernetes cluster on AWS or wherever) and then run one command to run their Snakemake workflow via taco, using their custom config/params, using their cluster in the cloud, and then they can put their feet up and sip a beverage of their choice while they wait for their workflow to finish.