---

A headless workflow execution engine for bioinformatics that supports local, cloud, and HPC.

Request Feature · Report Bug · ⭐ Consider starring the repo! ⭐

Overview

crankshaft is a headless workflow execution engine written in Rust: it's being developed in coordination with the sprocket project with the goal of enabling large-scale bioinformatics analyses. There is no associated crankshaft command line tool—the end user is really engine developers who want to include it as a core task execution library in their own command line tools.

Guiding Principles

• crankshaft aims to be a high-performance workflow execution engine capable of concurrently managing and executing upwards of 20,000 concurrent tasks. The core focus is enabling middle- to large-scale bioinformatics analyses, though it can also be used to design smaller scale execution engines.
• crankshaft is headless, which means that it doesn't do anything on its own; in fact, it must be driven by some external orchestration code. This allows the crankshaft library itself to focus on performance improvements that can be enjoyed across the entire community.
• crankshaft is developed independently of any particular workflow language. Though it's part of the Sprocket project, it's not based on WDL, and, in theory, multiple frontends based on different workflow languages can exist (and we hope this is the case)!

📚 Getting Started

Installation

To use crankshaft, you'll need to install Rust. We recommend using rustup to accomplish this. Once Rust is installed, you can create a new project and add the latest version of crankshaft using the following command.

cargo add crankshaft

Once you've added crankshaft to your dependencies, you should head over to the /examples to see how you can use the library in your projects.

🖥️ Development

To bootstrap a development environment, please use the following commands.

# Clone the repository
git clone git@github.com:stjude-rust-labs/crankshaft.git
cd crankshaft

# Build the crate in release mode
cargo build --release

🚧️ Tests

Before submitting any pull requests, please make sure the code passes the following checks (from the root directory).

# Run the project's tests.
cargo test --all-features

# Run the tests for the examples.
cargo test --examples --all-features

# Ensure the project doesn't have any linting warnings.
cargo clippy --all-features

# Ensure the project passes `cargo fmt`.
cargo fmt --check

# Ensure the docs build.
cargo doc

🤝 Contributing

Contributions, issues and feature requests are welcome! Feel free to check issues page.

📝 License

This project is licensed as either Apache 2.0 or MIT at your discretion. Additionally, please see the disclaimer that applies to all crates and command line tools made available by St. Jude Rust Labs.

Copyright © 2024-Present St. Jude Children's Research Hospital.