0.11.1-rc.5

Overview

This release adds support for AWS S3 wildcards and Funnel deployments in Kubernetes!

Getting Started

See step by step guides on using Funnel to run and track tasks on the documentation page at https://ohsu-comp-bio.github.io/funnel

Installing

Homebrew

Funnel is available on macOS through Homebrew:

brew tap ohsu-comp-bio/formula
brew install funnel@0.11.1

• Homebrew Formula: https://github.com/ohsu-comp-bio/homebrew-formula
• Docker image: https://quay.io/repository/ohsu-comp-bio/funnel?tab=tags

Install Script

Alternatively, Funnel may be installed with the installer script:

/bin/bash -c "$(curl -fsSL https://github.com/ohsu-comp-bio/funnel/releases/download/0.11.1-rc.5/install.sh)" -- --verbose --release 0.11.1-rc.5

What's Changed

• AWS S3 wildcard support
• K8s support

Full Changelog: 0.11.0...0.11.1-rc.4

0.11.1-rc.4

Overview

This release adds support for AWS S3 wildcards and Funnel deployments in Kubernetes!

Getting Started

See step by step guides on using Funnel to run and track tasks on the documentation page at https://ohsu-comp-bio.github.io/funnel

Installing

Homebrew

Funnel is available on macOS through Homebrew:

brew tap ohsu-comp-bio/formula
brew install funnel@0.11.1

• Homebrew Formula: https://github.com/ohsu-comp-bio/homebrew-formula
• Docker image: https://quay.io/repository/ohsu-comp-bio/funnel?tab=tags

Install Script

Alternatively, Funnel may be installed with the installer script:

/bin/bash -c "$(curl -fsSL https://github.com/ohsu-comp-bio/funnel/releases/download/0.11.1-rc.4/install.sh)" -- --verbose --release 0.11.1-rc.4

What's Changed

• AWS S3 wildcard support
• K8s support

Full Changelog: 0.11.0...0.11.1-rc.4

0.11.1-rc.3

Overview

This release adds support for AWS S3 wildcards and Funnel deployments in Kubernetes!

Getting Started

See step by step guides on using Funnel to run and track tasks on the documentation page at https://ohsu-comp-bio.github.io/funnel

Installing

Homebrew

Funnel is available on macOS through Homebrew:

brew tap ohsu-comp-bio/formula
brew install funnel@0.11.1

• Homebrew Formula: https://github.com/ohsu-comp-bio/homebrew-formula
• Docker image: https://quay.io/repository/ohsu-comp-bio/funnel?tab=tags

Install Script

Alternatively, Funnel may be installed with the installer script:

/bin/bash -c "$(curl -fsSL https://github.com/ohsu-comp-bio/funnel/releases/download/0.11.1-rc.3/install.sh)" -- --verbose --release 0.11.1-rc.3

What's Changed

• AWS S3 wildcard support
• K8s support

Full Changelog: 0.11.0...0.11.1-rc.3

0.11.1-rc.2

Overview

This release adds support for custom container engines and commands that override the default docker commands (#962).

The following configuration example demonstrates how to specify what container engine to run.

Custom Docker Example

Worker:
  Container:
    DriverCommand: docker
    RunCommand: run -i --security-opt seccomp=unconfined --security-opt systempaths=unconfined --name {{.Name}} {{range .Volumes}}-v {{.HostPath}}:{{.ContainerPath}}:rw{{end}} {{.Image}} {{.Command}}
    PullCommand: pull {{.Image}}
    StopCommand: stop {{.Name}}

To run this example, save the following into a config file (e.g. docker.yaml) and pass it to Funnel.

1. Start Server

➜ funnel server run --config docker.yaml

2. Create Task

➜ funnel examples hello-world > hello-world.json

➜ funnel task create hello-world.json
# cqi53isbajf230tr0gpg

3. Verify System Logs

➜ funnel tasks get cqi53isbajf230tr0gpg | grep cmd
# "level='info' msg='Running command' ... cmd='docker run -i --security-opt seccomp=unconfined --security-opt systempaths=unconfined ... alpine echo hello world'" ✅

Default Configuration

The default configurations are as follows (and are used when no other configuration is provided):

Worker:
  Container:
    DriverCommand: docker
    RunCommand: run -i --read-only --rm --name {{.Name}} -w {{.Workdir}} {{range .Volumes}}-v {{.HostPath}}:{{.ContainerPath}}:rw {{end}} {{.Image}} {{.Command}}
    PullCommand: pull {{.Image}}
    StopCommand: rm -f {{.Name}}

Getting Started

See step by step guides on using Funnel to run and track tasks on the documentation page at https://ohsu-comp-bio.github.io/funnel

Installing

Homebrew

Funnel is available on macOS through Homebrew:

brew tap ohsu-comp-bio/formula
brew install funnel@0.11.1

• Homebrew Formula: https://github.com/ohsu-comp-bio/homebrew-formula
• Docker image: https://quay.io/repository/ohsu-comp-bio/funnel?tab=tags

Install Script

Alternatively, Funnel may be installed with the installer script:

/bin/bash -c "$(curl -fsSL https://github.com/ohsu-comp-bio/funnel/releases/download/0.11.1-rc.2/install.sh)" -- --verbose --release 0.11.1-rc.2

What's Changed

• Configurable command templating

Full Changelog: 0.11.0...0.11.1-rc.2

0.11.0

Overview

This release adds support for version 1.1 of the TES schema as part of the GA4GH community.

Getting Started

See step by step guides on using Funnel to run and track tasks on the documentation page at https://ohsu-comp-bio.github.io/funnel

Installing

Homebrew

Funnel is available on macOS through Homebrew:

brew tap ohsu-comp-bio/formula
brew install funnel

• Homebrew Formula: https://github.com/ohsu-comp-bio/homebrew-formula
• Docker image: https://quay.io/repository/ohsu-comp-bio/funnel?tab=tags

Install Script

Alternatively, Funnel may be installed with the installer script. By default it installs the latest release into the $HOME/.local/bin directory:

/bin/bash -c "$(curl -fsSL https://github.com/ohsu-comp-bio/funnel/releases/download/0.11.0/install.sh)"

What's Changed

• TESv1.1 support
• Added initial Nextflow support
• Updated Homebrew installation and Docker images

Credits

Developed by the Ellrott Lab at Oregon Health & Science University.

Development was lead by Kyle Ellrott (@kellrott) with contributions from Alex Kanitz (@uniqueg), Adam Struck (@adamstruck), Adam Novak (@adamnovak), and Liam Beckman (@lbeckman314) with Data Analysis and HPC guidance by Jordan Lee (@jordan2lee).

Special thanks to Lakshya Garg (@LakiG) and Alex for their work on the TES Compliance Suite and to @bentsherman and @vsmalladi for continuing collaboration with Nextflow and TES on Azure.

Changelog

• PR #749 Add support for TES v1.1
• PR #761 Add Nextflow (nf-canary) tests to Funnel
• PR #724 Create dependabot.yml

Full Changelog: 0.10.1...0.11.0

0.11.0-rc.5

Overview

This release adds support for version 1.1 of the TES schema as part of the GA4GH community.

Getting Started

See step by step guides on using Funnel to run and track tasks on the documentation page at https://ohsu-comp-bio.github.io/funnel

Installing

Homebrew

Funnel is available on macOS through Homebrew:

brew tap ohsu-comp-bio/formula
brew install funnel

• Homebrew Formula: https://github.com/ohsu-comp-bio/homebrew-formula
• Docker image: https://quay.io/repository/ohsu-comp-bio/funnel?tab=tags

Install Script

Alternatively, Funnel may be installed with the installer script. By default it installs the latest release into the $HOME/.local/bin directory:

/bin/bash -c "$(curl -fsSL https://github.com/ohsu-comp-bio/funnel/releases/download/0.11.0-rc.5/install.sh)" -- 0.11.0-rc.5

What's Changed

• TESv1.1 support
• Added initial Nextflow support
• Updated Homebrew installation and Docker images

Credits

Developed by the Ellrott Lab at Oregon Health & Science University.

Development was lead by Kyle Ellrott (@kellrott) with contributions from Alex Kanitz (@uniqueg), Adam Struck (@adamstruck), Adam Novak (@adamnovak), and Liam Beckman (@lbeckman314) with Data Analysis and HPC guidance by Jordan Lee (@jordan2lee). Special thanks to Lakshya Garg (@LakiG) and Alex for their work on the TES Compliance Suite.

Changelog

• PR #761 Add Nextflow (nf-canary) tests to Funnel
• PR #724 Create dependabot.yml

Full Changelog: 0.10.1...0.11.0

0.10.1

• PR #647 FIX: Open File leak
• PR #636 Updates for version 0.10.0
• PR #635 add .dockerignore
• PR #633 Adding new docker build
• PR #632 storage: add s3 presigned url support to https backend
• PR #626 Kubernetes Backend; Go Module Support
• PR #625 AWS Batch: limit methods used for resource creation
• PR #623 storage/s3: added sse-c and sse-kms support
• PR #619 worker: added config to limit parallel uploads/downloads
• PR #617 Optimize TaskStateCounts aggregate pipeline
• PR #613 docker: change base image tag; goreleaser: fix deprecation notices
• PR #612 database/mongodb: copy session for each op and maintain a connection pool
• PR #610 React web dashboard

0.10.0

• PR #636 Updates for version 0.10.0
• PR #635 add .dockerignore
• PR #633 Adding new docker build
• PR #632 storage: add s3 presigned url support to https backend
• PR #626 Kubernetes Backend; Go Module Support
• PR #625 AWS Batch: limit methods used for resource creation
• PR #623 storage/s3: added sse-c and sse-kms support
• PR #619 worker: added config to limit parallel uploads/downloads
• PR #617 Optimize TaskStateCounts aggregate pipeline
• PR #613 docker: change base image tag; goreleaser: fix deprecation notices
• PR #612 database/mongodb: copy session for each op and maintain a connection pool
• PR #610 React web dashboard

0.9.0

• PR #605 bump version to 0.9.0
• PR #604 handle and log errors in reconcile
• PR #600 Running npm audit to remove possible vulnerabilities
• PR #596 worker: fix log message typo
• PR #594 cmd/worker: run task from file
• PR #593 storage/ftp: add FTP support

Docker images

• docker pull ohsucompbio/funnel:0.9.0

0.8.0

• PR #591 webdash: fix state filter bug
• PR #590 storage: more verbose errors
• PR #583 storage/local: fix directory list url join
• PR #581 allow multi-user basic auth
• PR #580 dynamodb: fix potential panic in tableIsAlive method
• PR #577 add version info when running 'make install'
• PR #576 webdash: optimize task list
• PR #575 build: use template for website download links
• PR #574 build: use external release notes builder
• PR #573 server: return WWW-Authenticate response header
• PR #572 compute/node: random node ID, use hostname in dashboard
• PR #571 database/mongo: add task state counts to service info
• PR #570 Fix typos in website
• PR #569 Update website
• PR #567 webdash: fix input/output description render; add created at column in task list
• PR #566 Typo
• PR #563 Fix HPC backend config templates on website
• PR #562 automatically get dependencies for 'tidy' and 'serve-doc' Makefile commands
• PR #561 database/badger: tweak retry timing
• PR #560 docs: example systemd service file for funnel node
• PR #559 Fix HPC templates in default-config.yaml
• PR #557 worker: use os/exec to connect to docker
• PR #556 databases: add Badger
• PR #555 Use dep to vendor dependencies
• PR #554 compute/manual: node draining
• PR #552 storage/swift: better retries and bug fixes
• PR #551 cmd/storage: add get and put
• PR #549 cmd/run: task field cleanup

Docker images

• docker pull ohsucompbio/funnel:0.8.0