English | 简体中文

nf-tencentcloud

nf-tencentcloud is a nextflow plugin designed to add Tencent Cloud Object storage and Tencent Healthcare Omics Platform executor adaptation support to the nextflow workflow engine. Through this component, it can also implement some detailed function adaptations required by the platform, such as metadata file generation, to ensure the efficient operation and management of the workflow. Its design goal is to extend Tencent Cloud's native support for Nextflow workflows, allowing users to run Nextflow workflows using Tencent Cloud resources in a simple and easy-to-use manner.

we use Task Execution Schema (TES) protocol as the protocol for nextflow to dock with the Tencent Healthcare Omics Platform. Compared to the official plugin, we utilize some custom fields to adapt to the platform features.

Feature

• Supports docking with Tencent Cloud COS object storage, allowing direct dependence on Tencent Cloud files in files and configurations.
• Supports docking with Tencent Healthcare Omics Platform for quick access to high-performance elastic computing capabilities (For more information about Tencent Health Genomics Platform, please refer to: Tencent Healthcare Omics Platform).

Installation

• Make sure that nextflow is already installed on your system.
• Run the following command to install nf-tencentcloud.

  nextflow plugin install nf-tencentcloud

  You can install the nf-tencentcloud plugin that meets your current environment.
• You can also specify the plugin in the configuration file for installation.

  plugins {
      id 'nf-tencentcloud@2.0.0'
  }

• Or you can use the -plugins command line option.

  nextflow run <pipeline> -plugins nf-tencentcloud@2.0.0

Using Tencent Cloud Object Storage

This plugin integrates COS object storage support, and you can conveniently integrate and use Tencent Cloud object storage in nextflow after completing the object storage-related service opening in Tencent Cloud.

Get the key on the Tencent Cloud CAM console page and follow the steps below:

• Configure the key in the nextflow configuration file.

  tencentcloud {
      secretId = "your_secret_id"
      secretKey = "your_secret_key"
  }

• If you use temporary key authorization, you can configure accessToken to enable temporary key.

  tencentcloud {
      secretId = "your_secret_id"
      secretKey = "your_secret_key"
      accessToken = "your_access_token"
  }

• After the configuration is complete, you can use COS storage anywhere in the process and configuration, such as:

  workDir = "cos://test-bucket-1258888888/nextflow/workdir"

Using Tencent Healthcare Omics Platform

This plugin integrates the Tencent Healthcare Omics Platform executor, and you can enable sandbox in the omics platform for debugging, or wait for the subsequent opening of the external network access terminal interface, and use the elastic computing resources provided by the Tencent Genomics Platform after making the corresponding configuration on your own computer.

In order to use the Tencent Health Genomics Platform executor, you need to activate and use the Tencent Healthcare Omics Platform on Tencent Cloud, and then follow the steps below:

• You need to obtain the request endpoint and accessToken in the omics platform for interface calling and authentication, and configure them in the nextflow configuration file as follows:

  tencentcloud {
      omics {
          endpoint = "http://your_endpoint/api"
          accessToken = "your_access_token"
      }
  }

  Please note: The accessToken here is used for Tencent Healthcare Omics Platform oauth authentication, which is different from the Tencent Cloud cam temporary authorization accessToken mentioned above, and cannot be mixed.

• After completing the relevant configuration, set the executor to tencentcloud-omics to use the elastic computing capabilities of the Tencent Health Genomics Platform.

  process {
      executor = 'tencentcloud-omics'
  }

Contributing

For more information about contributing issues or pull requests, see our nf-tencentcloud Contributing Guide。

License

nf-tencentcloud is under the Apache-2.0. See the LICENSE file for details.

Support

This plugin is developed by Tencent Healthcare Omics Platform team and can be used upon Tencent Healthcare Omics Platform. Tencent Healthcare Omics Platform, built on a PaaS architecture, provides genomic enterprises with an efficient deployment, flexible scheduling, and user-friendly bioinformatics cloud environment along with various computing resources

For inquiries about product testing and usage, please contact via email: omics@tencent.com