mspipelines

Requirements

To use this repository, please install the following dependencies:

• Bash
• Java (Java 11 or higher)
• Docker
• Nextflow
• Viash

Assuming you’ve already got Bash, Java, Docker installed. To install Nextflow and Viash, create a bin/ directory in your home directory and make sure it’s on your PATH variable.

mkdir "$HOME/bin"
curl -fsSL get.viash.io | bash -s -- --bin $HOME/bin --tools false
curl -s https://get.nextflow.io | bash; mv nextflow $HOME/bin

Make sure that Viash and Nextflow are on the $PATH by checking whether the following commands work:

viash -v
nextflow -v

Step 1: Build all the components

The src/ folder contains Viash components for processing Mass Spec data. With the command below, you can turn your script into a containerized executable and a containerized Nextflow module. Use the -q 'xxx' parameter to build a subset of components in the repository.

viash ns build --query 'maxquant' --parallel --setup cachedbuild

Exporting maxquant (maxquant) =nextflow=> mspipelines/target/nextflow/maxquant/maxquant
Exporting maxquant (maxquant) =docker=> mspipelines/target/docker/maxquant/maxquant
Exporting maxquant_to_h5ad (convert) =docker=> mspipelines/target/docker/convert/maxquant_to_h5ad
Exporting maxquant_to_h5ad (convert) =nextflow=> mspipelines/target/nextflow/convert/maxquant_to_h5ad
[notice] Building container 'ghcr.io/czbiohub/mspipelines/maxquant_maxquant:dev' with Dockerfile
[notice] Building container 'ghcr.io/czbiohub/mspipelines/convert_maxquant_to_h5ad:dev' with Dockerfile
Not all configs built successfully
  6/10 configs were disabled
  4/10 configs built successfully

Viash will build a whole namespace (ns) into executables and Nextflow pipelines into the target/docker and target/nextflow folders respectively.

• By adding the -q/--query flag, you can filter which components to build using a regex.
• By adding the --parallel flag, these components are built in parallel (otherwise it will take a really long time).
• The flag --setup cachedbuild will automatically start building Docker containers for each of these methods.

The command might take a while to run, since it is building a docker container for each of the components.

Step 2: Download the test resources

Use the sync_test_resources component to download the test resources.

viash run src/download/sync_test_resources/config.vsh.yaml

Completed 256.0 KiB/7.2 MiB (302.6 KiB/s) with 6 file(s) remaining
Completed 512.0 KiB/7.2 MiB (595.8 KiB/s) with 6 file(s) remaining
Completed 768.0 KiB/7.2 MiB (880.3 KiB/s) with 6 file(s) remaining
Completed 1.0 MiB/7.2 MiB (1.1 MiB/s) with 6 file(s) remaining    
Completed 1.2 MiB/7.2 MiB (1.3 MiB/s) with 6 file(s) remaining
...

That’s it!

You’re all set to start developing new components. Below are a few more examples to help you get started.

Example 1: Unit test a component

viash test src/maxquant/maxquant/config.vsh.yaml

Running tests in temporary directory: '/home/rcannood/workspace/viash_temp/viash_test_maxquant750761290439648338'
====================================================================
+/home/rcannood/workspace/viash_temp/viash_test_maxquant750761290439648338/build_executable/maxquant ---verbosity 6 ---setup cachedbuild
[notice] Building container 'ghcr.io/czbiohub/mspipelines/maxquant_maxquant:test_jrHF3N' with Dockerfile
...
Successfully built d502546463de
Successfully tagged ghcr.io/czbiohub/mspipelines/maxquant_maxquant:test_jrHF3N
====================================================================
+/home/rcannood/workspace/viash_temp/viash_test_maxquant750761290439648338/test_test/test_executable
>> Running maxquant
Configuring 
Assemble run info 
Finish run info 
Testing fasta files 
Testing raw files 
...
Writing tables 
Finish writing tables  
>> Checking whether output files can be found
>> All tests succeeded!
====================================================================
SUCCESS! All 1 out of 1 test scripts succeeded!
Cleaning up temporary directory

Example 2: Run a pipeline using the local code

dir=resources_test/msdial_demo_files/raw/LCMS_DDA
nextflow run . \
  -main-script target/nextflow/msdial/msdial_lcms/main.nf \
  -with-docker \
  --input "$dir/Nega_Ida_QC_1_1.mzML" \
  --input "$dir/Nega_Ida_QC_1_9.mzML" \
  --class_id foo \
  --class_id bar \
  --ms1_data_type Profile \
  --ms2_data_type Profile \
  --ion_mode Negative \
  --minimum_peak_height 1 \
  --adduct_list "[M-H]-,[M+FA-H]-,[M-H2O-H]-" \
  --retention_time_tolerance_for_identification 4 \
  --accurate_ms1_tolerance_for_identification 0.025 \
  --accurate_ms2_tolerance_for_identification 0.25 \
  --publish_dir output