Skip to content

Commit

Permalink
GHA use shared singularity cache (#3738)
Browse files Browse the repository at this point in the history
* Add Singularity cachedir for reducing storage footprint

* Use runner temp directory for reducing storage footprint

* Add default NXF directories for reducing memory footprint

* Replace variable with env because GHA scoping

* Add some fake stuff for triggering tests

* Remove working dir stuff.

* Create singularity dirs

* fixup

* Add more fake testing criteria

* Replace parabricks weird cp stuff with stageInMode directive

* parabricks/fq2bam: Replace copy in parabricks with stageInMode

The parabricks/fq2bam module had hard copies from the previous working dir to the local working directory to remove the affect of symlinks. However, this will fail on a number of cases therefore this commit replaces them with the stageInDirective of Nextflow.

* Revert deepvariant to nf-core image

* Revert "parabricks/fq2bam: Replace copy in parabricks with stageInMode"

This reverts commit f94d971.

* Revert "Replace parabricks weird cp stuff with stageInMode directive"

This reverts commit 44ed12a.

* Remove fake stuff from parabricks/fq2bam

* Ignore Singularity cache for uploading output(s)

* Fixup
  • Loading branch information
adamrtalbot authored Aug 14, 2023
1 parent 5c73153 commit 485028c
Showing 1 changed file with 12 additions and 0 deletions.
12 changes: 12 additions & 0 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,10 @@ concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: true

env:
NXF_SINGULARITY_CACHEDIR: ${{ github.workspace }}/.singularity
NXF_SINGULARITY_LIBRARYDIR: ${{ github.workspace }}/.singularity

jobs:
prettier:
runs-on: ubuntu-latest
Expand Down Expand Up @@ -251,6 +255,7 @@ jobs:
env:
NXF_ANSI_LOG: false
SENTIEON_LICENSE_BASE64: ${{ secrets.SENTIEON_LICENSE_BASE64 }}

steps:
- uses: actions/checkout@v3

Expand Down Expand Up @@ -289,6 +294,12 @@ jobs:
with:
singularity-version: 3.7.1

- name: Set up Singularity
if: matrix.profile == 'singularity'
run: |
mkdir -p $NXF_SINGULARITY_CACHEDIR
mkdir -p $NXF_SINGULARITY_LIBRARYDIR
- name: Set up miniconda
if: matrix.profile == 'conda'
uses: conda-incubator/setup-miniconda@v2
Expand Down Expand Up @@ -348,6 +359,7 @@ jobs:
/home/runner/pytest_workflow_*/*/work
!/home/runner/pytest_workflow_*/*/work/conda
!/home/runner/pytest_workflow_*/*/work/singularity
!${{ github.workspace }}/.singularity
nf-test-changes:
name: nf-test-changes
Expand Down

0 comments on commit 485028c

Please sign in to comment.