Pulling biocontainers:v1.2.0_cv1 Singularity image fails

[pcrxn]

Description of the bug

When attempting to download the nf-core/nanoseq v3.1.0 workflow for offline use with the option to save Singularity images (nf-core download) , the image biocontainers/biocontainers:v1.2.0_cv1 cannot be downloaded.

The singularity pull error is:

FATAL:   While making image from oci registry: error fetching image to cache: failed to get checksum for docker://quay.io/biocontainers/biocontainers:v1.2.0_cv1: reading manifest v1.2.0_cv1 in quay.io/biocontainers/biocontainers: unauthorized: access to the requested resource is not authorized

Command used and terminal output

$ nf-core download --force nanoseq

                                          ,--./,-.
          ___     __   __   __   ___     /,-._.--~\
    |\ | |__  __ /  ` /  \ |__) |__         }  {
    | \| |       \__, \__/ |  \ |___     \`-._,-`-,
                                          `._,._,'

    nf-core/tools version 2.10 - https://nf-co.re


? Select release / branch: 3.1.0  [release]
? Include the nf-core's default institutional configuration files into the download? No

In addition to the pipeline code, this tool can download software containers.
? Download software container images: singularity

Nextflow and nf-core can use an environment variable called $NXF_SINGULARITY_CACHEDIR that is a path to a directory where remote Singularity images are stored. This allows downloaded images to be cached in a central location.
? Define $NXF_SINGULARITY_CACHEDIR for a shared Singularity image download folder? [y/n]: n

If transferring the downloaded files to another system, it can be convenient to have everything compressed in a single file.
This is not recommended when downloading Singularity images, as it can take a long time and saves very little space.
? Choose compression type: none
WARNING  Deleting existing output directory: 'nf-core-nanoseq_3.1.0'                                                                                                                                                                         
INFO     Saving 'nf-core/nanoseq'                                                                                                                                                                                                            
          Pipeline revision: '3.1.0'                                                                                                                                                                                                         
          Use containers: 'singularity'                                                                                                                                                                                                      
          Container library: 'quay.io'                                                                                                                                                                                                       
          Output directory: 'nf-core-nanoseq_3.1.0'                                                                                                                                                                                          
          Include default institutional configuration: 'False'                                                                                                                                                                               
INFO     Downloading workflow files from GitHub                                                                                                                                                                                              
INFO     Processing workflow revision 3.1.0, found 40 container images in total.                                                                                                                                                             
Downloading singularity images ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━  18% • 7/40 completed
bioconductor-bambu:3.0.8--r42hc247a5b_0 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 64.2% • 306.7/477.5 MB • 2.2 MB/s
Pulling singularity images ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━  82% • 33/40 completed
biocontainers/biocontainers:v1.2.0_cv1  ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ERROR    "biocontainers/biocontainers:v1.2.0_cv1" failed for unclear reasons.                                                                                                                                                                
                                                                                                                                                                                                                                             
INFO     Pulling of "biocontainers/biocontainers:v1.2.0_cv1" failed.                                                                                                                                                                         
         Please troubleshoot the command                                                                                                                                                                                                     
         "singularity pull --name /media/liam/Cerulean/nf-core/nf-core-nanoseq_3.1.0/singularity-images/biocontainers-biocontainers-v1.2.0_cv1.img docker://quay.io/biocontainers/biocontainers:v1.2.0_cv1" manually.


$ singularity pull --name /media/liam/Cerulean/nf-core/nf-core-nanoseq_3.1.0/singularity-images/biocontainers-biocontainers-v1.2.0_cv1.img docker://quay.io/biocontainers/biocontainers:v1.2.0_cv1
FATAL:   While making image from oci registry: error fetching image to cache: failed to get checksum for docker://quay.io/biocontainers/biocontainers:v1.2.0_cv1: reading manifest v1.2.0_cv1 in quay.io/biocontainers/biocontainers: unauthorized: access to the requested resource is not authorized

Relevant files

No response

System information

• Nextflow = 23.04.2 build 5870
• nf-core = 2.10
• Singularity = 3.8.6

[philip-alstrup]

I have had the same issue with some containers when trying to pull with nf-tools and singularity.
I installed singularity 4.0.0 in a machine where I had sudo rights and could then pull directly from the docker registry. Whether it was the elevated rights or the singularity upgrade i'm not sure, but my guess is that upgrading singularity 4.0.0 might do it.

[ahwanpandey]

I am having the same issue with biocontainers/biocontainers:v1.2.0_cv1 and google/deepvariant:1.4.0. Please help.

$ nf-core download \
> nanoseq --revision 3.1.0 \
> --outdir nf-core-nanoseq \
> --container-system singularity \
> --container-cache-utilisation amend \
> --parallel-downloads 4 \
> --compress none \
> --download-configuration


                                          ,--./,-.
          ___     __   __   __   ___     /,-._.--~\
    |\ | |__  __ /  ` /  \ |__) |__         }  {
    | \| |       \__, \__/ |  \ |___     \`-._,-`-,
                                          `._,._,'

    nf-core/tools version 2.11.1 - https://nf-co.re


WARNING  Could not find GitHub authentication token. Some API requests may fail.
INFO     Saving 'nf-core/nanoseq'
          Pipeline revision: '3.1.0'
          Use containers: 'singularity'
          Container library: 'quay.io'
          Using $NXF_SINGULARITY_CACHEDIR': /config/binaries/singularity/containers_devel/nextflow'
          Output directory: 'nf-core-nanoseq'
          Include default institutional configuration: 'True'
INFO     Downloading centralised configs from GitHub
INFO     Downloading workflow files from GitHub
INFO     Processing workflow revision 3.1.0, found 40 container images in total.
Pulling singularity images ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸━━━━━━━━  95% • 38/40 completed
biocontainers/biocontainers:v1.2.0_cv1  ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ERROR    "biocontainers/biocontainers:v1.2.0_cv1" failed for unclear reasons.

INFO     Pulling of "biocontainers/biocontainers:v1.2.0_cv1" failed.
         Please troubleshoot the command
         "singularity pull --name /config/binaries/singularity/containers_devel/nextflow/biocontainers-biocontainers-v1.2.0_cv1.img docker://quay.io/biocontainers/biocontainers:v1.2.0_cv1" manually.

ERROR    "biocontainers/biocontainers:v1.2.0_cv1" failed for unclear reasons.

ERROR    Pulling of "biocontainers/biocontainers:v1.2.0_cv1" failed.
         Please troubleshoot the command
         "singularity pull --name /config/binaries/singularity/containers_devel/nextflow/biocontainers-biocontainers-v1.2.0_cv1.img docker://quay.io/biocontainers/biocontainers:v1.2.0_cv1" manually.

ERROR    Not able to pull image of ['biocontainers/biocontainers:v1.2.0_cv1', '/config/binaries/singularity/containers_devel/nextflow/biocontainers-biocontainers-v1.2.0_cv1.img', None]. Service might be down or
Pulling singularity images ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸━━━━  98% • 39/40 completed
google/deepvariant:1.4.0  ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ERROR    "google/deepvariant:1.4.0" failed for unclear reasons.

INFO     Pulling of "google/deepvariant:1.4.0" failed.
         Please troubleshoot the command
         "singularity pull --name /config/binaries/singularity/containers_devel/nextflow/google-deepvariant-1.4.0.img docker://quay.io/google/deepvariant:1.4.0" manually.

ERROR    "google/deepvariant:1.4.0" failed for unclear reasons.

ERROR    Pulling of "google/deepvariant:1.4.0" failed.
         Please troubleshoot the command
         "singularity pull --name /config/binaries/singularity/containers_devel/nextflow/google-deepvariant-1.4.0.img docker://quay.io/google/deepvariant:1.4.0" manually.

Pulling singularity images ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% • 40/40 completed


$ singularity pull --name /config/binaries/singularity/containers_devel/nextflow/biocontainers-biocontainers-v1.2.0_cv1.img docker://quay.io/biocontainers/biocontainers:v1.2.0_cv1
FATAL:   While making image from oci registry: error fetching image to cache: failed to get checksum for docker://quay.io/biocontainers/biocontainers:v1.2.0_cv1: reading manifest v1.2.0_cv1 in quay.io/biocontainers/biocontainers: unauthorized: access to the requested resource is not authorized

$ singularity pull --name /config/binaries/singularity/containers_devel/nextflow/google-deepvariant-1.4.0.img docker://quay.io/google/deepvariant:1.4.0
FATAL:   While making image from oci registry: error fetching image to cache: failed to get checksum for docker://quay.io/google/deepvariant:1.4.0: reading manifest 1.4.0 in quay.io/google/deepvariant: unauthorized: access to the requested resource is not authorized

Nextflow = 23.10.1.5891
nf-core/tools = 2.11.1
Singularity = 3.8.6

[ahwanpandey]

OK managed to get the google/deepvariant:1.4.0 container using the command parameters -l quay.io -l ghcr.io -l docker.io.

nf-core download \
nanoseq --revision 3.1.0 \
--outdir nf-core-nanoseq \
--container-system singularity \
--container-cache-utilisation amend \
--parallel-downloads 4 \
--compress none \
-l quay.io -l ghcr.io -l docker.io \
--download-configuration

For biocontainers/biocontainers:v1.2.0_cv1, I didn't try it with -l quay.io -l ghcr.io -l docker.io but instead downloaded manually from:

https://containers.biocontainers.pro/s3/SingImgsRepo/biocontainers/v1.2.0_cv1/biocontainers_v1.2.0_cv1.img