kingfisher executable file not found in $PATH in Singularity

[JonathonMifsud]

Hi Ben,

I have been using it through conda and docker and had no problems but recently moved over to a cluster that predominately uses singularity. I am not too familiar with singularity so this is probably just a problem on my end but any insight would be super helpful.

module load singularity/3.11.4
singularity pull --name kingfisher:0.3.0.sif docker://wwood/kingfisher:0.3.0

singularity exec kingfisher:0.3.0.sif kingfisher
FATAL: "kingfisher": executable file not found in $PATH

I've replicated this with all versions of kingfisher available on DockerHub

Here is the output of singularity inspect:

singularity inspect kingfisher:0.3.0.sif org.label-schema.build-arch: amd64 org.label-schema.build-date: Friday_22_September_2023_11:7:54_AWST org.label-schema.schema-version: 1.0 org.label-schema.usage.singularity.deffile.bootstrap: docker org.label-schema.usage.singularity.deffile.from: wwood/kingfisher:0.3.0 org.label-schema.usage.singularity.version: 3.11.4 org.opencontainers.image.created: 2022-11-29T15:51:20.245Z org.opencontainers.image.description: Rapid builds of small Conda-based containers using micromamba. org.opencontainers.image.licenses: Apache-2.0 org.opencontainers.image.revision: 4427b199720b9962a7c135fb159fbce50e1ba7b8 org.opencontainers.image.source: https://github.com/mamba-org/micromamba-docker org.opencontainers.image.title: micromamba-docker org.opencontainers.image.url: https://github.com/mamba-org/micromamba-docker org.opencontainers.image.version: latest

Thanks for the great tool.

[wwood]

Hi,

Thanks for the kind words. I think this might be quite straightforward actually, just run

singularity exec kingfisher:0.3.0.sif

I haven't checked singularity specifically but on docker your extra "kingfisher" is causing that issue.

[wwood]

actually, no, that isn't it - will look.

[wwood]

I think what might be happening is that singularity doesn't respect the files in /tmp in the container, they appear to be invisible.

Fixed I think by just putting the kingfisher install in the root directory, rather than /tmp.

Does using this image work for you?

singularity pull --name kingfisher:0.0.0.sif docker://wwood/kingfisher:0.0.0.dev

singularity run kingfisher:0.0.0.sif

If so I'll release again properly.
Thanks.

[JonathonMifsud]

Thanks for the quick response. Close!

singularity pull --name kingfisher:0.0.0.sif docker://wwood/kingfisher:0.0.0.dev
singularity run kingfisher:0.0.0.sif

Traceback (most recent call last):
  File "/kingfisher/bin/kingfisher", line 20, in <module>
    import kingfisher
  File "/kingfisher/bin/../kingfisher/__init__.py", line 13, in <module>
    from .ena import EnaDownloader
  File "/kingfisher/bin/../kingfisher/ena.py", line 5, in <module>
    import pandas as pd
  File "/opt/conda/lib/python3.11/site-packages/pandas/__init__.py", line 16, in <module>
    raise ImportError(
ImportError: Unable to import required dependencies:
numpy: 

IMPORTANT: PLEASE READ THIS FOR ADVICE ON HOW TO SOLVE THIS ISSUE!

Importing the numpy C-extensions failed. This error can happen for
many reasons, often due to issues with your setup or how NumPy was
installed.

We have compiled some common reasons and troubleshooting tips at:

    https://numpy.org/devdocs/user/troubleshooting-importerror.html

Please note and check the following:

  * The Python version is: Python3.11 from "/opt/conda/bin/python3"
  * The NumPy version is: "1.23.4"

and make sure that they are the versions you expect.
Please carefully study the documentation linked above for further help.

Original error was: No module named 'numpy.core._multiarray_umath'

[wwood]

So we have a package in conda distributed through docker converted to singularity.

But we still can't import numpy.

I'm having trouble reproducing this error - I ran the singularity run on 2 different machines with very different (Linux) OSes and both ran without issue. Maybe can you check with your HPC admins? I'm not sure how to help you otherwise..

[JonathonMifsud]

I cleaned out the singularity cache and pulled the image again and that seemed to do it somehow. Many thanks for your help :)