[Misc]: missing python inside the container v0.8.1

[longqu]

Anything you want to discuss about vllm.

I pull the apptainer container as following:

apptainer pull docker://vllm/vllm-openai:v0.8.1

the python is not included inside the container:

apptainer shell vllm-openai_v0.8.1.sif
Apptainer> python3
bash: python3: command not found
Apptainer> vllm
bash: /opt/venv/bin/vllm: /opt/venv/bin/python3: bad interpreter: No such file or directory

and the python is actually pointed to an unexisting file:

Apptainer> source /opt/venv/bin/activate
(venv) Apptainer> ls -l /opt/venv/bin/python
lrwxrwxrwx 1 root root 75 Mar 20 04:35 /opt/venv/bin/python -> /root/.local/share/uv/python/cpython-3.12.9-linux-x86_64-gnu/bin/python3.12
(venv) Apptainer> ls /root/.local/share/uv/python/cpython-3.12.9-linux-x86_64-gnu/bin/python3.12
ls: cannot access '/root/.local/share/uv/python/cpython-3.12.9-linux-x86_64-gnu/bin/python3.12': No such file or directory

This issue starts since v0.8.0

Before submitting a new issue...

• Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.

[burakaytan]

Same here! Starting with version v0.8.0, Docker no longer recognizes Python without root user privileges. This issue began in v0.8.0, and the Dockerized method needs to be reverted to how it was prior to v0.8.0.

[longqu]

I just execute using docker:

docker pull vllm/vllm-openai:v0.8.1
docker run -d --name vllm_server vllm/vllm-openai:v0.8.1
2df8914a0c2d572b20016115a0d816c9c68fc9fd5bbf6e049c6058fa50e97859
root@ecm-83ef:/mnt/scratch/tmp2/vllm-0.7.3# docker exec -it vllm_server bash
root@2df8914a0c2d:/vllm-workspace# ls -l /opt/venv/bin/python
lrwxrwxrwx 1 root root 75 Mar 19 20:35 /opt/venv/bin/python -> /root/.local/share/uv/python/cpython-3.12.9-linux-x86_64-gnu/bin/python3.12

Actually, in the docker side, the location existes "/root/.local/share/uv/python/cpython-3.12.9-linux-x86_64-gnu/bin/python3.12".

I have no idea why these can be differ between docker and apptainer. (they both follow the same OCI standard)
The uv python may need to be reviewed. It seems not the standard way to package the container.

Same here! Starting with version v0.8.0, Docker no longer recognizes Python without root user privileges. This issue began in v0.8.0, and the Dockerized method needs to be reverted to how it was prior to v0.8.0.

I think the original issue comes from the introduction of uv python in this PR:

[CI/Build] Use uv python for docker rather than ppa:deadsnakes/ppa #13569

@mgoin would you please provide some feedback on this, we are gettingstuck. thanks

[mru4913]

I had the same issue and couldn't use the docker command to call the server. @DarkLight1337

Test code: docker run --runtime nvidia --gpus all -v xxx p 8015:8000 vllm/vllm-openai:v0.8.1
docker: Error response from daemon: failed to create task for container: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: exec: "python3": executable file not found in $PATH: unknown.

[yankay]

I had the same issue and couldn't use the docker command to call the server. @DarkLight1337

Test code: docker run --runtime nvidia --gpus all -v xxx p 8015:8000 vllm/vllm-openai:v0.8.1 docker: Error response from daemon: failed to create task for container: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: exec: "python3": executable file not found in $PATH: unknown.

If using docker it can work currectly

root@gpu3090:~# docker run --runtime nvidia -it --gpus all \
    -v ~/.cache:/root/.cache \
    --ipc=host \
    vllm/vllm-openai:v0.8.1

INFO 03-20 08:02:46 [__init__.py:256] Automatically detected platform cuda.
INFO 03-20 08:02:51 [api_server.py:977] vLLM API server version 0.8.1

[burakaytan]

We are unable to run the application as a non-root user because Python seems to be installed under the root user. This issue did not occur in previous versions, so it seems that the Docker image was not built in the same way.

Using a Dockerfile like the one below, Python3 cannot be found:

FROM vllm/vllm-openai:v0.8.1

USER root

RUN useradd -ms /bin/bash appuser && \
    chown -R appuser:appuser /home/appuser && \
    chmod -R 777 /home/appuser

COPY startup.sh /home/appuser/

USER appuser

ENTRYPOINT ["/home/appuser/startup.sh"]

[yankay]

According to the Apptainer documentation, Apptainer automatically mounts the host's root path into the container. This is why Python is actually pointing to a non-existent file.

The issue can likely be resolved by specifying the --no-home or --no-mount home option when running the container.

apptainer shell  --no-home vllm-openai_v0.8.1.sif

[burakaytan]

you can see the difference below.

Before version 0.8.0, everything worked fine with the following setup, but after version 0.8.0, Python can no longer be found:

v0.8.1

root@41918dcfe23b:/vllm-workspace# which python
/opt/venv/bin/python
root@41918dcfe23b:/vllm-workspace# useradd -ms /bin/bash appuser
root@41918dcfe23b:/vllm-workspace# chown -R appuser:appuser /home/appuser
root@41918dcfe23b:/vllm-workspace# chmod -R 777 /home/appuser
root@41918dcfe23b:/vllm-workspace# su appuser
appuser@41918dcfe23b:/vllm-workspace$ python3
bash: python3: command not found

v.0.7.0

root@568d9d7739e9:/vllm-workspace# which python3
/usr/bin/python3
root@568d9d7739e9:/vllm-workspace# useradd -ms /bin/bash appuser
root@568d9d7739e9:/vllm-workspace# chown -R appuser:appuser /home/appuser
root@568d9d7739e9:/vllm-workspace# chmod -R 777 /home/appuser
root@568d9d7739e9:/vllm-workspace# su appuser
appuser@568d9d7739e9:/vllm-workspace$ python3
Python 3.12.8 (main, Dec  4 2024, 08:54:12) [GCC 11.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> 

[mru4913]

I had the same issue and couldn't use the docker command to call the server. @DarkLight1337
Test code: docker run --runtime nvidia --gpus all -v xxx p 8015:8000 vllm/vllm-openai:v0.8.1 docker: Error response from daemon: failed to create task for container: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: exec: "python3": executable file not found in $PATH: unknown.

If using docker it can work currectly

root@gpu3090:~# docker run --runtime nvidia -it --gpus all \
    -v ~/.cache:/root/.cache \
    --ipc=host \
    vllm/vllm-openai:v0.8.1

INFO 03-20 08:02:46 [__init__.py:256] Automatically detected platform cuda.
INFO 03-20 08:02:51 [api_server.py:977] vLLM API server version 0.8.1

I used to run by mounting a local folder, like -v /disk/xx/modelHub:/root/modelHub. It worked just fine before version 0.8.0, but with 0.8.1, it's not working. @yankay This is unusual.

[longqu]

According to the Apptainer documentation, Apptainer automatically mounts the host's root path into the container. This is why Python is actually pointing to a non-existent file.

The issue can likely be resolved by specifying the --no-home or --no-mount home option when running the container.

apptainer shell  --no-home vllm-openai_v0.8.1.sif

Well, after testing multiple combinations, i finally succeed in launching the vllm server.

During execution, multiple files are generated inside /root/.triton/ and /root/.config/. So I set --write-tmpfs with a 64MB limit for temporary files.

I need to determine the maximum size these files can reach to ensure 64MB is sufficient is there any idea on this ? @yankay