[Usage]: RuntimeError: Cannot re-initialize CUDA in forked subprocess. To use CUDA with multiprocessing, you must use the 'spawn' start method

[TianSongS]

Your current environment

I used the same service deployment command, but when I upgraded from 0.5.5 to 0.6.1 today, the deployment went wrong

How would you like to use vllm

I want to run inference of a [specific model](put link here). I don't know how to integrate it with vllm.

Before submitting a new issue...

• Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.

[youkaichao]

should be solved by #8390

can you give it a try?

[BrandonStudio]

@youkaichao How can I install it? 0.6.1.post1 does not seem to be released

[youkaichao]

follow https://docs.vllm.ai/en/latest/getting_started/installation.html and install the wheel for the latest commit, or wait for the release, which will finish soon #8440 .