Add openEuler based container image for vLLM Ascend #489

wxsIcey · 2025-04-09T03:41:12Z

What this PR does / why we need it?

Provide users with openEuler-based vllm images, so modify the quick start readme

Does this PR introduce any user-facing change?

None

How was this patch tested?

1. Run the image：

export DEVICE=/dev/davinci4
export IMAGE=quay.io/ascend/vllm-ascend:main-openeuler
docker run --rm \ --name vllm-ascend \ --device $DEVICE \ --device /dev/davinci_manager \ --device /dev/devmm_svm \ --device /dev/hisi_hdc \ -v /usr/local/dcmi:/usr/local/dcmi \ -v /usr/local/bin/npu-smi:/usr/local/bin/npu-smi \ -v /usr/local/Ascend/driver/lib64/:/usr/local/Ascend/driver/lib64/ \ -v /usr/local/Ascend/driver/version.info:/usr/local/Ascend/driver/version.info \ -v /etc/ascend_install.info:/etc/ascend_install.info \ -v /root/.cache:/root/.cache \ -it $IMAGE bash

2. Test offline script

from vllm import LLM, SamplingParams

prompts = [
    "Hello, my name is",
    "The future of AI is",
]
sampling_params = SamplingParams(temperature=0.8, top_p=0.95)
# The first run will take about 3-5 mins (10 MB/s) to download models
llm = LLM(model="Qwen/Qwen2.5-0.5B-Instruct")

outputs = llm.generate(prompts, sampling_params)

for output in outputs:
    prompt = output.prompt
    generated_text = output.outputs[0].text
    print(f"Prompt: {prompt!r}, Generated text: {generated_text!r}")

3. Results

Yikun · 2025-04-09T06:09:21Z

docs/source/quick_start.md

 ```
+::::
+
+::::{tab-item} OpenEuler OS


Suggested change

::::{tab-item} OpenEuler OS

::::{tab-item} openEuler OS

docs/source/quick_start.md

Yikun

You can click below readthedocs CI to prereview: https://vllm-ascend--489.org.readthedocs.build/en/489/quick_start.html

Yikun · 2025-04-09T06:21:28Z

docs/source/quick_start.md

+-p 8000:8000 \
+-it $IMAGE bash
+```



missing: ::::

ZhengZhenyu

some comments inline

ZhengZhenyu · 2025-04-09T11:46:57Z

Dockerfile.openEuler

+ARG PIP_INDEX_URL="https://mirrors.tuna.tsinghua.edu.cn/pypi/web/simple"
+
+# Define environments
+ENV DEBIAN_FRONTEND=noninteractive


This is useless for openEuler image

ZhengZhenyu · 2025-04-09T11:47:44Z

Dockerfile.openEuler

+
+FROM quay.io/ascend/cann:8.0.0-910b-openeuler22.03-py3.10
+
+ARG PIP_INDEX_URL="https://mirrors.tuna.tsinghua.edu.cn/pypi/web/simple"


Why use Tsinghua mirror? Seems github action is faster with original mirror

wangxiyuan · 2025-04-10T00:49:38Z

.github/workflows/image_openeuler.yml

+jobs:
+
+  build:
+    name: vllm-ascend image


please rename to a new one

Yikun · 2025-04-10T03:17:44Z

.github/workflows/image_openeuler.yml

+            type=ref,event=branch
+            type=ref,event=pr
+            type=pep440,pattern={{raw}}-openeuler


It seems this should also be changed.

type=ref,event=branch ----> v0.7.3-dev / main (this should be changed)

type=ref,event=pr ----> pr

type=pep440,pattern={{raw}}-openeuler ---> v0.7.3rc2-openeuler

Yikun · 2025-04-10T04:01:57Z

.github/workflows/image_openeuler.yml

+          type=ref,event=branch,suffix=-openeuler
+          type=ref,event=pr,suffix=-openeuler
+          type=pep440,pattern={{raw}}-openeuler
+          type=raw,value=latest-openeuler


Suggested change

type=ref,event=branch,suffix=-openeuler

type=ref,event=pr,suffix=-openeuler

type=pep440,pattern={{raw}}-openeuler

type=raw,value=latest-openeuler

type=ref,event=branch,suffix=-openeuler

type=ref,event=pr,suffix=-openeuler

type=pep440,pattern={{raw}}-openeuler

we don't need a latest tag

Yikun · 2025-04-10T04:03:36Z

docs/source/quick_start.md

 ## Setup environment using container

+:::::{tab-set}
+::::{tab-item} Ubuntu OS


Suggested change

::::{tab-item} Ubuntu OS

::::{tab-item} Ubuntu

Yikun · 2025-04-10T04:03:42Z

docs/source/quick_start.md

 ```
+::::
+
+::::{tab-item} openEuler OS


Suggested change

::::{tab-item} openEuler OS

::::{tab-item} openEuler

Yikun · 2025-04-10T04:05:56Z

.github/workflows/image_openeuler.yml

+        build-args: |
+            PIP_INDEX_URL=https://pypi.org/simple


Yikun

LGTM if you can run succesfully: https://vllm-ascend--489.org.readthedocs.build/en/489/quick_start.html

Please add your test results: How was this patch tested?

Signed-off-by: Icey <1790571317@qq.com>

wxsIcey · 2025-04-10T06:29:56Z

LGTM if you can run succesfully: https://vllm-ascend--489.org.readthedocs.build/en/489/quick_start.html

Please add your test results: How was this patch tested?

测试报告如下：

测试命令：

进入容器：
export DEVICE=/dev/davinci4
export IMAGE=quay.io/ascend/vllm-ascend:main-openeuler
docker run --rm \ --name vllm-ascend \ --device $DEVICE \ --device /dev/davinci_manager \ --device /dev/devmm_svm \ --device /dev/hisi_hdc \ -v /usr/local/dcmi:/usr/local/dcmi \ -v /usr/local/bin/npu-smi:/usr/local/bin/npu-smi \ -v /usr/local/Ascend/driver/lib64/:/usr/local/Ascend/driver/lib64/ \ -v /usr/local/Ascend/driver/version.info:/usr/local/Ascend/driver/version.info \ -v /etc/ascend_install.info:/etc/ascend_install.info \ -v /root/.cache:/root/.cache \ -it $IMAGE bash
测试用例：
测试结果：

Fix the nix in my follow-up

Yikun

Post CI works as well (with main-openeuler): https://github.com/vllm-project/vllm-ascend/actions/runs/14373846953/job/40301777097
Doc is also OK: https://vllm-ascend.readthedocs.io/en/main/quick_start.html

🎉🥳 Thanks! @wxsIcey

### What this PR does / why we need it? There was a bug when we release v0.8.4rc1 (openEuler image tag was wrong set to 0.8.4rc1), according doc of docker-meta-action, it should be append suffix: ``` tags: | type=pep440,enable=true,priority=900,prefix=,suffix=,pattern=,value= ``` This patch just fix openEuler image suffix to make pep440 tag rule work. This patch also remove the cache step because the cache step bring more than 10mins export, but reduce less time in next trigger. ### Does this PR introduce _any_ user-facing change? Yes, docker image tag set to right ### How was this patch tested? I test with in my fork repo by setting default branch: - release a tag: v0.7.88rc1 (pep440 tag) - The log show `--label org.opencontainers.image.version=v0.7.88rc1-openeuler` is right rule https://github.com/Yikun/vllm-ascend/actions/runs/14560411481/job/40842950165#step:9:205 Related: #489 Signed-off-by: Yikun Jiang <yikunkero@gmail.com>

### What this PR does / why we need it? Provide users with openEuler-based vllm images, so modify the quick start readme ### Does this PR introduce _any_ user-facing change? None ### How was this patch tested? There is no need for performing any test. --------- Signed-off-by: Icey <1790571317@qq.com>

### What this PR does / why we need it? There was a bug when we release v0.8.4rc1 (openEuler image tag was wrong set to 0.8.4rc1), according doc of docker-meta-action, it should be append suffix: ``` tags: | type=pep440,enable=true,priority=900,prefix=,suffix=,pattern=,value= ``` This patch just fix openEuler image suffix to make pep440 tag rule work. This patch also remove the cache step because the cache step bring more than 10mins export, but reduce less time in next trigger. ### Does this PR introduce _any_ user-facing change? Yes, docker image tag set to right ### How was this patch tested? I test with in my fork repo by setting default branch: - release a tag: v0.7.88rc1 (pep440 tag) - The log show `--label org.opencontainers.image.version=v0.7.88rc1-openeuler` is right rule https://github.com/Yikun/vllm-ascend/actions/runs/14560411481/job/40842950165#step:9:205 Related: vllm-project#489 Signed-off-by: Yikun Jiang <yikunkero@gmail.com>

### What this PR does / why we need it? Provide users with openEuler-based vllm images, so modify the quick start readme ### Does this PR introduce _any_ user-facing change? None ### How was this patch tested? There is no need for performing any test. --------- Signed-off-by: Icey <1790571317@qq.com>

### What this PR does / why we need it? There was a bug when we release v0.8.4rc1 (openEuler image tag was wrong set to 0.8.4rc1), according doc of docker-meta-action, it should be append suffix: ``` tags: | type=pep440,enable=true,priority=900,prefix=,suffix=,pattern=,value= ``` This patch just fix openEuler image suffix to make pep440 tag rule work. This patch also remove the cache step because the cache step bring more than 10mins export, but reduce less time in next trigger. ### Does this PR introduce _any_ user-facing change? Yes, docker image tag set to right ### How was this patch tested? I test with in my fork repo by setting default branch: - release a tag: v0.7.88rc1 (pep440 tag) - The log show `--label org.opencontainers.image.version=v0.7.88rc1-openeuler` is right rule https://github.com/Yikun/vllm-ascend/actions/runs/14560411481/job/40842950165#step:9:205 Related: vllm-project#489 Signed-off-by: Yikun Jiang <yikunkero@gmail.com>

github-actions bot added the documentation Improvements or additions to documentation label Apr 9, 2025

Yikun previously requested changes Apr 9, 2025

View reviewed changes

Yikun reviewed Apr 9, 2025

View reviewed changes

github-actions bot added ci/build and removed documentation Improvements or additions to documentation labels Apr 9, 2025

wxsIcey requested a review from Yikun April 9, 2025 08:00

github-actions bot added the documentation Improvements or additions to documentation label Apr 9, 2025

ZhengZhenyu reviewed Apr 9, 2025

View reviewed changes

wangxiyuan approved these changes Apr 10, 2025

View reviewed changes

Yikun reviewed Apr 10, 2025

View reviewed changes

wxsIcey added 17 commits April 10, 2025 05:21

Change quick start readme

dcb56d4

Signed-off-by: Icey <1790571317@qq.com>

Add openEuler Dockerfile and yml

a6433e1

Signed-off-by: Icey <1790571317@qq.com>

Reset readme

7b8ca47

Signed-off-by: Icey <1790571317@qq.com>

Change openeuler image tag

224e3c4

Signed-off-by: Icey <1790571317@qq.com>

Change quick start readme

aacdf66

Signed-off-by: Icey <1790571317@qq.com>

fix

d558328

Signed-off-by: Icey <1790571317@qq.com>

fix

5480cf4

Signed-off-by: Icey <1790571317@qq.com>

fix

caf7c00

Signed-off-by: Icey <1790571317@qq.com>

Change Dockerfile

b9e7bf8

Signed-off-by: Icey <1790571317@qq.com>

Remove tsinnghua source

60783c1

Signed-off-by: Icey <1790571317@qq.com>

fix

36ec83c

Signed-off-by: Icey <1790571317@qq.com>

fix

fe78c9f

Signed-off-by: Icey <1790571317@qq.com>

fix

580a995

Signed-off-by: Icey <1790571317@qq.com>

fix

57d848e

Signed-off-by: Icey <1790571317@qq.com>

fix

fef3829

Signed-off-by: Icey <1790571317@qq.com>

fix

125e273

Signed-off-by: Icey <1790571317@qq.com>

fix

93e9d62

Signed-off-by: Icey <1790571317@qq.com>

wxsIcey added 2 commits April 10, 2025 05:21

fix

e3e7455

Signed-off-by: Icey <1790571317@qq.com>

fix

593ca06

Signed-off-by: Icey <1790571317@qq.com>

wangxiyuan approved these changes Apr 10, 2025

View reviewed changes

Yikun changed the title ~~Change quick start readme~~ Add openEuler based container image for vLLM Ascend Apr 10, 2025

wangxiyuan merged commit d05ea17 into vllm-project:main Apr 10, 2025
6 checks passed

Yikun reviewed Apr 10, 2025

View reviewed changes

Yikun mentioned this pull request Apr 20, 2025

[Docker] Fix openEuler image suffix #586

Merged


		FROM quay.io/ascend/cann:8.0.0-910b-openeuler22.03-py3.10

		ARG PIP_INDEX_URL="https://mirrors.tuna.tsinghua.edu.cn/pypi/web/simple"

Add openEuler based container image for vLLM Ascend #489

Add openEuler based container image for vLLM Ascend #489

Uh oh!

Conversation

wxsIcey commented Apr 9, 2025 • edited by Yikun Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What this PR does / why we need it?

Does this PR introduce any user-facing change?

How was this patch tested?

1. Run the image：

2. Test offline script

3. Results

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Yikun left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ZhengZhenyu left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Yikun left a comment

Choose a reason for hiding this comment

Uh oh!

wxsIcey commented Apr 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Yikun left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

wxsIcey commented Apr 9, 2025 •

edited by Yikun

Loading

wxsIcey commented Apr 10, 2025 •

edited

Loading

Yikun left a comment •

edited

Loading