[CI/Build] Fix OpenAI API correctness on AMD CI

[zhewenl]

Purpose

OpenAI API correctness is failing on AMD CI consistently(example) because in encoder-decoder model openai/whisper-large-v3 is used in test: entrypoints/openai/correctness/test_transcription_api_correctness.

Since AMD doesn't support encoder models(see #27442), this PR skips the test from the test suite

Test Plan

 pytest -s tests/entrypoints/openai/correctness/ (manual skip)
...
================================================================== warnings summary ==================================================================
<frozen importlib._bootstrap>:488
  <frozen importlib._bootstrap>:488: DeprecationWarning: builtin type SwigPyPacked has no __module__ attribute

<frozen importlib._bootstrap>:488
  <frozen importlib._bootstrap>:488: DeprecationWarning: builtin type SwigPyObject has no __module__ attribute

tests/entrypoints/openai/correctness/test_lmeval.py::test_lm_eval_accuracy_v1_engine
tests/entrypoints/openai/correctness/test_lmeval.py::test_lm_eval_accuracy_v1_engine
  /home/zhewenli/uv_env/vllm-fork/lib64/python3.12/site-packages/lm_eval/models/vllm_causallms.py:38: DeprecationWarning: vllm.utils.get_open_port is deprecated and will be removed in a future version. Use vllm.utils.network_utils.get_open_port instead.
    from vllm.utils import get_open_port

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
================================================ 1 passed, 1 skipped, 4 warnings in 133.91s (0:02:13) ================================================

CI: https://buildkite.com/vllm/amd-ci/builds/874

[gemini-code-assist]

Code Review

This pull request addresses a consistent test failure on the AMD CI by skipping a test that utilizes an encoder-decoder model, which is not supported on ROCm. The change to the CI configuration is appropriate. However, I've identified a misleading comment that was added to explain this change. My review includes a suggestion to correct the comment for clarity and accuracy.

[gemini-code-assist]

This comment contains a typo and a logical error. It incorrectly states that encoder-decoder models are supported, which contradicts the reason for skipping this test. The comment should be updated to accurately reflect that the test is skipped because these models are not supported on ROCm.

  # Transcription WER check is skipped because encoder-decoder models are not supported on ROCm, see https://github.com/vllm-project/vllm/issues/27442

[DarkLight1337]

Can you address this?

[zhewenl]

Updated! (also appreciate some eyes on #27926), thanks!

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

[chatgpt-codex-connector]

Skip ignores wrong path for transcription test

The new --ignore argument is supposed to skip test_transcription_api_correctness.py, but the path passed to pytest leaves out the tests/ prefix. The file actually lives at tests/entrypoints/openai/correctness/test_transcription_api_correctness.py, so the ignore rule will not match and the unsupported encoder–decoder test will still run and fail on AMD CI. Point the ignore option at the real file path so the job actually excludes it.

Useful? React with 👍 / 👎.