[AMD][FP8][BugFix] Remove V1 check in arg_utils.py for FP8 since it is not necessary #17215
Conversation
Signed-off-by: Randall Smith <Randall.Smith@amd.com>
rasmith
requested review from
mgoin,
robertgshaw2-redhat and
tlrmchlsmth
as code owners
|
👋 Hi! Thank you for contributing to the vLLM project. 💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels. Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging. To run CI, PR reviewers can either: Add 🚀 |
rasmith
changed the title
…s not necessary (vllm-project#17215) Signed-off-by: Randall Smith <Randall.Smith@amd.com>
…s not necessary (vllm-project#17215) Signed-off-by: Randall Smith <Randall.Smith@amd.com>
…s not necessary (vllm-project#17215) Signed-off-by: Randall Smith <Randall.Smith@amd.com> Signed-off-by: Agata Dobrzyniewicz <adobrzyniewicz@habana.ai>
…s not necessary (vllm-project#17215) Signed-off-by: Randall Smith <Randall.Smith@amd.com> Signed-off-by: Mu Huai <tianbowen.tbw@antgroup.com>
…s not necessary (vllm-project#17215) Signed-off-by: Randall Smith <Randall.Smith@amd.com> Signed-off-by: Yuqi Zhang <yuqizhang@google.com>
4 participants
This PR removes a check in arg_utils.py for FP8 that is not necessary. It turns out that a different backend is used for FP8 when running FP8 on V1 so it will not hit the Triton backend, even if VLLM_USE_TRITON_FLASH_ATTENTION=1 is set.