[BugFix] Allow use_cudagraph to work with dynamic VLLM_USE_V1 #19390
Conversation
Summary: If a test dynamically changes VLLM_USE_V1, the default value of use_cudagraph doesn't get updated. This PR fixes that by setting a default factory for use_cudagraph. I'm not sure if this is relevant in real life but it is definitely relevant during testing (we dynamically set VLLM_USE_V1 all the time in tests). Test Plan: - added new test (`pytest tests/compile/test_config.py -v -k "use_cudagraphs_dynamic"`) Signed-off-by: rzou <zou3519@gmail.com>
|
Warning You have reached your daily quota limit. Please wait up to 24 hours and I will start processing your requests again! |
zou3519
changed the title
|
👋 Hi! Thank you for contributing to the vLLM project. 💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels. Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging. To run CI, PR reviewers can either: Add 🚀 |
|
I kicked off a rerun of the entrypoints tests but as far as I can tell those pass locally for me. |
2 participants
Summary:
If a test dynamically changes VLLM_USE_V1, the default value of use_cudagraph doesn't get updated. This PR fixes that by setting a default factory for use_cudagraph. I'm not sure if this is relevant in real life but it is definitely relevant during testing (we dynamically set VLLM_USE_V1 all the time in tests).
Test Plan:
pytest tests/compile/test_config.py -v -k "use_cudagraphs_dynamic")