Skip to content

Commit 846ad03

Browse files
committed
[easy] Pass compile_fx only the config patches
Previously we were passing compile_fx the entire default inductor config with the patches applied to it. compile_fx only needs to be passed the patches (reference: https://github.com/pytorch/pytorch/blob/29317f8585ecb232412df3f39734490f0f6d8230/torch/_inductor/compile_fx.py#L1873-L1880) This PR changes vLLM to only pass the patches. This makes debugging things easier (I can stare at just the delta and see what vLLM changed). Test Plan: I ran the following command and verified that performance didn't change. ``` VLLM_USE_V1=1 python benchmark_latency.py --model meta-llama/Meta-Llama-3-8B --batch-size 1 -O '{"level": 3, "compile_sizes": {1, 2}}' ``` Signed-off-by: rzou <zou3519@gmail.com>
1 parent 205d84a commit 846ad03

File tree

1 file changed

+1
-2
lines changed

1 file changed

+1
-2
lines changed

vllm/compilation/compiler_interface.py

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -167,8 +167,7 @@ def compile(
167167
compiler_config: Dict[str, Any],
168168
runtime_shape: Optional[int] = None
169169
) -> Tuple[Optional[Callable], Optional[Any]]:
170-
from torch._inductor import config
171-
current_config = config.get_config_copy()
170+
current_config = {}
172171
from torch._inductor.compile_fx import compile_fx
173172

174173
# disable remote cache

0 commit comments

Comments
 (0)