Optimize configuration access with LRU cache in custom ops

[skyloevil]

Changes

• Added get_cached_compilation_config() function with
  @lru_cache(maxsize=1) to cache compilation config access
• Added cache invalidation in set_current_vllm_config() context
  manager to ensure cache freshness
• Fixed code style (line length) issues

Benefits

• Reduces repeated configuration lookups in custom operations
• Improves performance for frequently accessed compilation config
• Maintains correctness with proper cache invalidation

Files Modified

• vllm/config.py: Added caching function and invalidation logic
• vllm/model_executor/custom_op.py: Updated to use cached config access

[github-actions]

👋 Hi! Thank you for contributing to the vLLM project.

💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels.

Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which starts running only a small and essential subset of CI tests to quickly catch errors. You can run other CI tests on top of those by going to your fastcheck build on Buildkite UI (linked in the PR checks section) and unblock them. If you do not have permission to unblock, ping simon-mo or khluu to add you in our Buildkite org.

Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can either: Add ready label to the PR or enable auto-merge.

🚀

[gemini-code-assist]

Code Review

This pull request introduces a cached function get_cached_compilation_config using lru_cache to optimize repeated access to the compilation configuration. While the intent to improve performance is good, the current implementation has a critical flaw. The caching mechanism doesn't account for changes in the global configuration context, which can lead to stale cache data and incorrect behavior, especially in testing environments. I've provided a detailed comment on how to address this to ensure correctness.

[gemini-code-assist]

Using lru_cache on this function is problematic because it depends on a mutable global variable (_current_vllm_config in vllm.config). The set_current_vllm_config context manager can change this global, but the cache won't be invalidated, leading to stale data. This will break tests that iterate over different configurations, like tests/model_executor/test_enabled_custom_ops.py.

A robust solution is to clear the cache when the context changes. This can be done by moving get_cached_compilation_config to vllm/config.py and calling get_cached_compilation_config.cache_clear() in the finally block of the set_current_vllm_config context manager.

This change would require modifying vllm/config.py and importing this function from there.

Example change in vllm/config.py:

@contextmanager
def set_current_vllm_config(vllm_config: VllmConfig, ...):
    ...
    try:
        ...
        yield
    finally:
        get_cached_compilation_config.cache_clear()
        ...

[mgoin]

Nit: you could use cache instead if you aren't going to use the lru part

[mgoin]

LGTM