Skip to content

[Feature]: Clean up vllm.utils #26900

@DarkLight1337

Description

@DarkLight1337

🚀 The feature, motivation and pitch

vllm/utils/__init__.py has >3k lines with a long list of imports. This is not ideal for the following reasons:

  • This file, being imported by many files in vLLM while also importing many external modules, makes it very difficult to defer expensive imports.
  • The risk of merge conflicts is high whenever two PRs change this file, especially when imports are updated.
  • The code because less organized.

We should split this up into multiple files under vllm/utils directory, where each file has a theme that can be used to group the various utility functions. Also, downstream code should import specific submodules of vllm.utils instead of the root vllm.utils to make deferring imports easier.

Alternatives

No response

Additional context

cc @hmellor @mgoin @youkaichao

Before submitting a new issue...

  • Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions