[Quantization] Enable BNB support for more MoE models

[jeejeelee]

Essential Elements of an Effective PR Description Checklist

• The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
• The test plan, such as providing test command.
• The test results, such as pasting the results comparison before and after, or e2e results
• (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.

Purpose

Following #21100

Test Plan

# SPDX-License-Identifier: Apache-2.0
# SPDX-FileCopyrightText: Copyright contributors to the vLLM project

from vllm import LLM, SamplingParams

# Sample prompts.
prompts = [
    "Hello, my name is",
    "The president of the United States is",
    "The capital of France is",
    "The future of AI is",
]
# Create a sampling params object.
sampling_params = SamplingParams(temperature=0.8, top_p=0.95)


def main():
    # Create an LLM.
    llm = LLM(
        model="rednote-hilab/dots.llm1.inst",
        quantization="bitsandbytes",
        trust_remote_code=True,
        tensor_parallel_size=4,
    )
    # Generate texts from the prompts.
    # The output is a list of RequestOutput objects
    # that contain the prompt, generated text, and other information.
    outputs = llm.generate(prompts, sampling_params)
    # Print the outputs.
    print("\nGenerated Outputs:\n" + "-" * 60)
    for output in outputs:
        prompt = output.prompt
        generated_text = output.outputs[0].text
        print(f"Prompt:    {prompt!r}")
        print(f"Output:    {generated_text!r}")
        print("-" * 60)


if __name__ == "__main__":
    main()

Test Result

I have tested the above script, and the generated results look reasonable

Generated Outputs:
------------------------------------------------------------
Prompt:    'Hello, my name is'
Output:    " James. I'm a 10-year-old student. My school is small"
------------------------------------------------------------
Prompt:    'The president of the United States is'
Output:    ' Donald Trump. The president of Russia is Vladimir Putin. Who is the president of'
------------------------------------------------------------
Prompt:    'The capital of France is'
Output:    ' Paris.\n\nThe capital of France is Paris. The answer to the question is correct'
------------------------------------------------------------
Prompt:    'The future of AI is'
Output:    ' open (2023)\n> It is just a matter:Dumb people'

(Optional) Documentation Update

[gemini-code-assist]

Code Review

This pull request refactors the weight loading mechanism for dots1 and glm4_moe models to enable BNB quantization support. The changes primarily involve:

• dots1 model: Migrating to AutoWeightsLoader for weight loading, which simplifies the code and aligns it with other models. This also includes adding SupportsLoRA and the necessary packed_modules_mapping.
• Both dots1 and glm4_moe models: Refactoring the expert parameter mapping logic into a get_expert_mapping method within the respective Model classes. This improves code clarity and reusability.

The changes are well-structured and consistent across both models. I did not find any issues of high or critical severity in the provided diffs. The refactoring appears to be a solid step towards better maintainability and feature support.

[github-actions]

👋 Hi! Thank you for contributing to the vLLM project.

💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels.

Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which starts running only a small and essential subset of CI tests to quickly catch errors. You can run other CI tests on top of those by going to your fastcheck build on Buildkite UI (linked in the PR checks section) and unblock them. If you do not have permission to unblock, ping simon-mo or khluu to add you in our Buildkite org.

Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can either: Add ready label to the PR or enable auto-merge.

🚀

[Isotr0py]

LGTM