[Refactor][Kernel] support loading kernel from other place

[ILikeIneine]

Purpose

add platform interface to support loading kernel
relate to: #25822

Test Plan

Test Result

---

Essential Elements of an Effective PR Description Checklist

• The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
• The test plan, such as providing test command.
• The test results, such as pasting the results comparison before and after, or e2e results
• (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.
• (Optional) Release notes update. If your change is user facing, please update the release notes draft in the Google Doc.

[gemini-code-assist]

Code Review

This pull request refactors kernel loading to be platform-specific by introducing import_general_kernels and import_moe_kernels methods in the Platform interface. The changes in _custom_ops.py now delegate kernel imports to the current platform. The default implementation is provided in interface.py, and TpuPlatform and XPUPlatform provide overrides.

My review identifies a couple of areas for improvement. The error logging in _custom_ops.py should be generalized to avoid confusion. Additionally, for consistency and correctness, the TpuPlatform and XPUPlatform should also override the import_moe_kernels method, similar to how import_general_kernels is handled.

[NickLucche]

Clean change thank you @ILikeIneine !
Left a minor comment agreeing with @ProExpertProg .
I am not too enthusiastic with the import_general_kernels function name, but I am also not great with names. What do you think of:
-import_core_kernels
-import_base_kernels
-import_common_kernels

Other than that this is LGTM.

[ILikeIneine]

@NickLucche @ProExpertProg Changes are updated! Still I'm using import_core_kernels, need it being try_import_core_kernels? Though it should stop if import_core_kernels failed. I don't think we should give it a try😂

[NickLucche]

Though it should stop if import_core_kernels failed

I agree in principle, I think the try-except guard we have right now it's just for cases where vllm is compiled without a device (eg you just run benchmarks).
Let's wait for @ProExpertProg to ack here.

[ProExpertProg]

Much cleaner, thanks! Name seems fine to me now

[mergify]

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @ILikeIneine.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

[ILikeIneine]

Hi, seems the pr tests are not blocked by this pr, could you take a look and re-trigger the CI? or merge directly? @NickLucche @ProExpertProg

[youkaichao]

this attribute is only used here:

vllm/vllm/_custom_ops.py

Line 1539 in 17edd8a

if supports_moe_ops and hasattr(torch.ops._moe_C, "marlin_gemm_moe"):

I wonder if we can remove supports_moe_ops and just do:

if hasattr(torch.ops, "_moe_C") and hasattr(torch.ops._moe_C, "marlin_gemm_moe"):

then having a simple import_kernels interface for the platform class would sounds better.

cc @NickLucche

[youkaichao]

@ProExpertProg do you know why we don't use if hasattr(torch.ops, "_moe_C") and hasattr(torch.ops._moe_C, "marlin_gemm_moe"): in the first place? or maybe @tlrmchlsmth @bnellnm have better ideas.

[bnellnm]

Using hasattr seems reasonable to me.

[ProExpertProg]

Yeah ++ that seems simpler