torch.compile based model optimizer

[bnellnm]

This PR implements the first cut of the optimizer decribed here: [RFC] A Graph Optimization System in vLLM using torch.compile

Issue: #6378

[peaceorwell]

@bnellnm I'd like to ask, I didn't see any pattern matching related code in the PR. Did I miss it?

[peaceorwell]

I have a small question. The operator has already been registered to torch in torch_binding.cpp above, why does it need to be registered again here?

[bnellnm]

This version of silu_mul_quant includes allocating the output/tmp buffers. The function names here don't really mean anything since we will be going by the torch op name (but probably should be changed to avoid confusion).

[bnellnm]

@bnellnm I'd like to ask, I didn't see any pattern matching related code in the PR. Did I miss it?

We don't use pytorch's pattern matching routines. The code in fusion.py searches for and combines fusable operations.

[bnellnm]

I think this should be

    if isinstance(n, float):
        ty = "float"
    elif isinstance(n, int):
        ty = "int"
    elif n.type is not None:
        ...

so that the proper C++/Python type is returned.

[bnellnm]

I think the signature of this function should remain as it was before so that we could do a proper job of generating a meta function at some point in the future. The compose stuff doesn't really work for non-unary meta functions.

[lnykww]

@bnellnm is this feature still ongoing？

[bnellnm]

lnykww

@bnellnm is this feature still ongoing？

It is, although this PR will probably be superceded by others.

[hmellor]

Closing as torch.compile support is now added