[WIP] ngram spec

[XiaotongJiang]

mvp passing unit test with torch backend

[yukavio]

Could you please share some performance results?

[merrymercy]

1. Can you compare with Speculative decoding with lookahead #2790?
2. Can you use flashinfer backend?

[XiaotongJiang]

1. Can you compare with Speculative decoding with lookahead #2790?
2. Can you use flashinfer backend?

I think it actually implemented a more general approach, this PR only support single branch strategy mentioned in the other PR. Ill close this one

[michaelfeil]

@XiaotongJiang The other branch implemented lookahead decoding, yours is more close to tok-1 proposer n-gram prompt lookup decoding.
Both are interesting and they are pretty different. It would be useful to implement a more generic version of lookahead decoding that works with prompt lookup decoding.