[Bug]: Spec decode not allocating lookahead req for req in WAITING queue #16612
Description
🐛 Describe the bug
PR #16370 uses num lookahead tokens for computing preallocated kv blocks in EAGLE. However, it skipped for WAITING req which would be req who enter the continous batching for the 1st time or the resumed requests. cc: @LiuXiaoxuanPKU
Before submitting a new issue...
- Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.