File tree Expand file tree Collapse file tree 1 file changed +8
-6
lines changed Expand file tree Collapse file tree 1 file changed +8
-6
lines changed Original file line number Diff line number Diff line change 33| Feature | Supported | Note |
44| ---------| -----------| ------|
55| Chunked Prefill | ✗ | Plan in 2025 Q1 |
6- | Automatic Prefix Caching | ✅ | Improve performance in 2025 Q1 |
6+ | Automatic Prefix Caching | ✅ | Improve performance in 2025 Q2 |
77| LoRA | ✗ | Plan in 2025 Q1 |
8- | Prompt adapter | ✅ | |
9- | Speculative decoding | ✅ | Improve accuracy in 2025 Q1|
10- | Pooling | ✗ | Plan in 2025 Q1 |
11- | Enc-dec | ✗ | Plan in 2025 Q1 |
8+ | Prompt adapter | ✗ | Plan in 2025 Q1 |
9+ | Speculative decoding | ✗ | Plan in 2025 Q1 |
10+ | Pooling | ✗ | Plan in 2025 Q2 |
11+ | Enc-dec | ✗ | Plan in 2025 Q2 |
1212| Multi Modality | ✅ (LLaVA/Qwen2-vl/Qwen2-audio/internVL)| Add more model support in 2025 Q1 |
1313| LogProbs | ✅ ||
1414| Prompt logProbs | ✅ ||
1515| Async output | ✅ ||
16- | Multi step scheduler | ✅ | |
16+ | Multi step scheduler | ✗ | Plan in 2025 Q1 |
1717| Best of | ✅ ||
1818| Beam search | ✅ ||
1919| Guided Decoding | ✗ | Plan in 2025 Q1 |
20+ | Tensor Parallel | ✅ | Only "mp" supported now |
21+ | Pipeline Parallel | ✅ | Only "mp" supported now |
You can’t perform that action at this time.
0 commit comments