Skip to content

Pull requests: huggingface/text-generation-inference

New pull request New
27 Open 1,226 Closed
27 Open 1,226 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Reviews
Filter by reviews
No reviews Review required Approved review Changes requested
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Pull requests list

Fix the tool_choice format for named choice by adapting OpenAIs scheme
#2634 opened Oct 10, 2024 by linusbierhoff Loading…
propagate caught TERM/INT/HUP signals from entrypoint to tgi child process
#2633 opened Oct 10, 2024 by oOraph Loading…
Make moe-kernels and marlin-kernels mandatory in CUDA installs
#2632 opened Oct 10, 2024 by danieldk Loading…
5 tasks
Clarify gated description and quicktour
#2631 opened Oct 9, 2024 by osanseviero Loading…
feat: add basic test for the warmup step and memory allocation of the…
#2629 opened Oct 9, 2024 by drbh Loading…
Add support for FP8 KV cache scales
#2628 opened Oct 9, 2024 by danieldk • Draft
5 tasks
Break cycle between the attention implementations and KV cache
#2627 opened Oct 9, 2024 by danieldk • Draft
5 tasks
1
feat: add test for positional rotary embeddings
#2623 opened Oct 8, 2024 by drbh Loading…
Test Marlin MoE with desc_act=true
#2622 opened Oct 8, 2024 by danieldk Loading…
5 tasks
[DOCS] Add Google Cloud TGI integration via dedicated DLCs
#2612 opened Oct 5, 2024 by alvarobartt Loading…
1 of 5 tasks
3
Simplify the attention function
#2609 opened Oct 4, 2024 by danieldk Loading…
5 tasks
10
feat: prefill chunking
#2600 opened Oct 2, 2024 by OlivierDehaene Loading…
4
Cpu perf
#2596 opened Oct 1, 2024 by Narsil Loading…
5 tasks
feat: Add automatic nightly benchmarks
#2591 opened Sep 30, 2024 by Hugoch Loading…
1 of 5 tasks
@Hugoch
Fp8 e4m3_fnuz support for rocm
#2588 opened Sep 30, 2024 by mht-sharma Loading…
5 tasks
7
break when there's nothing to read
#2582 opened Sep 28, 2024 by sywangyi Loading…
5 tasks
9
feat: propagate max_concurrent_requests to queue state entries instead of hardcoded value in v2 and v3 backends
#2578 opened Sep 26, 2024 by Venkat2811 Loading…
1
feat: enable pytorch xpu support for non-attention models
#2561 opened Sep 24, 2024 by dvrogozh Loading…
11
We can have a tokenizer anywhere.
#2527 opened Sep 17, 2024 by Narsil Loading…
5 tasks
4
CI for add gptq and awq int4 support in intel platform
#2494 opened Sep 5, 2024 by ErikKaum Loading…
3
Small fixes for supported models
#2471 opened Aug 29, 2024 by osanseviero Loading…
12
fix: skip cuda graphs that will oom and improve free memory logging
#2450 opened Aug 22, 2024 by drbh Loading…
add gptq and awq int4 support in intel platform
#2444 opened Aug 22, 2024 by sywangyi Loading…
5 tasks
@ErikKaum
12
Improve vlm support (add idefics3 support)
#2437 opened Aug 20, 2024 by drbh • Draft
4 tasks
[TENSORRT-LLM] - Implement new looper thread based backend
#2357 opened Aug 2, 2024 by mfuntowicz • Draft
ProTip! Filter pull requests by the default branch with base:main.