Files

benchmarks

May 29, 2024

616e600 · May 29, 2024

This branch is 7 commits ahead of, 5499 commits behind vllm-project/vllm:main.

Name	Name	Last commit message	Last commit date
parent directory ..
kernels	kernels	Marlin 24 prefill performance improvement (about 25% better on averag…	May 23, 2024
overheads	overheads	[Core][Hash][Automatic Prefix caching] Accelerating the hashing funct…	May 14, 2024
README.md	README.md	Change the name to vLLM (vllm-project#150 )	Jun 17, 2023
backend_request_func.py	backend_request_func.py	[Misc] Make Serving Benchmark More User-friendly (vllm-project#5044 )	May 25, 2024
benchmark_latency.py	benchmark_latency.py	[Misc] add gpu_memory_utilization arg (vllm-project#5079 )	May 29, 2024
benchmark_prefix_caching.py	benchmark_prefix_caching.py	[Core] Enable prefix caching with block manager v2 enabled (vllm-proj…	May 1, 2024
benchmark_serving.py	benchmark_serving.py	[Misc] Make Serving Benchmark More User-friendly (vllm-project#5044 )	May 25, 2024
benchmark_throughput.py	benchmark_throughput.py	[Misc] Load FP8 kv-cache scaling factors from checkpoints (vllm-proje…	May 22, 2024
launch_tgi_server.sh	launch_tgi_server.sh	[Doc]Add documentation to benchmarking script when running TGI (vllm-…	May 20, 2024
sonnet.txt	sonnet.txt	feat(benchmarks): Add Prefix Caching Benchmark to Serving Benchmark (v…	Mar 27, 2024

You can download the dataset by running:

wget https://huggingface.co/datasets/anon8231489123/ShareGPT_Vicuna_unfiltered/resolve/main/ShareGPT_V3_unfiltered_cleaned_split.json