added gemma2 9b and 27b vllm with streaming#318

Merged

dsingal0 merged 11 commits intobasetenlabs:mainfrom dsingal0:gemma2

Jul 16, 2024

+330

Commits on Jul 2, 2024

Commits on Jul 7, 2024

Merge branch 'basetenlabs:main' into gemma2
dsingal0
authored
added vllm implementation. cleanup local-gemma implementation. updated local-gemma readme.
dsingal0
committed

Commits on Jul 11, 2024

Commits on Jul 15, 2024

removed local-gemma implementations. added HF tokenizer for vllm chat template. todo: confirm 27B working
dsingal0
committed
bump vllm and flashinfer version. multiprocessing fix for CUDA with TP > 1
dsingal0
committed

Commits on Jul 16, 2024