Code

Mod vllm generate#833

Code

Merged

Qubitium merged 12 commits intoModelCloud:mainModelCloud/GPTQModel:mainfrom LRL-ModelCloud:mod-vllm-generateLRL-ModelCloud/GPTQModel:mod-vllm-generateCopy head branch name to clipboardDec 13, 2024

+113-58

Commits

Commits on Nov 16, 2024

• mod vllm generate

  

  LRL-ModelCloud

  committed
  68b168f

Commits on Dec 13, 2024

• Merge branch 'refs/heads/main' into mod-vllm-generate

  LRL-ModelCloud

  committed
  c3fe88c

• fix vllm get device

  LRL-ModelCloud

  committed
  e3668f3

• Merge branch 'refs/heads/main' into mod-vllm-generate

  LRL-ModelCloud

  committed
  1e2c343

• fix vllm/gptq_marlin

  LRL-ModelCloud

  committed
  7bd70b5

• fix vllm generate

  LRL-ModelCloud

  committed
  6046de7

• fix test_vllm

  LRL-ModelCloud

  committed
  85a1d13

• cleanup

  LRL-ModelCloud

  committed
  d6b2d84

• set max_tokens

  LRL-ModelCloud

  committed
  afb7deb

• cleanup

  LRL-ModelCloud

  committed
  7ef8ebc

• fix vllm samplingparams

  LRL-ModelCloud

  committed
  1533f77

• use top_k=1

  LRL-ModelCloud

  committed
  78e9243