Code

Support tensor parallel#2

Code

Merged

WoosukKwon merged 21 commits intomainvllm-project/vllm:mainfrom tensor_parallelvllm-project/vllm:tensor_parallelCopy head branch name to clipboardMar 21, 2023

+2,482-176

Commits

Commits on Feb 28, 2023

• copy code from fairseq

  zhuohan123

  committed
  e8d661c

  Copy full SHA for e8d661c

• remove files from fairscale

  zhuohan123

  committed
  827f85f

  Copy full SHA for 827f85f

• copy files from megatron

  zhuohan123

  committed
  76ed019

  Copy full SHA for 76ed019

• [WIP] add distributed init

  zhuohan123

  committed
  55e5d86

  Copy full SHA for 55e5d86

Commits on Mar 6, 2023

• Parallelize the Transformer layers

  zhuohan123

  committed
  7100db2

  Copy full SHA for 7100db2

Commits on Mar 9, 2023

• Load weight on a single GPU

  zhuohan123

  committed
  1e86393

  Copy full SHA for 1e86393

• support multi-gpu tensor parallelism

  zhuohan123

  committed
  90970e1

  Copy full SHA for 90970e1

• support tensor parallelism on multiple gpus

  zhuohan123

  committed
  88960f7

  Copy full SHA for 88960f7

• fix correctness

  zhuohan123

  committed
  900eace

  Copy full SHA for 900eace

Commits on Mar 17, 2023

• Merge branch 'main' into tensor_parallel

  zhuohan123

  committed
  6a6f7cc

  Copy full SHA for 6a6f7cc

• fix merging errors

  zhuohan123

  committed
  d5a70ab

  Copy full SHA for d5a70ab

Commits on Mar 18, 2023

• add filelock

  zhuohan123

  committed
  60bf11e

  Copy full SHA for 60bf11e

• support parallel decoding

  zhuohan123

  committed
  a7be5b8

  Copy full SHA for a7be5b8

Commits on Mar 19, 2023

• update readme

  zhuohan123

  committed
  538d067

  Copy full SHA for 538d067

• remove unused files

  zhuohan123

  committed
  893d4b3

  Copy full SHA for 893d4b3

Commits on Mar 20, 2023

• fix loading for large models

  zhuohan123

  committed
  e0f9f48

  Copy full SHA for e0f9f48

Commits on Mar 21, 2023

• Fix some smaller issues raised by Woosuk first.

  zhuohan123

  committed
  6ef5111

  Copy full SHA for 6ef5111

• Fix more review issues

  zhuohan123

  committed
  6727083

  Copy full SHA for 6727083

• remove duplicate set_seed

  zhuohan123

  committed
  ddc1ab0

  Copy full SHA for ddc1ab0

• Support the case where embedding_size != hidden_size

  zhuohan123

  committed
  1d532c5

  Copy full SHA for 1d532c5

• Resolve comments on weight loading and device id comments.

  zhuohan123

  committed
  64e3950

  Copy full SHA for 64e3950