Code

Support tensor parallel#2

Code

Merged

WoosukKwon merged 21 commits intomainvllm-project/vllm:mainfrom tensor_parallelvllm-project/vllm:tensor_parallelCopy head branch name to clipboardMar 21, 2023

+2,482-176

Commits

Commits on Feb 28, 2023

• copy code from fairseq

  zhuohan123

  committed
  e8d661c

• remove files from fairscale

  zhuohan123

  committed
  827f85f

• copy files from megatron

  zhuohan123

  committed
  76ed019

• [WIP] add distributed init

  zhuohan123

  committed
  55e5d86

Commits on Mar 6, 2023

• Parallelize the Transformer layers

  zhuohan123

  committed
  7100db2

Commits on Mar 9, 2023

• Load weight on a single GPU

  zhuohan123

  committed
  1e86393

• support multi-gpu tensor parallelism

  zhuohan123

  committed
  90970e1

• support tensor parallelism on multiple gpus

  zhuohan123

  committed
  88960f7

• fix correctness

  zhuohan123

  committed
  900eace

Commits on Mar 17, 2023

• Merge branch 'main' into tensor_parallel

  zhuohan123

  committed
  6a6f7cc

• fix merging errors

  zhuohan123

  committed
  d5a70ab

Commits on Mar 18, 2023

• add filelock

  zhuohan123

  committed
  60bf11e

• support parallel decoding

  zhuohan123

  committed
  a7be5b8

Commits on Mar 19, 2023

• update readme

  zhuohan123

  committed
  538d067

• remove unused files

  zhuohan123

  committed
  893d4b3

Commits on Mar 20, 2023

• fix loading for large models

  zhuohan123

  committed
  e0f9f48

Commits on Mar 21, 2023

• Fix some smaller issues raised by Woosuk first.

  zhuohan123

  committed
  6ef5111

• Fix more review issues

  zhuohan123

  committed
  6727083

• remove duplicate set_seed

  zhuohan123

  committed
  ddc1ab0

• Support the case where embedding_size != hidden_size

  zhuohan123

  committed
  1d532c5

• Resolve comments on weight loading and device id comments.

  zhuohan123

  committed
  64e3950