Code

[Core] generate from input embeds#6869

Code

Open

Nan2018 wants to merge 126 commits intovllm-project:mainvllm-project/vllm:mainfrom Nan2018:feature-input-embedsNan2018/vllm:feature-input-embedsCopy head branch name to clipboard

+885-236

Commits

Commits on Jul 27, 2024

• feat: add support for generate from prompt embeddings

  Nan2018

  committed
  ed8eb22

  Copy full SHA for ed8eb22

• Merge remote-tracking branch 'vllm/main' into feature-input-embeds

  Nan2018

  committed
  7ccbf01

  Copy full SHA for 7ccbf01

• Merge branch 'vllm-project:main' into feature-input-embeds

  Nan2018

  authored
  7c663a1

  Copy full SHA for 7c663a1

• fix ci errors

  Nan2018

  committed
  3bd6423

  Copy full SHA for 3bd6423

Commits on Aug 5, 2024

• Merge remote-tracking branch 'vllm/main' into feature-input-embeds

  Nan2018

  committed
  86777a7

  Copy full SHA for 86777a7

Commits on Aug 6, 2024

• fix: tensor parallel

  Nan2018

  committed
  9a9d406

  Copy full SHA for 9a9d406

• style: yapf

  Nan2018

  committed
  48fc6a8

  Copy full SHA for 48fc6a8

• fix: model_runner in a WorkerWrapper

  Nan2018

  committed
  737d01b

  Copy full SHA for 737d01b

Commits on Aug 7, 2024

• Merge remote-tracking branch 'vllm/main' into feature-input-embeds

  Nan2018

  committed
  4b99109

  Copy full SHA for 4b99109

• fix: spec decoding model

  Nan2018

  committed
  03344ab

  Copy full SHA for 03344ab

• fix: ruff

  Nan2018

  committed
  40038e0

  Copy full SHA for 40038e0

Commits on Aug 8, 2024

• fix: move param prompt_embeds_shape to the last of RequestOutput

  Nan2018

  committed
  b00bbc7

  Copy full SHA for b00bbc7

• feat: all *ForCausalLM models support inputs_embeds

  Nan2018

  committed
  3c1a6fa

  Copy full SHA for 3c1a6fa

• fix: format

  Nan2018

  committed
  c454647

  Copy full SHA for c454647

• fix: format

  Nan2018

  committed
  535ad97

  Copy full SHA for 535ad97

• fix: format

  Nan2018

  committed
  c05a8ff

  Copy full SHA for c05a8ff

Commits on Sep 4, 2024

• Merge remote-tracking branch 'vllm/main' into feature-input-embeds

  Nan2018

  committed
  2b50573

  Copy full SHA for 2b50573

• Merge remote-tracking branch 'vllm/main' into feature-input-embeds

  Nan2018

  committed
  7ddc863

  Copy full SHA for 7ddc863

• fix: engines

  Nan2018

  committed
  d83915b

  Copy full SHA for d83915b

• fix: format

  Nan2018

  committed
  dfd9301

  Copy full SHA for dfd9301

• fix: format

  Nan2018

  committed
  fd455eb

  Copy full SHA for fd455eb

Commits on Sep 5, 2024

• fix: sequence property and embeddings for phi3v

  Nan2018

  committed
  03fcf3b

  Copy full SHA for 03fcf3b

• Merge remote-tracking branch 'vllm/main' into feature-input-embeds

  Nan2018

  committed
  40f485b

  Copy full SHA for 40f485b

Commits on Sep 6, 2024

• fix: ultravox

  Nan2018

  committed
  a72930f

  Copy full SHA for a72930f

• refactor: rename parameter

  Nan2018

  committed
  5e3eec9

  Copy full SHA for 5e3eec9

Commits on Sep 11, 2024

• Merge remote-tracking branch 'vllm/main' into feature-input-embeds

  Nan2018

  committed
  beba406

  Copy full SHA for beba406

• Merge remote-tracking branch 'vllm/main' into feature-input-embeds

  Nan2018

  committed
  5cde3b4

  Copy full SHA for 5cde3b4

• refactor: supports_input_embeds

  Nan2018

  committed
  2b39026

  Copy full SHA for 2b39026

Commits on Sep 12, 2024

• Merge remote-tracking branch 'vllm/main' into feature-input-embeds

  Nan2018

  committed
  29525e1

  Copy full SHA for 29525e1

• feat: inputs_embeds for new models

  Nan2018

  committed
  49fe3f7

  Copy full SHA for 49fe3f7

Commits on Sep 17, 2024

• Merge remote-tracking branch 'vllm/main' into feature-input-embeds

  Nan2018

  committed
  40ca516

  Copy full SHA for 40ca516

Commits on Sep 19, 2024

• Fix typing

  DarkLight1337

  committed
  56b9ac5

  Copy full SHA for 56b9ac5

• Support embeds in minicpm

  DarkLight1337

  committed
  decb8ab

  Copy full SHA for decb8ab

• Fix typing 2

  DarkLight1337

  committed
  ee41cb7

  Copy full SHA for ee41cb7

• Disable inputs_embeds for multimodal models as it conflicts with multimodal embeds

  DarkLight1337

  committed
  fd58d4b

  Copy full SHA for fd58d4b

• Reformat

  DarkLight1337

  committed
  5849509

  Copy full SHA for 5849509

• Merge branch 'main' into feature-input-embeds

  DarkLight1337

  committed
  f03d980

  Copy full SHA for f03d980

• Optimize

  DarkLight1337

  committed
  23a4876

  Copy full SHA for 23a4876

• Fix unbound variables

  DarkLight1337

  committed
  fbf3f10

  Copy full SHA for fbf3f10

• Cleanup

  DarkLight1337

  committed
  9822652

  Copy full SHA for 9822652

• Cleanup 2

  DarkLight1337

  committed
  53962c4

  Copy full SHA for 53962c4

• Fix unbound prompt_embeds in validation and clean it up

  DarkLight1337

  committed
  b8137aa

  Copy full SHA for b8137aa

• Indent

  DarkLight1337

  committed
  2cf3b4b

  Copy full SHA for 2cf3b4b

• Fix unbound local and cleanup 2

  DarkLight1337

  committed
  716b64a

  Copy full SHA for 716b64a

• fix: gemma

  Nan2018

  committed
  7dd3d86

  Copy full SHA for 7dd3d86

Commits on Sep 20, 2024

• Have two distinct SequenceData classes, one for tokens and one for embeddings

  DarkLight1337

  committed
  8059014

  Copy full SHA for 8059014

• Rename PromptInputs to PromptType

  DarkLight1337

  committed
  7f8ed8c

  Copy full SHA for 7f8ed8c

• Fix type errors

  DarkLight1337

  committed
  89b5753

  Copy full SHA for 89b5753

• format

  DarkLight1337

  committed
  8e91af3

  Copy full SHA for 8e91af3

• Rename LLMInputs to DecoderOnlyInputs and fix input processing for VLMs

  DarkLight1337

  committed
  31e2a1b

  Copy full SHA for 31e2a1b

• Fix error on import

  DarkLight1337

  committed
  60bc7b5

  Copy full SHA for 60bc7b5

• Revert class splitting

  DarkLight1337

  committed
  a8483a4

  Copy full SHA for a8483a4

• Fix init error for embeds

  DarkLight1337

  committed
  f451192

  Copy full SHA for f451192

• Be a bit more efficient

  DarkLight1337

  committed
  3d436e9

  Copy full SHA for 3d436e9

• Rename

  DarkLight1337

  committed
  dfecf4b

  Copy full SHA for dfecf4b

• Rename 2

  DarkLight1337

  committed
  29b9d5c

  Copy full SHA for 29b9d5c

• Fix encoder-decoder test prompts

  DarkLight1337

  committed
  d5ec13a

  Copy full SHA for d5ec13a

• Fix validation for encoder-decoder models

  DarkLight1337

  committed
  b5632f9

  Copy full SHA for b5632f9

• Merge branch 'main' into feature-input-embeds

  DarkLight1337

  committed
  881e6da

  Copy full SHA for 881e6da

• Fix naming

  DarkLight1337

  committed
  3da5ad6

  Copy full SHA for 3da5ad6

• Rename PromptInputs to PromptType, and inputs to prompt

  DarkLight1337

  committed
  94ace38

  Copy full SHA for 94ace38

• Remove unnecessary comments

  DarkLight1337

  committed
  065a304

  Copy full SHA for 065a304

• Merge branch 'rename-prompt' into feature-input-embeds

  DarkLight1337

  committed
  affbd12

  Copy full SHA for affbd12

Commits on Sep 21, 2024

• Merge branch 'main' into feature-input-embeds

  DarkLight1337

  committed
  2e3cb4a

  Copy full SHA for 2e3cb4a

• Merge branch 'main' into feature-input-embeds

  DarkLight1337

  committed
  e5a771a

  Copy full SHA for e5a771a

Commits on Sep 22, 2024

• Merge branch 'main' into feature-input-embeds

  DarkLight1337

  committed
  dfc108d

  Copy full SHA for dfc108d

• Fix import

  DarkLight1337

  committed
  741d4c1

  Copy full SHA for 741d4c1

• Format

  DarkLight1337

  committed
  9350022

  Copy full SHA for 9350022

Commits on Sep 28, 2024

• Merge branch 'main' into feature-input-embeds

  DarkLight1337

  committed
  d29d563

  Copy full SHA for d29d563

• Add validation for embedding inputs

  DarkLight1337

  committed
  1de2b99

  Copy full SHA for 1de2b99

• Update for mllama

  DarkLight1337

  committed
  9124115

  Copy full SHA for 9124115

• format

  DarkLight1337

  committed
  6c366eb

  Copy full SHA for 6c366eb

Commits on Oct 1, 2024

• Merge branch 'main' into feature-input-embeds

  DarkLight1337

  committed
  280596c

  Copy full SHA for 280596c

• Fix failing tests

  DarkLight1337

  committed
  ad6c364

  Copy full SHA for ad6c364

Commits on Oct 5, 2024

• Merge branch 'main' into feature-input-embeds

  DarkLight1337

  committed
  529e91d

  Copy full SHA for 529e91d

• format

  DarkLight1337

  committed
  13bbd02

  Copy full SHA for 13bbd02

Commits on Oct 7, 2024

• Merge branch 'main' into feature-input-embeds

  DarkLight1337

  committed
  0946dc6

  Copy full SHA for 0946dc6

• Improve type annotations

  DarkLight1337

  committed
  4000b90

  Copy full SHA for 4000b90

Commits on Oct 16, 2024

• Merge branch 'main' into feature-input-embeds

  DarkLight1337

  committed
  7a26dd0

  Copy full SHA for 7a26dd0

• Update

  DarkLight1337

  committed
  1bafe1b

  Copy full SHA for 1bafe1b

Commits on Oct 22, 2024

• Merge branch 'main' into feature-input-embeds

  DarkLight1337

  committed
  0d6331d

  Copy full SHA for 0d6331d

• Fix KeyError; debug

  DarkLight1337

  committed
  2da9eea

  Copy full SHA for 2da9eea

Commits on Oct 23, 2024

• Make encoder-decoder inputs a composed structure

  DarkLight1337

  committed
  0c872b3

  Copy full SHA for 0c872b3

• Merge branch 'main' into refactor-preprocessing

  DarkLight1337

  committed
  9287a1b

  Copy full SHA for 9287a1b

• Merge branch 'refactor-preprocessing' into feature-input-embeds

  DarkLight1337

  committed
  70db2db

  Copy full SHA for 70db2db

• Rename

  DarkLight1337

  committed
  fa5ad17

  Copy full SHA for fa5ad17

• Fix type error

  DarkLight1337

  committed
  44fd058

  Copy full SHA for 44fd058

• Merge branch 'refactor-preprocessing' into feature-input-embeds

  DarkLight1337

  committed
  3ce9236

  Copy full SHA for 3ce9236

• Fix bad merge

  DarkLight1337

  committed
  d167df3

  Copy full SHA for d167df3

• Fix test

  DarkLight1337

  committed
  b73a345

  Copy full SHA for b73a345

• Fix llama-3.2

  DarkLight1337

  committed
  fa968b5

  Copy full SHA for fa968b5

• Merge branch 'refactor-preprocessing' into feature-input-embeds

  DarkLight1337

  committed
  6e0934c

  Copy full SHA for 6e0934c

• Fix wrong variable

  DarkLight1337

  committed
  5ccc390

  Copy full SHA for 5ccc390

• Impl get_inputs_embeds

  DarkLight1337

  committed
  2fe159c

  Copy full SHA for 2fe159c

• Fix tests

  DarkLight1337

  committed
  7986553

  Copy full SHA for 7986553

• format

  DarkLight1337

  committed
  4c072ca

  Copy full SHA for 4c072ca

• Don't use prompt_embeds in get_len

  DarkLight1337

  committed
  3b22bbc

  Copy full SHA for 3b22bbc

Commits on Oct 24, 2024

• Remove force_bos

  DarkLight1337

  committed
  906ee1e

  Copy full SHA for 906ee1e

• Add example output

  DarkLight1337

  committed
  005ad95

  Copy full SHA for 005ad95

• format

  DarkLight1337

  committed
  a5f0c16

  Copy full SHA for a5f0c16

• Fix

  DarkLight1337

  committed
  6ab44e4

  Copy full SHA for 6ab44e4

Commits on Oct 29, 2024

• Merge branch 'main' into refactor-preprocessing

  DarkLight1337

  committed
  21be11f

  Copy full SHA for 21be11f

Commits on Oct 31, 2024

• Merge branch 'main' into refactor-preprocessing

  DarkLight1337

  committed
  1f927d2

  Copy full SHA for 1f927d2

• Fix merge

  DarkLight1337

  committed
  760db05

  Copy full SHA for 760db05

• Update mllama processing

  DarkLight1337

  committed
  acb8e6f

  Copy full SHA for acb8e6f

• Fix line

  DarkLight1337

  committed
  3bed519

  Copy full SHA for 3bed519

• format

  DarkLight1337

  committed
  ea861e0

  Copy full SHA for ea861e0

• Avoid repeated lookups

  DarkLight1337

  committed
  f654421

  Copy full SHA for f654421

• Remove unused import

  DarkLight1337

  committed
  594794e

  Copy full SHA for 594794e

• Fix mypy

  DarkLight1337

  committed
  08ea824

  Copy full SHA for 08ea824

• Merge branch 'main' into refactor-preprocessing

  DarkLight1337

  committed
  b622f41

  Copy full SHA for b622f41

• Merge branch 'main' into refactor-preprocessing

  DarkLight1337

  committed
  800960d

  Copy full SHA for 800960d

• Fix merge

  DarkLight1337

  committed
  283bc2c

  Copy full SHA for 283bc2c

Commits on Nov 2, 2024

• Merge branch 'main' into refactor-preprocessing

  Show description for e8169ea
  DarkLight1337

  committed
  e8169ea

  Copy full SHA for e8169ea

Commits on Nov 3, 2024

• Merge branch 'main' into refactor-preprocessing

  DarkLight1337

  committed
  61bf1d1

  Copy full SHA for 61bf1d1

• Fix missing import

  Show description for b45cdc9
  DarkLight1337

  committed
  b45cdc9

  Copy full SHA for b45cdc9

• Improve error message

  Show description for 4d33b1e
  DarkLight1337

  committed
  4d33b1e

  Copy full SHA for 4d33b1e

• Add missing export

  Show description for 0a549e5
  DarkLight1337

  committed
  0a549e5

  Copy full SHA for 0a549e5

• Improve error message.

  DarkLight1337

  authored
  f741a75

  Copy full SHA for f741a75

• Format

  DarkLight1337

  authored
  cd231fa

  Copy full SHA for cd231fa

Commits on Nov 4, 2024

• Merge branch 'refactor-preprocessing' into feature-input-embeds

  Show description for e4f3a93
  DarkLight1337

  committed
  e4f3a93

  Copy full SHA for e4f3a93

Commits on Nov 5, 2024

• Merge branch 'main' into feature-input-embeds

  Show description for c61d246
  DarkLight1337

  committed
  c61d246

  Copy full SHA for c61d246

Commits on Nov 6, 2024

• Merge branch 'main' into feature-input-embeds

  Show description for b8aaa8e
  DarkLight1337

  committed
  b8aaa8e

  Copy full SHA for b8aaa8e

• Update get_inputs_embeds to be compatible with torch.compile

  Show description for c8fc1fe
  DarkLight1337

  committed
  c8fc1fe

  Copy full SHA for c8fc1fe

• Merge branch 'main' into feature-input-embeds

  DarkLight1337

  committed
  85124f7

  Copy full SHA for 85124f7

• Merge branch 'main' into feature-input-embeds

  DarkLight1337

  committed
  a7429ad

  Copy full SHA for a7429ad