Code

Implement multimodal models (LLaVA)#3436

Code

Merged

monatis merged 36 commits intomasterggerganov/llama.cpp:masterfrom llavaggerganov/llama.cpp:llavaCopy head branch name to clipboardOct 12, 2023

+10,214-2

Commits

Commits on Oct 2, 2023

• WIP: start implementing LLaVA

  monatis

  committed
  59aa1ac

  Copy full SHA for 59aa1ac

• rm scratch buf for now, will revert after cleanup

  monatis

  committed
  0f0e7c6

  Copy full SHA for 0f0e7c6

• LLaVA image encoder is working. will combine with llama

  monatis

  committed
  7e9120f

  Copy full SHA for 7e9120f

Commits on Oct 3, 2023

• Add llava inference code, but it's buggy. debugging

  monatis

  committed
  d37ed47

  Copy full SHA for d37ed47

Commits on Oct 7, 2023

• LLaVA is working e2e, needs to optimize memory allocation + cleanup

  monatis

  committed
  8690f42

  Copy full SHA for 8690f42

Commits on Oct 8, 2023

• Use ggml_allocr + rm unnecessary code

  monatis

  committed
  94eeac3

  Copy full SHA for 94eeac3

• fix: crlf -> lf

  monatis

  committed
  0c2bd79

  Copy full SHA for 0c2bd79

• fix: new line at EoF

  monatis

  committed
  204d08b

  Copy full SHA for 204d08b

• fix: trailing whitespace

  monatis

  committed
  95da79e

  Copy full SHA for 95da79e

• Merge branch 'master' into llava

  monatis

  committed
  2a04d0b

  Copy full SHA for 2a04d0b

Commits on Oct 9, 2023

• Add readme

  monatis

  committed
  444dbce

  Copy full SHA for 444dbce

• Update readme

  monatis

  committed
  8af7e21

  Copy full SHA for 8af7e21

• Some cleanup

  monatis

  committed
  54495c9

  Copy full SHA for 54495c9

• Are you happy editorconfig?

  monatis

  committed
  9b0ec4d

  Copy full SHA for 9b0ec4d

• rm unused batch image preprocessing

  monatis

  committed
  8278a73

  Copy full SHA for 8278a73

• rm unused import

  monatis

  committed
  d78e816

  Copy full SHA for d78e816

• fix: rm designated initializers

  monatis

  committed
  4759bfd

  Copy full SHA for 4759bfd

• introduce pad-to-square mode for non-square images

  monatis

  committed
  325d240

  Copy full SHA for 325d240

• are you happy editorconfig?

  monatis

  committed
  d75a031

  Copy full SHA for d75a031

• gitignore /llava

  monatis

  committed
  ae01c85

  Copy full SHA for ae01c85

• Handle cases where image file does not exist

  monatis

  committed
  5009ae9

  Copy full SHA for 5009ae9

• add llava target to Makefile

  monatis

  committed
  96171de

  Copy full SHA for 96171de

Commits on Oct 10, 2023

• add support for 13b model variant

  monatis

  committed
  d640aae

  Copy full SHA for d640aae

Commits on Oct 11, 2023

• Maybe seed is unlucky?

  monatis

  committed
  587bde8

  Copy full SHA for 587bde8

• Merge branch 'master' into llava

  monatis

  committed
  f1564bb

  Copy full SHA for f1564bb

• Check if apples are compared to apples

  monatis

  committed
  ab21587

  Copy full SHA for ab21587

• are you happy editorconfig?

  monatis

  committed
  0409ae0

  Copy full SHA for 0409ae0

• Use temperature = 0.1 by default

  monatis

  committed
  f0f7834

  Copy full SHA for f0f7834

• command line: use gpt_params_parse()

  monatis

  committed
  2bc1710

  Copy full SHA for 2bc1710

• Merge master and fix conflicts

  monatis

  committed
  1403d87

  Copy full SHA for 1403d87

Commits on Oct 12, 2023

• minor

  monatis

  committed
  dc913ea

  Copy full SHA for dc913ea

• handle default n_predict

  monatis

  committed
  56ccf97

  Copy full SHA for 56ccf97

• fix typo

  monatis

  committed
  e9534ea

  Copy full SHA for e9534ea

• Merge branch 'master' into llava

  ggerganov

  committed
  346e3c1

  Copy full SHA for 346e3c1

• llava : code formatting, rename files, fix compile warnings

  ggerganov

  committed
  4bc5c9c

  Copy full SHA for 4bc5c9c

• do not use Wno-cast-qual for MSVC

  monatis

  committed
  0bd7e69

  Copy full SHA for 0bd7e69