Code

Enable Mixtral LLM model#36

Code

Open

archana-ramalingam wants to merge 21 commits intonod-ai:mainnod-ai/shark-ai:mainfrom archana-ramalingam:mixtralarchana-ramalingam/sharktank:mixtralCopy head branch name to clipboard

Commits

Commits on May 21, 2024

• Add Mixtral LLM

  archana-ramalingam

  committed
  002f959

  Copy full SHA for 002f959

Commits on May 22, 2024

• Refactoring attention, moe and ffn blocks

  archana-ramalingam

  committed
  ea9123f

  Copy full SHA for ea9123f

• Merge branch 'main' into mixtral

  archana-ramalingam

  authored
  967c524

  Copy full SHA for 967c524

• Allow _optional_int_prop to handle missing hyperparameters

  archana-ramalingam

  committed
  95e3b09

  Copy full SHA for 95e3b09

Commits on May 23, 2024

• Fixing circular dep and imports

  archana-ramalingam

  committed
  d577efb

  Copy full SHA for d577efb

Commits on May 29, 2024

• Fix multiple expert layer weight handling + other issues

  archana-ramalingam

  committed
  1346208

  Copy full SHA for 1346208

Commits on Jun 13, 2024

• Add ffn_moe layers and other fixes

  archana-ramalingam

  committed
  7f92421

  Copy full SHA for 7f92421

• Edit theta slicing

  archana-ramalingam

  committed
  8a4f5f2

  Copy full SHA for 8a4f5f2

Commits on Jun 14, 2024

• Fix ffn_moe theta parsing & wraping

  archana-ramalingam

  committed
  5e170fe

  Copy full SHA for 5e170fe

• Extract tensor unmerging into a function

  archana-ramalingam

  committed
  211b5f7

  Copy full SHA for 211b5f7

Commits on Aug 19, 2024

• Cleaning up debug statements

  archana-ramalingam

  committed
  e29c591

  Copy full SHA for e29c591

• Resolve conflict

  archana-ramalingam

  committed
  790b70b

  Copy full SHA for 790b70b

• Fix test failure

  archana-ramalingam

  committed
  d602cf1

  Copy full SHA for d602cf1

• Merge branch 'main' into mixtral

  archana-ramalingam

  authored
  9f46f45

  Copy full SHA for 9f46f45

• Merge branch 'main' into mixtral

  archana-ramalingam

  authored
  4f86051

  Copy full SHA for 4f86051

• Add rope_freq_base to llama

  archana-ramalingam

  committed
  a8e714a

  Copy full SHA for a8e714a

• Merge branch 'mixtral' of https://github.com/archana-ramalingam/sharktank into mixtral

  archana-ramalingam

  committed
  823d95c

  Copy full SHA for 823d95c

Commits on Sep 5, 2024

• Merge branch 'main' into mixtral

  archana-ramalingam

  authored
  eb91f73

  Copy full SHA for eb91f73

• Fix rotary_embedding.py

  archana-ramalingam

  authored
  dd29409

  Copy full SHA for dd29409

Commits on Nov 23, 2024

• Merge branch 'main' into mixtral

  archana-ramalingam

  authored
  b01d5a5

  Copy full SHA for b01d5a5

• Revert extra lines

  archana-ramalingam

  committed
  5a903f6

  Copy full SHA for 5a903f6