Code

Add OLMo model family#29890

Code

Merged

ArthurZucker merged 71 commits intohuggingface:mainhuggingface/transformers:mainfrom 2015aroras:add-olmo-implementation2015aroras/transformers:add-olmo-implementationCopy head branch name to clipboardApr 17, 2024

+2,482-3

Commits

Commits on Mar 26, 2024

• Add OLMo using add-new-model-like with Llama

  2015aroras

  committed
  0afbd4a

  Copy full SHA for 0afbd4a

• Fix incorrect tokenizer for OLMo

  2015aroras

  committed
  9842c5c

  Copy full SHA for 9842c5c

• Copy-paste relevant OLMo methods and their imports

  2015aroras

  committed
  fc21050

  Copy full SHA for fc21050

• Add OLMo config

  2015aroras

  committed
  3a748b9

  Copy full SHA for 3a748b9

• Modify OLMo config to follow HF conventions

  2015aroras

  committed
  e15ed3c

  Copy full SHA for e15ed3c

• Remove unneeded Llama code from OLMo model

  2015aroras

  committed
  0090b1a

  Copy full SHA for 0090b1a

• Add ability for OLMo model to output attentions

  2015aroras

  committed
  8719408

  Copy full SHA for 8719408

• Add OLMoPreTrainedModel and OLMoModel

  2015aroras

  committed
  8b1f082

  Copy full SHA for 8b1f082

• Add OLMoForCausalLM

  2015aroras

  committed
  49bb675

  Copy full SHA for 49bb675

• Minor fixes to OLMo model for style and missing functions

  2015aroras

  committed
  32ec380

  Copy full SHA for 32ec380

• Implement OLMo tokenizer

  2015aroras

  committed
  cba91d4

  Copy full SHA for cba91d4

• Implement OLMo to HF conversion script

  2015aroras

  committed
  7cc2a9a

  Copy full SHA for 7cc2a9a

• Add tests for OLMo model

  2015aroras

  committed
  a1f9294

  Copy full SHA for a1f9294

• Add tests for OLMo fast tokenizer

  2015aroras

  committed
  f4fa804

  Copy full SHA for f4fa804

• Add auto-generated dummy objects

  2015aroras

  committed
  7b287ca

  Copy full SHA for 7b287ca

• Remove unimplemented OLMo classes from auto and init classes and re-format

  2015aroras

  committed
  d1ba9c7

  Copy full SHA for d1ba9c7

• Add README and associated auto-generated files

  2015aroras

  committed
  159b124

  Copy full SHA for 159b124

Commits on Mar 27, 2024

• Use OLMo names for common properties

  2015aroras

  committed
  8d8baf5

  Copy full SHA for 8d8baf5

• Run make fixup

  2015aroras

  committed
  42492b9

  Copy full SHA for 42492b9

• Remove | from OLMo typing

  2015aroras

  committed
  a92d52a

  Copy full SHA for a92d52a

• Remove unneeded tokenization_olmo.py

  2015aroras

  committed
  f59ea67

  Copy full SHA for f59ea67

Commits on Mar 30, 2024

• Revert model, config and converter to add-new-model-like Llama

  2015aroras

  committed
  4bc9b6b

  Copy full SHA for 4bc9b6b

• Move logic for adding bos/eos token into GPTNeoxTokenizerFast

  2015aroras

  committed
  4fdd1da

  Copy full SHA for 4fdd1da

• Change OLMoConfig defaults to match OLMo-7B

  2015aroras

  committed
  3e8ceb1

  Copy full SHA for 3e8ceb1

• Use GPTNeoXToknizerFast in OLMo tokenizer tests

  2015aroras

  committed
  f68a915

  Copy full SHA for f68a915

• Modify auto-generated OLMoModelTests to work for OLMo

  2015aroras

  committed
  43eaeff

  Copy full SHA for 43eaeff

• Add non-parametric layer norm OLMoLayerNorm

  2015aroras

  committed
  6dd1852

  Copy full SHA for 6dd1852

• Update weight conversion script for OLMo

  2015aroras

  committed
  7b438cd

  Copy full SHA for 7b438cd

• Fix __init__ and auto structure for OLMo

  2015aroras

  committed
  3346195

  Copy full SHA for 3346195

• Fix errors from make fixup

  2015aroras

  committed
  7ba083f

  Copy full SHA for 7ba083f

• Remove OLMoTokenizerFast from documentation

  2015aroras

  committed
  47affa6

  Copy full SHA for 47affa6

Commits on Apr 5, 2024

• Merge remote-tracking branch 'upstream/main' into add-olmo-implementation

  2015aroras

  committed
  b468a0a

  Copy full SHA for b468a0a

• Add missing 'Copied from' for OLMoModel._update_causal_mask

  2015aroras

  committed
  98db6a9

  Copy full SHA for 98db6a9

• Run make fix-copies

  2015aroras

  committed
  dd0f76c

  Copy full SHA for dd0f76c

• Rearrange string replacements in OLMoForCausalLM Copied from

  2015aroras

  committed
  0340778

  Copy full SHA for 0340778

• Move OLMo and Llama CausalLM.forward example into global constants

  2015aroras

  committed
  edbc4d1

  Copy full SHA for edbc4d1

Commits on Apr 6, 2024

• Fix OLMO_GENERATION_EXAMPLE doc string typo

  2015aroras

  committed
  75f5cee

  Copy full SHA for 75f5cee

Commits on Apr 10, 2024

• Add option for qkv clipping to OLMo

  2015aroras

  committed
  94b31e1

  Copy full SHA for 94b31e1

• Rearrange OLMoConfig kwargs in convert_olmo_weights_to_hf

  2015aroras

  committed
  3805b97

  Copy full SHA for 3805b97

• Add clip_qkv to OLMoConfig in convert_olmo_weights_to_hf

  2015aroras

  committed
  a555819

  Copy full SHA for a555819

Commits on Apr 12, 2024

• Fix OLMo tokenization bug using conversion script

  2015aroras

  committed
  aea6b06

  Copy full SHA for aea6b06

• Keep model in full precision after conversion

  2015aroras

  committed
  0bbb55a

  Copy full SHA for 0bbb55a

• Do not add eos token automatically

  2015aroras

  committed
  21faf45

  Copy full SHA for 21faf45

• Update references to OLMo model in HF Hub

  2015aroras

  committed
  ee00583

  Copy full SHA for ee00583

• Do not add eos token during encoding by default

  2015aroras

  committed
  63a6fbd

  Copy full SHA for 63a6fbd

• Fix Llama generation example

  2015aroras

  committed
  76c7970

  Copy full SHA for 76c7970

• Run make fixup

  2015aroras

  committed
  450f038

  Copy full SHA for 450f038

Commits on Apr 13, 2024

• OLMo 7B integration test fix

  2015aroras

  committed
  4df56a4

  Copy full SHA for 4df56a4

• Merge branch 'main' into add-olmo-implementation

  2015aroras

  committed
  faa188f

  Copy full SHA for faa188f

• Remove unneeded special case for OLMoConfig

  2015aroras

  committed
  0b227bb

  Copy full SHA for 0b227bb

• OLMo 7B Twin 2T integration test fix

  2015aroras

  committed
  9ff65a4

  Copy full SHA for 9ff65a4

• Fix test_model_7b_greedy_generation

  2015aroras

  committed
  70f593d

  Copy full SHA for 70f593d

• Remove test_compile_static_cache

  2015aroras

  committed
  54f4aa5

  Copy full SHA for 54f4aa5

• Fix OLMo and Llama generation example

  2015aroras

  committed
  33d3be7

  Copy full SHA for 33d3be7

• Run make fixup

  2015aroras

  committed
  eab18b6

  Copy full SHA for eab18b6

Commits on Apr 15, 2024

• Revert "OLMo 7B integration test fix"

  Show description for 628f9d5
  2015aroras

  committed
  628f9d5

  Copy full SHA for 628f9d5

• Revert "OLMo 7B Twin 2T integration test fix"

  Show description for 751195f
  2015aroras

  committed
  751195f

  Copy full SHA for 751195f

• Ungate 7B integration tests and fix greedy generation test

  2015aroras

  committed
  68466dd

  Copy full SHA for 68466dd

• Add retries for flaky test_eager_matches_sdpa_generate

  2015aroras

  committed
  91f163f

  Copy full SHA for 91f163f

• Fix output of doc example for OLMoForCausalLM.forward

  2015aroras

  committed
  aa4a92f

  Copy full SHA for aa4a92f

• Downsize OLMo doc test for OLMoForCausalLM.forward to 1B model

  2015aroras

  committed
  43ea80e

  Copy full SHA for 43ea80e

• Try fix incorrect characters in OLMoForCausalLM.forward doct test

  2015aroras

  committed
  fcc457c

  Copy full SHA for fcc457c

• Try fix incorrect characters in OLMoForCausalLM.forward doc test using end quotes

  2015aroras

  committed
  e5fdf50

  Copy full SHA for e5fdf50

Commits on Apr 16, 2024

• Remove pretraining_tp from OLMo config and model

  2015aroras

  committed
  4f062d7

  Copy full SHA for 4f062d7

• Add missing 'Copied from' instances

  2015aroras

  committed
  3c60cf3

  Copy full SHA for 3c60cf3

• Remove unneeded causal_mask from OLMoModel

  2015aroras

  committed
  a037fca

  Copy full SHA for a037fca

• Revert Llama changes

  2015aroras

  committed
  004f80e

  Copy full SHA for 004f80e

• Ignore copy for OLMoForCausalLM.forward

  2015aroras

  committed
  826c91b

  Copy full SHA for 826c91b

• Change 'OLMo' to 'Olmo' in classes

  2015aroras

  committed
  c7f4e74

  Copy full SHA for c7f4e74

• Move minimal OLMo tokenization tests to model tests

  2015aroras

  committed
  ec344c3

  Copy full SHA for ec344c3

• Add missed 'Copied from' for repeat_kv

  2015aroras

  committed
  78148fe

  Copy full SHA for 78148fe