Code

Add qwen2#28436

Code

Merged

ArthurZucker merged 76 commits intohuggingface:mainhuggingface/transformers:mainfrom JustinLin610:add_qwen2JustinLin610/transformers:add_qwen2Copy head branch name to clipboardJan 17, 2024

+3,136-2

Commits

Commits on Jan 10, 2024

• add config, modeling, and tokenization

  JustinLin610

  committed
  f5bf018

• add auto and init

  JustinLin610

  committed
  3226f3a

• update readme

  JustinLin610

  committed
  52ab139

• update readme

  JustinLin610

  committed
  0fe20cd

• update team name

  JustinLin610

  committed
  8299266

• fixup

  JustinLin610

  committed
  b8a8e22

• fixup

  JustinLin610

  committed
  55802f3

• update config

  JustinLin610

  committed
  6b587b8

• update code style

  JustinLin610

  committed
  dfe7d2e

• update for fixup

  JustinLin610

  committed
  525de40

• update for fixup

  JustinLin610

  committed
  7f6f5e2

• update for fixup

  JustinLin610

  committed
  36fcf65

• update for testing

  JustinLin610

  committed
  42f561a

Commits on Jan 11, 2024

• update for testing

  JustinLin610

  committed
  105d98f

• fix bug for config and tokenization

  JustinLin610

  committed
  47e6e9e

• fix bug for bos token

  JustinLin610

  committed
  5061a26

• not doctest

  JustinLin610

  committed
  0987dab

• debug tokenizer

  JustinLin610

  committed
  bf4e928

• not doctest

  JustinLin610

  committed
  2658baf

• debug tokenization

  JustinLin610

  committed
  69b03e7

• debug init for tokenizer

  JustinLin610

  committed
  725ab32

• fix style

  JustinLin610

  committed
  57264f3

• update init

  JustinLin610

  committed
  0489d0f

• delete if in token auto

  JustinLin610

  committed
  e3b60a6

• add tokenizer doc

  JustinLin610

  committed
  e1b4a8c

• add tokenizer in init

  JustinLin610

  committed
  22d3402

• Update dummy_tokenizers_objects.py

  jklj077

  authored
  c05a7be

• Merge pull request #1 from jklj077/patch-1

  Show description for 783eec8
  JustinLin610

  authored
  783eec8

• update

  JustinLin610

  committed
  c24ee57

• update

  JustinLin610

  committed
  0f83bc1

• debug

  JustinLin610

  committed
  e68a65c

• Merge branch 'huggingface:main' into add_qwen2

  JustinLin610

  authored
  03ff228

• Update tokenization_qwen2.py

  jklj077

  authored
  2e00431

• Merge pull request #3 from jklj077/patch-1

  Show description for 73695ff
  JustinLin610

  authored
  73695ff

• debug

  JustinLin610

  committed
  e041c79

Commits on Jan 12, 2024

• Merge branch 'huggingface:main' into add_qwen2

  JustinLin610

  authored
  e96c5c2

Commits on Jan 13, 2024

• Update convert_slow_tokenizer.py

  jklj077

  authored
  9cda260

• add copies

  

  renxuancheng.rxc

  committed
  a81deed

• Merge pull request #4 from jklj077/patch-3

  Show description for e652aa9
  JustinLin610

  authored
  e652aa9

• add copied from and make style

  JustinLin610

  committed
  985fe0f

• update files map

  

  renxuancheng.rxc

  committed
  f7beb77

• update test

  

  renxuancheng.rxc

  committed
  35a7480

• Merge pull request #5 from jklj077/patch-4

  Show description for 26a24db
  JustinLin610

  authored
  26a24db

• fix style

  JustinLin610

  committed
  f419098

• fix merge reading and update tests

  

  renxuancheng.rxc

  committed
  6b3247b

• Merge pull request #6 from jklj077/patch-6

  Show description for 68968f5
  JustinLin610

  authored
  68968f5

• fix tests

  

  renxuancheng.rxc

  committed
  9c45e77

• fix tests

  

  renxuancheng.rxc

  committed
  93bae64

• Merge pull request #7 from jklj077/patch-6

  Show description for 1345aae
  JustinLin610

  authored
  1345aae

• fix style

  JustinLin610

  committed
  bd57c63

Commits on Jan 14, 2024

• debug a variable in readme

  JustinLin610

  committed
  25d43f0

Commits on Jan 15, 2024

• Update src/transformers/models/qwen2/configuration_qwen2.py

  Show description for e68a1b8
  

  JustinLin610

  and

  ArthurZucker

  authored
  e68a1b8

• update test and copied from

  JustinLin610

  committed
  4135193

• fix style

  JustinLin610

  committed
  8708e50

• update qwen2 tokenization and tests

  

  renxuancheng.rxc

  committed
  db82439

• Update tokenization_qwen2.py

  JustinLin610

  authored
  e0c7750

• Merge pull request #8 from jklj077/patch-7

  Show description for 7c13f74
  JustinLin610

  authored
  7c13f74

• delete the copied from after property

  JustinLin610

  committed
  8ebacad

• fix style

  JustinLin610

  committed
  7b670a8

• update tests

  

  renxuancheng.rxc

  committed
  304f0dc

• Merge branch 'patch-7' of https://github.com/jklj077/transformers into patch-7

  

  renxuancheng.rxc

  committed
  8b144e1

• Merge pull request #9 from jklj077/patch-7

  Show description for ca4a8c5
  JustinLin610

  authored
  ca4a8c5

Commits on Jan 16, 2024

• update tests

  

  renxuancheng.rxc

  committed
  1ce4e4d

• Merge pull request #10 from jklj077/patch-7

  Show description for 04c4a75
  JustinLin610

  authored
  04c4a75

• Merge branch 'huggingface:main' into add_qwen2

  JustinLin610

  authored
  93cbb5d

• add copied from

  JustinLin610

  committed
  1e0b937

• fix bugs

  JustinLin610

  committed
  22f7b1e

• update doc

  JustinLin610

  committed
  69d3f89

Commits on Jan 17, 2024

• add warning for sliding window attention

  

  renxuancheng.rxc

  committed
  2840f0a

• update qwen2 tokenization

  

  renxuancheng.rxc

  committed
  87f6cf7

• Merge pull request #11 from jklj077/patch-8

  Show description for 0a47113
  JustinLin610

  authored
  0a47113

• fix style

  JustinLin610

  committed
  047e372

• Merge branch 'huggingface:main' into add_qwen2

  JustinLin610

  authored
  6b6dae0

• Update src/transformers/models/qwen2/modeling_qwen2.py

  Show description for a4e80ee
  

  JustinLin610

  and

  ArthurZucker

  authored
  a4e80ee

• Merge branch 'huggingface:main' into add_qwen2

  JustinLin610

  authored
  39cc60b

• fix tokenizer fast

  JustinLin610

  committed
  a8b0c3c