Code

Falcon port#24523

Code

Merged

Rocketknight1 merged 82 commits intomainhuggingface/transformers:mainfrom falcon_porthuggingface/transformers:falcon_portCopy head branch name to clipboardJul 11, 2023

+2,104-5

Commits

Commits on Jul 10, 2023

• Initial commit

  Rocketknight1

  committed
  37f084a

  Copy full SHA for 37f084a

• Update src/transformers/models/falcon/configuration_falcon.py

  Show description for f7008b4
  

  Rocketknight1

  and

  sgugger

  committed
  f7008b4

  Copy full SHA for f7008b4

• Update src/transformers/models/falcon/configuration_falcon.py

  Show description for b3278e0
  

  Rocketknight1

  and

  sgugger

  committed
  b3278e0

  Copy full SHA for b3278e0

• Cleanup config docstring

  Rocketknight1

  committed
  d0c0db5

  Copy full SHA for d0c0db5

• Update src/transformers/models/falcon/configuration_falcon.py

  Show description for 89a2488
  

  Rocketknight1

  and

  sgugger

  committed
  89a2488

  Copy full SHA for 89a2488

• Convert to relative imports

  Rocketknight1

  committed
  3e9f11e

  Copy full SHA for 3e9f11e

• Remove torch < 1.8 warning

  Rocketknight1

  committed
  9633df5

  Copy full SHA for 9633df5

• Restructure cos_sin header

  Rocketknight1

  committed
  a4b91cd

  Copy full SHA for a4b91cd

• qkv -> query, key, value

  Rocketknight1

  committed
  f31d0ea

  Copy full SHA for f31d0ea

• Refactor attention calculation

  Rocketknight1

  committed
  97a7aa7

  Copy full SHA for 97a7aa7

• Add a couple of config variables to account for the different checkpoints

  Rocketknight1

  committed
  31e24c7

  Copy full SHA for 31e24c7

• Successful merging of the code paths!

  Rocketknight1

  committed
  88fca16

  Copy full SHA for 88fca16

• Fix misplaced line in the non-parallel attention path

  Rocketknight1

  committed
  92da7fe

  Copy full SHA for 92da7fe

• Update config and tests

  Rocketknight1

  committed
  e5c67d0

  Copy full SHA for e5c67d0

• Add a pad_token_id when testing

  Rocketknight1

  committed
  d4d91bd

  Copy full SHA for d4d91bd

• Support output_attentions when alibi is None

  Rocketknight1

  committed
  8febeb9

  Copy full SHA for 8febeb9

• make fixup

  Rocketknight1

  committed
  4001051

  Copy full SHA for 4001051

• Skip KV cache shape test

  Rocketknight1

  committed
  8900244

  Copy full SHA for 8900244

• No more _keys_to_ignore_on_load_missing

  Rocketknight1

  committed
  eb1de87

  Copy full SHA for eb1de87

• Simplify self attention a bit

  Rocketknight1

  committed
  3609c58

  Copy full SHA for 3609c58

• Simplify self attention a bit

  Rocketknight1

  committed
  03819d4

  Copy full SHA for 03819d4

• make fixup

  Rocketknight1

  committed
  625baa7

  Copy full SHA for 625baa7

• stash commit

  Rocketknight1

  committed
  2c42786

  Copy full SHA for 2c42786

• Some more attention mask updates

  Rocketknight1

  committed
  57e8ad1

  Copy full SHA for 57e8ad1

• Should pass all tests except assisted generation!

  Rocketknight1

  committed
  5cbec39

  Copy full SHA for 5cbec39

• Add big model generation test

  Rocketknight1

  committed
  92484ab

  Copy full SHA for 92484ab

• make fixup

  Rocketknight1

  committed
  677781a

  Copy full SHA for 677781a

• Add temporary workaround for test

  Rocketknight1

  committed
  80d8d20

  Copy full SHA for 80d8d20

• Test overrides for assisted generation

  Rocketknight1

  committed
  e53bf96

  Copy full SHA for e53bf96

• Update src/transformers/models/falcon/modeling_falcon.py

  Show description for aef8be7
  

  Rocketknight1

  and

  ArthurZucker

  committed
  aef8be7

  Copy full SHA for aef8be7

• Update src/transformers/models/falcon/modeling_falcon.py

  Show description for 025435e
  

  Rocketknight1

  and

  ArthurZucker

  committed
  025435e

  Copy full SHA for 025435e

• Update src/transformers/models/falcon/modeling_falcon.py

  Show description for d06ce2f
  

  Rocketknight1

  and

  ArthurZucker

  committed
  d06ce2f

  Copy full SHA for d06ce2f

• Update tests/models/falcon/test_modeling_falcon.py

  Show description for 3b39607
  

  Rocketknight1

  and

  ArthurZucker

  committed
  3b39607

  Copy full SHA for 3b39607

• Test overrides for assisted generation

  Rocketknight1

  committed
  2f73800

  Copy full SHA for 2f73800

• Add generation demo

  Rocketknight1

  committed
  eb83a30

  Copy full SHA for eb83a30

• Update copyright

  Rocketknight1

  committed
  65207cc

  Copy full SHA for 65207cc

• Make the docstring model actually small

  Rocketknight1

  committed
  0558553

  Copy full SHA for 0558553

• Add module-level docstring

  Rocketknight1

  committed
  b88bd8b

  Copy full SHA for b88bd8b

• Remove all assertions

  Rocketknight1

  committed
  e68c77c

  Copy full SHA for e68c77c

• Add copied from bloom

  Rocketknight1

  committed
  ee9a72f

  Copy full SHA for ee9a72f

• Reformat the QKV layer

  Rocketknight1

  committed
  21763d5

  Copy full SHA for 21763d5

• Add copied from bloom

  Rocketknight1

  committed
  a6ecc91

  Copy full SHA for a6ecc91

• Update src/transformers/models/falcon/modeling_falcon.py

  Show description for 6620500
  

  Rocketknight1

  and

  ArthurZucker

  committed
  6620500

  Copy full SHA for 6620500

• Remove unused line and reformat

  Rocketknight1

  committed
  8123582

  Copy full SHA for 8123582

• No single letter variables

  Rocketknight1

  committed
  96f6c78

  Copy full SHA for 96f6c78

• Cleanup return names

  Rocketknight1

  committed
  56f4443

  Copy full SHA for 56f4443

• Add copied from line

  Rocketknight1

  committed
  f1ff143

  Copy full SHA for f1ff143

• Remove the deprecated arguments blocks

  Rocketknight1

  committed
  339d465

  Copy full SHA for 339d465

• Change the embeddings test to an alibi on/off test

  Rocketknight1

  committed
  67e530f

  Copy full SHA for 67e530f

• Remove position_ids from FalconForQA

  Rocketknight1

  committed
  ae3cfc6

  Copy full SHA for ae3cfc6

• Remove old check for token type IDs

  Rocketknight1

  committed
  7623f3a

  Copy full SHA for 7623f3a

• Fix the alibi path when multi_query is False

  Rocketknight1

  committed
  b2facf7

  Copy full SHA for b2facf7

• Update src/transformers/models/falcon/modeling_falcon.py

  Show description for 1c6353b
  

  Rocketknight1

  and

  amyeroberts

  committed
  1c6353b

  Copy full SHA for 1c6353b

• Update src/transformers/models/falcon/modeling_falcon.py

  Show description for b9a2e26
  

  Rocketknight1

  and

  amyeroberts

  committed
  b9a2e26

  Copy full SHA for b9a2e26

• Update tests/models/falcon/test_modeling_falcon.py

  Show description for 5863002
  

  Rocketknight1

  and

  amyeroberts

  committed
  5863002

  Copy full SHA for 5863002

• Update config naming

  Rocketknight1

  committed
  456b523

  Copy full SHA for 456b523

• Fix typo for new_decoder_architecture

  Rocketknight1

  committed
  7a035ae

  Copy full SHA for 7a035ae

• Add some comments

  Rocketknight1

  committed
  0e661eb

  Copy full SHA for 0e661eb

• Fix docstring

  Rocketknight1

  committed
  f5709ad

  Copy full SHA for f5709ad

• Fix docstring

  Rocketknight1

  committed
  a9e0d0e

  Copy full SHA for a9e0d0e

• Create range in the right dtype from the start

  Rocketknight1

  committed
  3e0b978

  Copy full SHA for 3e0b978

• Review comment cleanup

  Rocketknight1

  committed
  58dda7e

  Copy full SHA for 58dda7e

• n_head_kv -> num_kv_heads

  Rocketknight1

  committed
  17c4b9d

  Copy full SHA for 17c4b9d

• self.alibi -> self.use_alibi

  Rocketknight1

  committed
  57fe9f2

  Copy full SHA for 57fe9f2

• self.num_kv -> self.num_kv_heads

  Rocketknight1

  committed
  302cc95

  Copy full SHA for 302cc95

• Reorder config args

  Rocketknight1

  committed
  b744ccd

  Copy full SHA for b744ccd

• Made alibi arguments Optional

  Rocketknight1

  committed
  e2f2db1

  Copy full SHA for e2f2db1

• Add all model docstrings

  Rocketknight1

  committed
  3b94831

  Copy full SHA for 3b94831

• Add extra checkpoints

  Rocketknight1

  committed
  ab66238

  Copy full SHA for ab66238

• Add author info for Falcon

  Rocketknight1

  committed
  94c33e9

  Copy full SHA for 94c33e9

• Stop removing token_type_ids because our checkpoints shouldn't return it anymore

  Rocketknight1

  committed
  131be5f

  Copy full SHA for 131be5f

• Add one hopeful comment for the future

  Rocketknight1

  committed
  e11ff0c

  Copy full SHA for e11ff0c

• Fix typo

  Rocketknight1

  committed
  0de24ed

  Copy full SHA for 0de24ed

• Update tests, fix cache issue for generation

  Rocketknight1

  committed
  2bdc92d

  Copy full SHA for 2bdc92d

• Use -1e9 instead of -inf to avoid float overflow

  Rocketknight1

  committed
  e711719

  Copy full SHA for e711719

• Recompute the rotary embeddings much less often

  Rocketknight1

  committed
  d7176e2

  Copy full SHA for d7176e2

• Re-enable disabled tests

  Rocketknight1

  committed
  21f84bd

  Copy full SHA for 21f84bd

• One final fix to attention mask calculation, and update tests

  Rocketknight1

  committed
  3359388

  Copy full SHA for 3359388

• Cleanup targeting falcon-40b equivalency

  Rocketknight1

  committed
  fb4e555

  Copy full SHA for fb4e555

• Post-rebase docs update

  Rocketknight1

  committed
  28d55b9

  Copy full SHA for 28d55b9

• Update docstrings, especially in the config

  Rocketknight1

  committed
  c1a3559

  Copy full SHA for c1a3559

• More descriptive variable names, and comments where we can't rename them

  Rocketknight1

  committed
  5125fb9

  Copy full SHA for 5125fb9