Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Feature request] Add example of finetuning the pretrained models on custom corpus #30

Closed
elyase opened this issue Nov 17, 2018 · 2 comments
Closed

[Feature request] Add example of finetuning the pretrained models on custom corpus #30

elyase opened this issue Nov 17, 2018 · 2 comments

Comments

@elyase
Copy link
Contributor

elyase commented Nov 17, 2018

No description provided.

@thomwolf
Copy link
Member

Hi I don't plan to add that in the near future but feel free to open a PR if you would like to share an additional example.

@davidefiocco
Copy link
Contributor

Necrobumping this for reference, as this is addressed in https://github.com/huggingface/pytorch-pretrained-BERT/blob/master/examples/run_lm_finetuning.py

@maeotaku maeotaku mentioned this issue May 23, 2019
Closed
stevezheng23 added a commit to stevezheng23/transformers that referenced this issue Mar 24, 2020
@stevezheng23
Merge pull request huggingface#30 from stevezheng23/dev/zheng/coqa
161edaf 
Dev/zheng/coqa
SaulLu pushed a commit to SaulLu/transformers that referenced this issue Jan 11, 2022
@NielsRogge
Merge pull request huggingface#30 from SaulLu/modeling_markup_lm_incl…
9624b9f 
…ude_tags_dic_into_tokenizer_config

Modeling markup lm include tags dic into tokenizer config
jameshennessytempus pushed a commit to jameshennessytempus/transformers that referenced this issue Jun 1, 2023
@jamesthesnake
Merge pull request huggingface#30 from jamesthesnake/pop
34b47e0 
Pop
@lwmlyy lwmlyy mentioned this issue Aug 15, 2023
Merged
1 task
younesbelkada pushed a commit that referenced this issue Feb 23, 2024
@RaymondLi0
add embed and residual dropout (#30)
4983a75
younesbelkada added a commit that referenced this issue Feb 28, 2024
@RaymondLi0 @jlamypoirier
@younesbelkada @ArthurZucker
Starcoder2 model - bis (#29215)
63caa37 
* Copy model

* changes

* misc

* fixes

* add embed and residual dropout (#30)

* misc

* remove rms norm and gated MLP

* remove copied mentions where its not a copy anymore

* remove unused _shape

* copied from mistral instead

* fix copies

* fix copies

* add not doctested

* fix

* fix copyright

* Update docs/source/en/model_doc/starcoder2.md

Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>

* Update src/transformers/models/starcoder2/configuration_starcoder2.py

Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>

* Update src/transformers/models/starcoder2/configuration_starcoder2.py

Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>

* fix doc

* revert some changes

* add fa2 tests

* fix styling nit

* fix

* push dummy docs

---------

Co-authored-by: Joel Lamy-Poirier <joel.lamy-poirier@servicenow.com>
Co-authored-by: younesbelkada <younesbelkada@gmail.com>
Co-authored-by: Younes Belkada <49240599+younesbelkada@users.noreply.github.com>
Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>
itazap pushed a commit that referenced this issue May 14, 2024
@RaymondLi0 @jlamypoirier
@younesbelkada @ArthurZucker
Starcoder2 model - bis (#29215)
7ba54b3 
* Copy model

* changes

* misc

* fixes

* add embed and residual dropout (#30)

* misc

* remove rms norm and gated MLP

* remove copied mentions where its not a copy anymore

* remove unused _shape

* copied from mistral instead

* fix copies

* fix copies

* add not doctested

* fix

* fix copyright

* Update docs/source/en/model_doc/starcoder2.md

Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>

* Update src/transformers/models/starcoder2/configuration_starcoder2.py

Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>

* Update src/transformers/models/starcoder2/configuration_starcoder2.py

Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>

* fix doc

* revert some changes

* add fa2 tests

* fix styling nit

* fix

* push dummy docs

---------

Co-authored-by: Joel Lamy-Poirier <joel.lamy-poirier@servicenow.com>
Co-authored-by: younesbelkada <younesbelkada@gmail.com>
Co-authored-by: Younes Belkada <49240599+younesbelkada@users.noreply.github.com>
Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>
@renovate renovate bot mentioned this issue May 17, 2024
Merged
1 task
@renovate-bot renovate-bot mentioned this issue Jun 6, 2024
Merged
1 task
ZYC-ModelCloud pushed a commit to ZYC-ModelCloud/transformers that referenced this issue Nov 14, 2024
@qwopqwop200
Merge pull request huggingface#30 from PanQiWei/add-gpt2
c4f82ac 
Add gpt2
ZYC-ModelCloud pushed a commit to ZYC-ModelCloud/transformers that referenced this issue Nov 14, 2024
@Qubitium
Remove caching for marlin conversion (huggingface#30)
f0d3beb
ZYC-ModelCloud pushed a commit to ZYC-ModelCloud/transformers that referenced this issue Nov 14, 2024
@Qubitium
Reduce torch/triton requirement (huggingface#30)
084eb5b
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@elyase @davidefiocco @thomwolf