Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Is there a plan to have a FP16 for GPU so to have larger batch size or longer text documents support ? #10

Closed
howardhsu opened this issue Nov 9, 2018 · 4 comments
Closed

Is there a plan to have a FP16 for GPU so to have larger batch size or longer text documents support ? #10

howardhsu opened this issue Nov 9, 2018 · 4 comments

Comments

@howardhsu
Copy link
Contributor

Is there a plan to have an FP16 for GPU so to have a larger batch size or longer text documents support?
@thomwolf
Copy link
Member

Yes probably. I am testing fp16 right now. If it works well I will push it to the repo.

@thomwolf
Copy link
Member

Ok I've added FP16 support (see updated readme)

@howardhsu
Copy link
Contributor Author

Thanks for this quick updates.

@Ashish-Gupta03
Copy link

I'm not able to work with FP16 for pytorch BERT code. Particularly for BertForSequenceClassification, which I tried and got the issue
Runtime error: Expected scalar type object Half but got scalar type Float for argument #2 target
when I enabled fp16.
Also when using
logits = logits.half() labels = labels.half()
then the epoch time also increased.

@maeotaku maeotaku mentioned this issue May 23, 2019
Closed
@HongyanJiao HongyanJiao mentioned this issue Sep 19, 2019
Closed
stevezheng23 added a commit to stevezheng23/transformers that referenced this issue Mar 24, 2020
@stevezheng23
Merge pull request huggingface#10 from stevezheng23/dev/zheng/quac
b8d0479 
fix issues in new quac-kd runner (cont.)
LysandreJik added a commit that referenced this issue Apr 10, 2020
@jysohn23 @LysandreJik
Add run_glue_tpu.py that trains models on TPUs (#3702)
551b450 
* Initial commit to get BERT + run_glue.py on TPU

* Add README section for TPU and address comments.

* Cleanup TPU bits from run_glue.py (#3)

TPU runner is currently implemented in:
https://github.com/pytorch-tpu/transformers/blob/tpu/examples/run_glue_tpu.py.

We plan to upstream this directly into `huggingface/transformers`
(either `master` or `tpu`) branch once it's been more thoroughly tested.

* Cleanup TPU bits from run_glue.py

TPU runner is currently implemented in:
https://github.com/pytorch-tpu/transformers/blob/tpu/examples/run_glue_tpu.py.

We plan to upstream this directly into `huggingface/transformers`
(either `master` or `tpu`) branch once it's been more thoroughly tested.

* No need to call `xm.mark_step()` explicitly (#4)

Since for gradient accumulation we're accumulating on batches from
`ParallelLoader` instance which on next() marks the step itself.

* Resolve R/W conflicts from multiprocessing (#5)

* Add XLNet in list of models for `run_glue_tpu.py` (#6)

* Add RoBERTa to list of models in TPU GLUE (#7)

* Add RoBERTa and DistilBert to list of models in TPU GLUE (#8)

* Use barriers to reduce duplicate work/resources (#9)

* Shard eval dataset and aggregate eval metrics (#10)

* Shard eval dataset and aggregate eval metrics

Also, instead of calling `eval_loss.item()` every time do summation with
tensors on device.

* Change defaultdict to float

* Reduce the pred, label tensors instead of metrics

As brought up during review some metrics like f1 cannot be aggregated
via averaging. GLUE task metrics depends largely on the dataset, so
instead we sync the prediction and label tensors so that the metrics can
be computed accurately on those instead.

* Only use tb_writer from master (#11)

* Apply huggingface black code formatting

* Style

* Remove `--do_lower_case` as example uses cased

* Add option to specify tensorboard logdir

This is needed for our testing framework which checks regressions
against key metrics writtern by the summary writer.

* Using configuration for `xla_device`

* Prefix TPU specific comments.

* num_cores clarification and namespace eval metrics

* Cache features file under `args.cache_dir`

Instead of under `args.data_dir`. This is needed as our test infra uses
data_dir with a read-only filesystem.

* Rename `run_glue_tpu` to `run_tpu_glue`

Co-authored-by: LysandreJik <lysandre.debut@reseau.eseo.fr>
@manchandasahil manchandasahil mentioned this issue Mar 22, 2021
Closed
2 tasks
amathews-amd referenced this issue in ROCm/transformers Aug 6, 2021
@raviskolli
Merge pull request #10 from microsoft/raviskolli/ort
ae1411f 
Update for ORTModule package
rraminen pushed a commit to rraminen/transformers that referenced this issue Jun 3, 2022
@stevenireeves
Merge pull request huggingface#10 from ROCmSoftwarePlatform/bert-tf2
8112f21 
 Adding evaluation mode to script.
jlamypoirier added a commit to jlamypoirier/transformers that referenced this issue Apr 4, 2023
@jlamypoirier
Profiling and misc (huggingface#10)
4be387d
sim-so added a commit to sim-so/transformers that referenced this issue Apr 23, 2023
@sim-so
# This is a combination of 11 commits.
910ba87 
# This is the 1st commit message:

Update docs/source/ko/tasks/summarization.mdx

Co-authored-by: Wonhyeong Seo <wonhseo@kakao.com>
# This is the commit message huggingface#2:

Update docs/source/ko/tasks/summarization.mdx

Co-authored-by: Wonhyeong Seo <wonhseo@kakao.com>
# This is the commit message huggingface#3:

Update docs/source/ko/tasks/summarization.mdx

Co-authored-by: Wonhyeong Seo <wonhseo@kakao.com>
# This is the commit message huggingface#4:

Update docs/source/ko/tasks/summarization.mdx

Co-authored-by: Wonhyeong Seo <wonhseo@kakao.com>
# This is the commit message huggingface#5:

Update docs/source/ko/tasks/summarization.mdx

Co-authored-by: Wonhyeong Seo <wonhseo@kakao.com>
# This is the commit message huggingface#6:

Update docs/source/ko/tasks/summarization.mdx

Co-authored-by: Wonhyeong Seo <wonhseo@kakao.com>
# This is the commit message huggingface#7:

Update docs/source/ko/tasks/summarization.mdx

Co-authored-by: Wonhyeong Seo <wonhseo@kakao.com>
# This is the commit message huggingface#8:

Update docs/source/ko/tasks/summarization.mdx

Co-authored-by: Wonhyeong Seo <wonhseo@kakao.com>
# This is the commit message huggingface#9:

Update docs/source/ko/tasks/summarization.mdx

Co-authored-by: Wonhyeong Seo <wonhseo@kakao.com>
# This is the commit message huggingface#10:

Update docs/source/ko/tasks/summarization.mdx

Co-authored-by: Wonhyeong Seo <wonhseo@kakao.com>
# This is the commit message huggingface#11:

Update docs/source/ko/tasks/summarization.mdx
jameshennessytempus pushed a commit to jameshennessytempus/transformers that referenced this issue Jun 1, 2023
@jamesthesnake
Merge pull request huggingface#10 from huggingface/main
9272a10 
si
@lwmlyy lwmlyy mentioned this issue Aug 15, 2023
Merged
1 task
ocavue pushed a commit to ocavue/transformers that referenced this issue Sep 13, 2023
@xenova
Fix unknown tokens (huggingface#10)
e98915d
lcong pushed a commit to lcong/transformers that referenced this issue Apr 9, 2024
@patrickloeber
Merge pull request huggingface#10 from guidogallopyn/patch-1
cc18036 
Update 05_1_gradientdescent_manually.py
SangbumChoi added a commit to SangbumChoi/transformers that referenced this issue Aug 22, 2024
@SangbumChoi
Merge pull request huggingface#10 from Superb-AI-Suite/main
d021087 
merge
ArthurZucker pushed a commit that referenced this issue Sep 25, 2024
@qubvel
Merge pull request #10 from huggingface/refactor-configuration
0350540 
Refactor configuration
ZYC-ModelCloud pushed a commit to ZYC-ModelCloud/transformers that referenced this issue Nov 14, 2024
@PanQiWei
Merge pull request huggingface#10 from eltociear/patch-1
a19f58a 
Update README.md
ZYC-ModelCloud pushed a commit to ZYC-ModelCloud/transformers that referenced this issue Nov 14, 2024
@Qubitium @LRL-ModelCloud
ADD GLM model support (huggingface#10)
f219c22 
Co-authored-by: LRL <lrl@lbx.dev>
ZYC-ModelCloud pushed a commit to ZYC-ModelCloud/transformers that referenced this issue Nov 14, 2024
@Qubitium
update (huggingface#10)
0047f22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@thomwolf @Ashish-Gupta03 @howardhsu