Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve training speed by allow padding to batch-wise max length instead of model supported max length #737

Open
zodiacg opened this issue May 25, 2023 · 0 comments
Labels
enhancement New feature or request

Comments

@zodiacg
Copy link

zodiacg commented May 25, 2023

Is your feature request related to a problem? Please describe.
I'm running A2T augmented training with codes from QData/TextAttack-A2T. As mentioned in QData/Textattack-A2T Issue#1, the training speed of the PLM model is extremely slow.

After some checking I notices in textattack.Trainer.training_step(), the input text is padded to the max length the model supported, instead of the max length in the batch. This causes much more computation overhead, and probably unnecessary.

I wrote an inherited Trainer class to override the training_step method which simply changed the padding parameter of tokenizer to True and the training speed is much faster.

Describe the solution you'd like
Improve the training speed of transformers models when using textattack.Trainer.
Previous test shows that on a 3090 card, when training SNLI task a clean epoch took 3 hours.

Describe alternatives you've considered
By allow the tokenizer to only pad the input text to the max length in the batch instead of the max length supported by the corresponding model. By doing so a clean epoch took 20 minutes on the same condition.

Additional context

  • Is there any specific necessary to pad the input text to the max length supported by model?
  • Similar consideration may also be applied to HuggingFaceModelWrapper
@jxmorris12 jxmorris12 added the enhancement New feature or request label Jul 11, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants