Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ak/fix train feeding #5

Open
wants to merge 3 commits into
base: master
Choose a base branch
from

Conversation

AlexeyKruglov
Copy link

  1. Fix index out of range error.
  2. About 8x (for me) training speed-up by feeding whole sample sequence through CuDNN, not char-by-char.
  3. Inference: GPU memory requirement not growing with generated seq length anymore -- by dropping Variable history in inference.

... by feeding the whole sequence to CuDNN RNN, as opposed to character-by-character.
@codeman38
Copy link

codeman38 commented Jan 10, 2018

For what it's worth, using the pre-built pytorch 1.3 on macOS (where CUDA is not available), on an early-2015 MacBook Pro, this patch doesn't seem to improve performance on CPU-- instead, it makes training more than twice as slow. [Edited to add: Tested with CPU on Linux, and got similar results there as well.]

After 100 iterations using the default hyperparameters:
master: ~1.09s/iteration
fix-train-feeding: ~3.71s/iteration

Not sure whether this is an issue with this specific CPU, an inefficiency in PyTorch itself, or something that could be further improved in the char-rnn code, but it's definitely worth pointing out either way.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants