Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add truncated backpropagation through time (TBPTT) example #19985

Open
svnv-svsv-jm opened this issue Jun 17, 2024 · 2 comments · May be fixed by #20422
Open

Add truncated backpropagation through time (TBPTT) example #19985

svnv-svsv-jm opened this issue Jun 17, 2024 · 2 comments · May be fixed by #20422
Labels
docs Documentation related help wanted Open to be worked on

Comments

@svnv-svsv-jm
Copy link

svnv-svsv-jm commented Jun 17, 2024 •
edited by github-actions bot
Loading

📚 Documentation

It seems like there is not example anymore about how to deal with Truncated Backpropagation Through Time in the latest versions. Am I wrong?

cc @Borda
@svnv-svsv-jm svnv-svsv-jm added docs Documentation related needs triage Waiting to be triaged by maintainers labels Jun 17, 2024
@awaelchli
Copy link
Contributor

Special support for TBPTT was removed in Lightning 2.0. The release notes here provide an example how you could implement it normally, see the section Truncated backpropagation through time (TBPTT).

Adding a TBTT example to the docs would be possible too. Contributions towards this would be welcomed.

@awaelchli awaelchli added help wanted Open to be worked on and removed needs triage Waiting to be triaged by maintainers labels Jun 22, 2024
@awaelchli awaelchli changed the title TBTT missing example Add truncated backpropagation through time (TBPTT) example Jun 22, 2024
@svnv-svsv-jm
Copy link
Author

Yes, that would be good.

However, removing this means that I still have to load a huge time-window in memory, then loop over it in the training_step, which kind of defeats the purpose of TBPTT.

One would have to pass DataLoader's to the training_step in the batch argument for it to be less memory-intensive.

@chualanagit chualanagit linked a pull request Nov 15, 2024 that will close this issue
Open
12 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
docs Documentation related help wanted Open to be worked on
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Add doc for TBPTT chualanagit/pytorch-lightning
2 participants
@awaelchli @svnv-svsv-jm