Skip to content
This repository has been archived by the owner on Nov 3, 2023. It is now read-only.

LR Scheduler fixes #3025

Merged
merged 8 commits into from
Sep 4, 2020
Merged

LR Scheduler fixes #3025

merged 8 commits into from
Sep 4, 2020

Conversation

stephenroller
Copy link
Contributor

@stephenroller stephenroller commented Aug 29, 2020

Patch description
Loading a model trained with cosine or linear LR schedules was throwing an exception on PyTorch 1.6. This is because pytorch 1.6 broke our method for forwarding an LR schedule with step(num_epochs).

List of changes:

  • Migrates us to the new sanctioned LR loading
  • Works on both pytorch 1.4 and 1.6 (didn't bother testing 1.5.1)
  • Adds a new test_agents/unigram model, which is a torch generator agent that always outputs fixed distributions. This is useful to have a TA that has extremely few parameters (tens or hundreds) and can be trained extremely fast. (Also added its own CI for this)
  • Add end2end tests for select LR schedulers
  • Add resume LR checks for select LR schedulers.

Testing steps

  • Ran select unit tests in both 1.4 and 1.6.
  • New CI

@stephenroller stephenroller changed the base branch from master to pytorch16 August 30, 2020 01:37
@github-actions
Copy link

Your PR contains a change to a task. Please paste the results of the following command into a comment:

python tests/datatests/test_new_tasks.py

@stephenroller stephenroller changed the base branch from pytorch16 to master August 30, 2020 15:43
@stephenroller stephenroller marked this pull request as ready for review August 31, 2020 00:50
Copy link
Contributor

@klshuster klshuster left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks for adding a bunch of tests too

parlai/agents/test_agents/unigram.py Outdated Show resolved Hide resolved
parlai/agents/test_agents/unigram.py Show resolved Hide resolved
parlai/nn/lr_scheduler.py Show resolved Hide resolved
parlai/nn/lr_scheduler.py Show resolved Hide resolved
parlai/nn/lr_scheduler.py Show resolved Hide resolved
@stephenroller stephenroller merged commit 5de0fbc into master Sep 4, 2020
@stephenroller stephenroller deleted the lrfixes branch September 4, 2020 21:13
@stephenroller stephenroller mentioned this pull request Sep 5, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants