Schedule free optimizer support

[muellerzr]

What does this PR do?

Facebook recently released an optimizer that doesn't require an LR scheduler. This PR adds support in by enabling optimizers to call .train() and .eval() as a passthrough.

See https://github.com/facebookresearch/schedule_free/tree/main

Before submitting

• This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
• Did you read the contributor guideline,
  Pull Request section?
• Was this discussed/approved via a Github issue or the forum? Please add a link
  to it if that's the case.
• Did you make sure to update the documentation with your changes? Here are the
  documentation guidelines, and
  here are tips on formatting docstrings.
• Did you write any new necessary tests?

Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@BenjaminBossan

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[muellerzr]

Some general observations off the nlp_example:

Seems to converge faster, here is our baseline vs the new example, read as (accuracy, f1 score):

Basic nlp_example.py:
78.92%, 0.8327
84.8%, 0.8927
85.5%, 89.87

With schedule-free:
79.16%, 0.8622
85.29%, 0.8976
85.29%, 0.8972

[pacman100]

Thank you @muellerzr for adding train and eval methods required for scheduler free optimizers! 🚀

[sdbds]

By the way, since ScheduleFree doesn't require schedule, do we need to create another noSchedule class specifically?

[muellerzr]

Nope, just don't pass a scheduler to Accelerator.prepare() :)

Not a very complicated implementation needed here

[BenjaminBossan]

LGTM, thanks for adding this feature and a script to test it. Btw. any idea what would happen if a user forgets to call optimizer.eval()?

I have a comment suggesting to add a training property too, but no blocker.

[BenjaminBossan]

How about adding a @property for self.optimizer.training too? I don't think we also need a setter for this, as train() and eval() should be enough.

[muellerzr]

That gets to be optimizer-specific, so not a fan of it unless its downstreamed, as they currently don't have that: https://github.com/facebookresearch/schedule_free/blob/main/schedulefree/adamw_schedulefree.py#L86

(Otherwise I'd agree, yes that's a good idea)

[BenjaminBossan]

Ah I see, good point.