Make clearer about zero_init requirements

[muellerzr]

What does this PR do?

@abhishekkrthakur pointed out that he was facing some confusing issues on when trying to use zero-init + model initialization, and he was initializing the model before TrainingArguments were made, so zero-init couldn't be setup.

There's no real way to catch this, so instead it flags beforehand that we need to instantiate the TrainingArguments beforehand in the docstring. The exception of this is if a user used a configured accelerate launch, which this PR also checks for/has logic for

Before submitting

• This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
• Did you read the contributor guideline,
  Pull Request section?
• Was this discussed/approved via a Github issue or the forum? Please add a link
  to it if that's the case.
• Did you make sure to update the documentation with your changes? Here are the
  documentation guidelines, and
  here are tips on formatting docstrings.
• Did you write any new necessary tests?

Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@pacman100 @amyeroberts

[abhishekkrthakur]

Thank you for adding this. It will hopefully save a lot of people a lot of time :)

[muellerzr]

@pacman100 the other option I'm considering:

Could we realistically check for this from accelerate launch via environmental variables set for zero init? This way the catch works for people not using accelerate launch, but we can do it ourselves if not

(Whether we raise an explicit error, or doing it behind the scenes I'm iffy on, I'd rather raise an err)

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[amyeroberts]

Thanks for adding! Having these error messages will definitely be useful!

Just a comment on the conditional tuple return in is_deepspeed_zero3_enabled

[muellerzr]

Thanks @amyeroberts, I've tweaked it so now it lives under its own new func, and we call both in order since it's only needed in two places

[pacman100]

Thank you @muellerzr for the fixes but I have left some overall comments.

[muellerzr]

@amyeroberts @pacman100 should be good for a last review. Post discussions offline it's best to just do an RTFM, since we can't reliably check beforehand and currently rely on having numerous examples instead

[amyeroberts]

Thanks!