[Pending Release] Updating to Mephisto 0.4.0

[JackUrb]

Patch description
Updates all of the scripts using Mephisto to be compliant with the Hydra 1.1 version that Mephisto 0.4.0 uses. Also updates the crowdsourcing test script.

NOTE
Mephisto v0.4.0 is not yet released, this branch is setting us up for after a few in-flight PRs land to cleanly upgrade everyone at once. v0.4.0 will also be the first version on PyPI, if that is helpful. Happy to make changes on how requirements are pulled given that information.

Testing steps
TBD - I launched a few of the scripts and found they didn't throw the @Package directive missing error that many were seeing before.

[jxmsML]

I checked out the Mephisto mturk-hit-id-sync branch, but when upgrading hydra to hydra core 1.1.1, I ran into this:

ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
parlai 1.3.0 requires hydra-core~=1.0.6, but you have hydra-core 1.1.1 which is incompatible.
parlai 1.3.0 requires omegaconf<2.1.0, but you have omegaconf 2.1.1 which is incompatible.

[EricMichaelSmith]

Thanks for making this change! One comment: I'd also change the README at https://github.com/facebookresearch/ParlAI/blob/master/parlai/crowdsourcing/README.md to reflect that now the YAML files are in hydra_configs/conf/. In particular, can you still do conf=example on the command line to specify the hydra_configs/conf/example.yaml file, or do you have to pass in a --config-dir arg?

[JackUrb]

Yes the behavior remains the same, hydra just uses the provided directory as the root. Can edit the README. Basically the new hydra requirements is that we always specify a config-dir.

[EricMichaelSmith]

Thanks for the change :) The conf/ mention on line 9 should also be changed to hydra_configs/conf/, right?