Skip to content
This repository has been archived by the owner on Nov 3, 2023. It is now read-only.

[Acute-Eval] Mephisto 1.0 compatibility #4551

Merged
merged 1 commit into from
May 18, 2022
Merged

[Acute-Eval] Mephisto 1.0 compatibility #4551

merged 1 commit into from
May 18, 2022

Conversation

EricMichaelSmith
Copy link
Contributor

@EricMichaelSmith EricMichaelSmith commented May 17, 2022

Patch description

  • Upgrade Acute-Eval frontend code to make it compatible with the style of output passing allowed by Mephisto 1.0
  • Update package-lock file accordingly to work with Mephisto 1.0

Testing steps

python parlai/crowdsourcing/tasks/acute_eval/run.py
python parlai/crowdsourcing/tasks/acute_eval/fast_eval.py

@JackUrb
Copy link
Contributor

JackUrb commented May 17, 2022

Were there any required changes for backend review/process scripts?

@EricMichaelSmith
Copy link
Contributor Author

Were there any required changes for backend review/process scripts?

@JackUrb Surprisingly not - this PR wraps the output in a 'final_data' key of a dict, as is required in acute_eval_agent_state.py and acute_eval_runner.py, and presumably Mephisto pre-1.0 did this automatically?

@JackUrb
Copy link
Contributor

JackUrb commented May 17, 2022

Ah okay that makes sense - the old solution we had for wrapping was hacky and inconsistent on the packaging + parsing side, so we did away with it. Good to know that keeping that data model for upgrades is as simple as doing the above.

@EricMichaelSmith EricMichaelSmith merged commit 989e29f into main May 18, 2022
@EricMichaelSmith EricMichaelSmith deleted the acute-fix branch May 18, 2022 13:30
kushalarora pushed a commit that referenced this pull request Jun 15, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants