New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Average reward of gpt-3.5-turbo #8

Open

George-Chia opened this issue Jun 6, 2024 · 0 comments

George-Chia commented Jun 6, 2024

Hello! When I run the following command:

"python -m eval_agent.main --agent_config openai --exp_config alfworld --split test --verbose"

I got the results:
All tasks done.
Output saved to outputs/gpt-3.5-turbo/alfworld
Average reward: 0.0299
Success rate: 0.0299.

, which is much lower than the results in Table 2 of the paper. Is there any config that needs to be set?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment