Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] DQN training not resulting in good results on task simulator #156

Open
kh522 opened this issue Oct 30, 2020 · 2 comments
Open

[BUG] DQN training not resulting in good results on task simulator #156

kh522 opened this issue Oct 30, 2020 · 2 comments
Labels
bug Something isn't working

Comments

@kh522
Copy link

kh522 commented Oct 30, 2020

Describe the bug
I don't seem to be able to get the dqn code to be working properly. Training DQN with default configurations doesn't seem to result in good performance when evaluating on the task simulator. I seem to only be able to achieve a 1% task success rate when evaluating on the simulator.

To Reproduce
Steps to reproduce the behavior:

  1. Run train.py file in dqn folder
  2. Run evaluation.py

Expected behavior
Is there an expected task success rate for DQN?

@kh522 kh522 added the bug Something isn't working label Oct 30, 2020
@zqwerty
Copy link
Member

zqwerty commented Nov 2, 2020

Currently, we got a similar result. Still working on this.

@ghost
Copy link

ghost commented Nov 21, 2020

I got exactly the same issue.
In addition, I can't download the pretrained model by the url in dqn.py, that is, https://convlab.blob.core.windows.net/convlab-2/dqn_policy_multiwoz.zip

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants