Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reproduce the result of LSTM on PersonaChat #10

Open
StevenTang1998 opened this issue Aug 8, 2021 · 1 comment
Open

Reproduce the result of LSTM on PersonaChat #10

StevenTang1998 opened this issue Aug 8, 2021 · 1 comment

Comments

@StevenTang1998
Copy link

StevenTang1998 commented Aug 8, 2021

Hi, I followed the instruction from this page.
I tested the result of the LSTM model on easy PersonaChat dataset.
I used the original script to preprocess the dataset combined with the vocab.txt in this page (because I can't find it in GLGE page).

./preprocessed.sh personachat easy

And then I used the original scipt to train and evaluate the lstm model without modifying anything.

./run.sh personachat easy lstm test

However, there is a big gap between the results of mine and the ones in the paper.
The results of paper are 42.2/35.9/0.2/0.7.
The results of mine are 31.75/19.63/0.3/1.2.

So, I want to know how to get the results in the paper or where I am wrong.

@araloak
Copy link

araloak commented Sep 8, 2021

Hi, are you aware how to get the test.tgt file?
I downloaded the GLGE dataset on this page, but the test.tgt file is not included.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants