Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Issue on the reproducing results on Table 1 #6

Open
bbuing9 opened this issue Mar 6, 2021 · 1 comment
Open

Issue on the reproducing results on Table 1 #6

bbuing9 opened this issue Mar 6, 2021 · 1 comment

Comments

@bbuing9
Copy link

bbuing9 commented Mar 6, 2021

Congratulations on this interesting work!

Now, I am trying to reproduce the results on your papers including cutoff and back-translation which also shows a great improvement.
But, now I'm struggling due to the absence of some details.

For example, I can reproduce the results for CoLA task after a huge amount of trials and error for finding proper hyper-parameter (cutoff_ratio, ce_loss, js_loss), however, can't reproduce the results for other tasks on GLUE.

It might be due to that I have not found proper hyper-parameters yet.. So, if you let me know or give the detailed hyper-parameters for each GLUE task, then it will be very helpful..!
Can you let me know some details (e.g., used value of alpha,beta for each task) for reproducing the results?

Also, in the case of back-translation, now I'm trying to reproduce it with WMT en-de, de-en 19 in fairseq. But, I can't improve the naive baseline. For example, my reproducing results is almost 88, but your reporting is 91.7 in RTE with RoBERTa large
So, can you let me know the details of back-translation and related hyper-parameters?

Thanks for your help and look forward to hearing from you!

@Jiaxin-Wen
Copy link

same question, any updates on this? @dinghanshen

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants