Reproducing results

[jessganoww]

I'm hoping to reproduce the FewRel results you shared in your paper. It seems that I'm getting false for all data points in the pred_out csv if I run on only a single random seed. Can you confirm whether this behaviour is expected?

Will you also be releasing the scripts or the rd_fold values to reproduce the results from Table 3?

Thank you!

[NikitaGautam]

I am having a similar problem with reproducing the results for the Triple extraction task. I used all the folds for both the fewrel and wikizsl data but the TE results (with the hyperparameters mentioned in the papers/ with different hyperparameters) cannot be reproduced.

[bosung]

Hi, thanks for reporting this. There is definitely a bug in the main branch -- I found it was not updated when we incorporate codes for the final upload. I fixed it and updated to bugfix branch. This is what I've got after running the fixed version on FewRel and n_unseen_rel=10.

FewRel, n_unseen_rel=10

fold_0: 0.34732314993326413
fold_1: 0.26671597633136096
fold_2: 0.24289855072463767
fold_3: 0.3052043516612761
fold_4: 0.2872695601683844

Thanks for your interests in ZETT!