Reproduction of the success rate on 'valid_seen' dataset

[jobin2725]

Hi, thanks for your outstanding research!

I was going through the codes you uploaded but struggled to reproduce the reported success rate on the 'valid_seen' dataset.
Although it should show 47.8% success rate (which is reported in the paper), it only shows about 24.77% success rate.
INFO:root:266m 48s (- -63m 31s) (1311 130%) reward -0.0891, SR 0.2477, pws 0.1536

I guess I have properly downloaded the checkpoints of the et_checkpoint for the obj_predictor(maskrcnn_model.pth), questioner(questioner_anytime_finetuned.pt), and performer(latest.pth). When generating the lmdb datset only for the valid_seen dataset, there were no errors observed. (I haven't done the fine-tuning of the performer nor questioner yet.)

Could you suggest some methods for me to obtain the desired success results?