Some question

[lolsd]

When I ran your code on Github, I found some bugs:

you used "loss, sample_size, logging_output = super().valid_step(sample, model, criterion)" in fairseq/tasks/translation.py. The main code in "super().valid_step()" is "loss, sample_size, logging_output = criterion(model, sample)" and "criterion" is Class LabelSmoothedCrossEntropyCriterion. In Class LabelSmoothedCrossEntropyCriterion, you called function "model.mix()" which need parameter "pair_sample", but you provided the parameter "sample" in the valid_step.

When I fixed this bug, although I got 28.1 on WMT14EN-DE, I got 35.1 on iWSLt14DE-en instead of 36.2.

[Zhaoyi-Li21]

hello, have you noticed that the description on sampling policy (for iwlst-exps) "we sample (X' , Y') uniformly from all examples with similar target Y' lengths (|Y |−|Y'| ≤ 5)."? (tips: this line is from supplement of this paper, C.1-paragraph)
maybe this is a factor accounting for why your result is a little bit lower than that reported.