Tiny bug with padding

[kkteru]

Hey @wzhouad,

This is probably a harmless bug in the current code base but just wanted to point this out FYI.

The padding token is hard-coded to 0 in this line. However, for many models padding token id is non-zero (for roberta based models it's 1 and 0 actually corresponds to BOS token). As long as one sends the right attention mask (as you do), this should be harmless. Otherwise, this is a silent bug that might go unnoticed.

Cheers,
Komal