Duplicate paragraph in HotpotQA training set

[boyang-nlp]

I noticed that some duplicate paragraphs exist in HotpotQA training set as shown in the following pictures, is it reasonable? Thanks!

[boyang-nlp]

@danqi @robinjia @alontalmor @ajfisch could you please help to look into this issue? thanks.

[ajfisch]

Hi @scissorsy! Could you give the qid for those examples? Thanks!

[boyang-nlp]

Hi, @ajfisch ! At first glance, there're many such cases, for example:

[alontalmor]

Found the bug and fixed it, HotpotQA data update soon to follow

[ajfisch]

Updated. Thanks for reporting!

[boyang-nlp]

Got it,thanks a lot!

[csarron]

Hi @ajfisch and @alontalmor, this doesn't seem fixed, I can still see the duplicated sentences in the dev split. Just manually check the first 10 examples, I found 3 of them still contain duplicates.

e.g., the first example (id=5a8c7595554299585d9e36b6)

and the fourth example, (id=5a85ea095542994775f606a8):

10th example, (id=5a7166395542994082a3e814)

the problem seems still exist, or it didn't get fully fixed.

Note: I downloaded the dev split fresh from https://s3.us-east-2.amazonaws.com/mrqa/release/v2/dev/HotpotQA.jsonl.gz.

Is it possible that the problem got fixed, but somehow the s3 cache was not purged?

[cemilcengiz]

Hi @ajfisch and @alontalmor, this doesn't seem fixed, I can still see the duplicated sentences in the dev split. Just manually check the first 10 examples, I found 3 of them still contain duplicates.

e.g., the first example (id=5a8c7595554299585d9e36b6)

and the fourth example, (id=5a85ea095542994775f606a8):

10th example, (id=5a7166395542994082a3e814)

the problem seems still exist, or it didn't get fully fixed.

Note: I downloaded the dev split fresh from https://s3.us-east-2.amazonaws.com/mrqa/release/v2/dev/HotpotQA.jsonl.gz.

Is it possible that the problem got fixed, but somehow the s3 cache was not purged?

This problem still exists in 2021.