-
Notifications
You must be signed in to change notification settings - Fork 6.8k
fix BucketSentenceIter bug related to #11430 #11580
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Would you mind adding a small test for this change?
Of course not thank you for the review. |
I meant you can absolutely feel free to write a small test. I'm sorry if the answer I wrote before confuses you. |
@szha I added test case , by adding smallest empty bucket, and it reproduces error in previous rnn/io.py version and it works good in patched version. below is the reproduced error. |
I rebased pr. |
* fix BucketSentenceIter bug * add test case for smallest empty bucket
* fix BucketSentenceIter bug * add test case for smallest empty bucket
Description
BucketSentenceIter has bug
when there are buckets with no item.
It keeps invalid buckets as self.buckets
but self.data contains only valid buckets and
it leads to the mismatch with sentence length of symbol and actual data shape.
I fixed the problem by checking valid buckets and handed it to self.buckets.
This fixes issue #11430.
Checklist
Essentials
Please feel free to remove inapplicable items for your PR.