-
Notifications
You must be signed in to change notification settings - Fork 404
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
ChecklistAugmenter AssertionError #638
Comments
I get the same error with |
I've changed the init params of sentence = Sentence(
self.text, use_tokenizer=textattack.shared.utils.words_from_text
) To: sentence = Sentence(
textattack.shared.utils.words_from_text(self.text), use_tokenizer=False
) And it works like a charm |
From PR #293-(Oct 6, 2020) , we send the function textattack.shared.utils.words_from_text for use_tokenizer in the pos_of_word_index function.
And as of PR "Major refactoring of internal label logic #2645" in the flairNLP Repo (Feb 11, 2022) in "https://github.com/flairNLP/flair/blob/master/flair/data.py" - Flair does not allow callable functions as input for the Sentence Class. To follow Flair's implementation and fix the errors seen in Issues #642 and #638, we should revert PR "have flair use textattack tokenization method #293". I can go ahead and submit a PR to revert PR #293 if it makes overall sense. |
With the recent update in the Flair Package, the Sentence Class no longer accepts a Callable function as the tokenization method. Adding a TokenizerWrapper to textattack.shared.utils.words_from_text is more suitable to retain funcionality instead of having to revert "PR have flair use textattack tokenization method QData#293" Flair also no longer supports get_tag attribute for token , updated strings.py function to follow new Flair requirements. Fixes QData#642 Fixes QData#638
Describe the bug
ChecklistAugmenter is bugged, error is
``AssertionError: Unexpected type of parameter 'use_tokenizer'. Parameter should be bool or Tokenizer```
To Reproduce
Run Augmentation example notebook:
https://colab.research.google.com/github/QData/TextAttack/blob/master/docs/2notebook/3_Augmentations.ipynb
The text was updated successfully, but these errors were encountered: