-
-
Notifications
You must be signed in to change notification settings - Fork 4.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Tokenizer.add_special_case raises KeyError #656
Comments
A bit of follow up: I was going through the definition for I am not sure if the values expected by EDIT: Even when replacing keys for import spacy
nlp = spacy.load('en')
nlp.tokenizer.add_special_case('gimme',
[
{
'F': 'gim',
'L': 'give',
'pos': 'VERB'},
{
'F': 'me' }])
# Traceback (most recent call last):
# File "test.py", line 13, in <module>
# 'F': 'me' }])
# File "spacy/tokenizer.pyx", line 377, in spacy.tokenizer.Tokenizer.add_special_case (spacy/tokenizer.cpp:8460)
# File "spacy/vocab.pyx", line 342, in spacy.vocab.Vocab.make_fused_token (spacy/vocab.cpp:7907)
# File "spacy/morphology.pyx", line 39, in spacy.morphology.Morphology.assign_tag (spacy/morphology.cpp:3919)
# KeyError: 97 |
Thanks for this. The docs have gotten ahead of the library here — the current/old behaviour is pretty inconsistent, so I wrote up the intended usage, but haven't had time to fix it yet. Will definitely have this resolved in the next release, which should be up this week. |
This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs. |
The usage example provided in the documentation for Tokenizer.add_special_case raises a
KeyError
.Steps to reproduce:
Environment
The text was updated successfully, but these errors were encountered: