[Bugfix] Invalid prefix exception with the new transformers library #24180
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
gshtras
requested review from
WoosukKwon,
alexm-redhat,
comaniac,
njhill,
robertgshaw2-redhat and
ywang96
as code owners
…n't match exactly Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com> Change condition to check for inclusion Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com> Fix DecodeStream API change Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
gshtras
force-pushed
the
transformers_prefix_fix_cont
branch
from
73e5873 to
4ad2710
Compare
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request addresses a bug where an Invalid prefix encountered exception from the transformers library was not being handled correctly due to a change in the exception message format. The fix correctly uses substring checking to handle both old and new message formats and also updates the DecodeStream constructor call to use keyword arguments for better compatibility. I've suggested a minor improvement to make the exception check more robust.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
#17448 fixed in #19449 has now re-appeared
Before 4.56 the exception text was simply
Invalid prefix encounteredWith transformers library == 4.56 the error message looks like
Exception: Invalid prefix encountered while decoding stream. Token ID: 30049, Expected prefix: '섬', Actual string: '�����ầand doesn't match exactly.
To filter it out and preserve the backwards compatibility with the old one, the common pattern would be to check for whether INVALID_PREFIX_ERR_MSG is included in the new message
Also the API for DecodeStream changed from
def __init__(self, skip_special_tokens):to
def __init__(self, ids=None, skip_special_tokens=False):