Additional fixes for UTF-8 surrogate escapes #629
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Surrogate escapes in Unicode (non UTF-8 encoding) will be properly escaped with backslashes when encountered, versus breaking the transport layer.
Fixes to re-raise exceptions with different reasons Removes erroneous bytes decode where bytes are desired
Tests that a surrogate escape sequence is properly escaped with backslashes to produce valid UTF-8.
Use a Unicode Surrogate that properly escapes in both Python2 and Python3
Updating test to pass once surrogatepass is used
This replicates behavior between Python 2 and Python 3
Since `surrogatepass` will only ever explicitly occur when there are surrogate bytes encountered, there is no need to let the error throw and catch it, also uses single-quotes for consistency.
This is the same fix as accepted for ElasticSearch requests, except applied to the request logging mechanisms.
|
Hi @tylerjharden, we have found your signature in our records, but it seems like you have signed with a different e-mail than the one used in yout Git commit. Can you please add both of these e-mails into your Github profile (they can be hidden), so we can match your e-mails to your Github profile? |
|
@karmi I believe your bot is mistaken. The e-mails match, but I added my work e-mail just to be safe. |
|
Thanks for the patch! |
honzakral
pushed a commit
that referenced
this pull request
Aug 15, 2017
* Fixes non UTF-8 surrogateescapes Surrogate escapes in Unicode (non UTF-8 encoding) will be properly escaped with backslashes when encountered, versus breaking the transport layer. * Removes erroneous bytes decode and reraises Fixes to re-raise exceptions with different reasons Removes erroneous bytes decode where bytes are desired * Adds test for surrogate escapes in body Tests that a surrogate escape sequence is properly escaped with backslashes to produce valid UTF-8. * Use proper byte sequence for surrogate * Use if/else versus pass * Proper Unicode surrogate escape Use a Unicode Surrogate that properly escapes in both Python2 and Python3 * Passing test once surrogatepass is used Updating test to pass once surrogatepass is used * Use surrogatepass instead of backslashreplace This replicates behavior between Python 2 and Python 3 * Fixes whitespace * Simplifies with no exception block Since `surrogatepass` will only ever explicitly occur when there are surrogate bytes encountered, there is no need to let the error throw and catch it, also uses single-quotes for consistency. * Fixes Unicode Surrogate Escapes in request logging This is the same fix as accepted for ElasticSearch requests, except applied to the request logging mechanisms.
rciorba
added a commit
to rciorba/elasticsearch-py
that referenced
this pull request
Mar 2, 2018
* Fixes non UTF-8 surrogateescapes Surrogate escapes in Unicode (non UTF-8 encoding) will be properly escaped with backslashes when encountered, versus breaking the transport layer. * Removes erroneous bytes decode and reraises Fixes to re-raise exceptions with different reasons Removes erroneous bytes decode where bytes are desired * Adds test for surrogate escapes in body Tests that a surrogate escape sequence is properly escaped with backslashes to produce valid UTF-8. * Use proper byte sequence for surrogate * Use if/else versus pass * Proper Unicode surrogate escape Use a Unicode Surrogate that properly escapes in both Python2 and Python3 * Passing test once surrogatepass is used Updating test to pass once surrogatepass is used * Use surrogatepass instead of backslashreplace This replicates behavior between Python 2 and Python 3 * Fixes whitespace * Simplifies with no exception block Since `surrogatepass` will only ever explicitly occur when there are surrogate bytes encountered, there is no need to let the error throw and catch it, also uses single-quotes for consistency. * Fixes Unicode Surrogate Escapes in request logging This is the same fix as accepted for ElasticSearch requests, except applied to the request logging mechanisms.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
My previously merged PR did not handle some additional spots in the code where similar encoding is done, namely success logging.