[lldb/test] Fix for flakiness in TestNSDictionarySynthetic #1262
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Summary: TestNSDictionarySynthetic sets up an NSURL which does not initialize its _baseURL member. When the test runs and we print out the NSURL, we print out some garbage memory pointed-to by the _baseURL member, like: ``` _baseURL = 0x0800010020004029 @"d��qX" ``` and this can cause a python unicode decoding error like: ``` UnicodeDecodeError: 'utf8' codec can't decode byte 0xa0 in position 10309: invalid start byte ``` There's a discrepancy here because lldb's StringPrinter facility tries to only print out "printable" sequences (see: isprint32()), whereas python rejects the StringPrinter output as invalid utf8. For the specific error seen above, lldb's `isprint32(0xa0) = true`, even though 0xa0 is not really "printable" in the usual sense. The problem is that lldb and python disagree on what exactly is "printable". Both have dismayingly hand-rolled utf8 validation code (c.f. _Py_DecodeUTF8Ex), and I can't really tell which one is more correct. I tried replacing lldb's isprint32() with a call to libc's iswprint(): this satisfied python, but broke emoji printing :|. Now, I believe that lldb (and python too) ought to just call into some battle-tested utf library, and that we shouldn't aim for compatibility with python's strict unicode decoding mode until then. FWIW I ran this test under an ASanified lldb hundreds of times but didn't turn up any other issues. rdar://62941711 Reviewers: JDevlieghere, jingham, shafik Subscribers: lldb-commits Tags: #lldb Differential Revision: https://reviews.llvm.org/D79645 (cherry picked from commit f807d0b)
|
@swift-ci please test macos |
|
👍 |
|
LGTM. I've been trying to merge this for 10+ days, but kept hitting Linux PR testing failures.. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary:
TestNSDictionarySynthetic sets up an NSURL which does not initialize its
_baseURL member. When the test runs and we print out the NSURL, we print
out some garbage memory pointed-to by the _baseURL member, like:
and this can cause a python unicode decoding error like:
There's a discrepancy here because lldb's StringPrinter facility tries
to only print out "printable" sequences (see: isprint32()), whereas python
rejects the StringPrinter output as invalid utf8. For the specific error
seen above, lldb's
isprint32(0xa0) = true, even though 0xa0 is notreally "printable" in the usual sense.
The problem is that lldb and python disagree on what exactly is
"printable". Both have dismayingly hand-rolled utf8 validation code
(c.f. _Py_DecodeUTF8Ex), and I can't really tell which one is more
correct.
I tried replacing lldb's isprint32() with a call to libc's iswprint():
this satisfied python, but broke emoji printing :|.
Now, I believe that lldb (and python too) ought to just call into some
battle-tested utf library, and that we shouldn't aim for compatibility
with python's strict unicode decoding mode until then.
FWIW I ran this test under an ASanified lldb hundreds of times but
didn't turn up any other issues.
rdar://62941711
Reviewers: JDevlieghere, jingham, shafik
Subscribers: lldb-commits
Tags: #lldb
Differential Revision: https://reviews.llvm.org/D79645
(cherry picked from commit f807d0b)