test_ucn fails on OpenVMS - AssertionError

[zessin5]

BPO	232817
Nosy	@tim-one

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2001-02-18.22:15:18.000>
created_at = <Date 2001-02-17.13:54:13.000>
labels = ['expert-unicode']
title = 'test_ucn fails on OpenVMS - AssertionError'
updated_at = <Date 2001-02-18.22:15:18.000>
user = 'https://bugs.python.org/zessin5'

bugs.python.org fields:

activity = <Date 2001-02-18.22:15:18.000>
actor = 'effbot'
assignee = 'effbot'
closed = True
closed_date = None
closer = None
components = ['Unicode']
creation = <Date 2001-02-17.13:54:13.000>
creator = 'zessin_5'
dependencies = []
files = []
hgrepos = []
issue_num = 232817
keywords = []
message_count = 8.0
messages = ['3438', '3439', '3440', '3441', '3442', '3443', '3444', '3445']
nosy_count = 3.0
nosy_names = ['tim.peters', 'effbot', 'zessin_5']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = None
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue232817'
versions = []

[zessin5]

test_ucn

On OpenVMS, this test fails with the following error message:

test test_ucn crashed -- exceptions.AssertionError: failed
to raise an exception when given a bogus character name

How can I diagnose this?

[tim-one]

Assigned to /F.

zessin_5, look at test_ucn.py, if you want. The error in question is raised in only one place, verifying that

"\N{blah}"

raises UnicodeError when passed thru unicode-escape. Don't know why it's not on your platform.

[zessin5]

Here is the result from my debugging:

UNICODEOBJECT.C
if (!ucnhash_CAPI->getcode(start, endBrace-start, &chr)) {

calls:
UNICODEDATA.C:
getcode(const char* name, int namelen, Py_UCS4* code)
{

There are 4 'return's in getcode().
After I replace the 'return -1;' with 'return 0;'
I get the exceptions.

[effbot]

thanks for spotting the typo. just wish I knew why
this didn't show up on any other platform...

[tim-one]

/F, I'm re-opening this until it's understood. It's clear that the test "worked" by accident before, yes? I stepped thru it ("N{blah}") in the debugger on Windows, under the old code. The reason it worked then is that, in PyUnicode_DecodeUnicodeEscape, the stack vrbl "chr" remained uninitialized all the way to label "store:", where a series of tests then checked this stack trash against various ranges. On Windows the trash happened to be the too-large-for-Unicode 0x0063f9dc, so triggered a different error by accident. Presumably under OpenVMS, the trash passed those tests by accident.

Perhaps it's impossible to ever test stack trash now, but I'd feel better if the code covered its ass there anyway. You?

[effbot]

Tim, "getcode" is supposed to set the "chr" value if it returns non-zero. After Uwe's fix, this is exactly what it does.

But DecodeUnicodeEscape is rather messy; I've discovered a few other cases where it can misbehave. I'll do something about it asap. /F

[tim-one]

I just had in mind initializing chr to a known-bogus value and assert'ing it doesn't still have that value when it's finally referenced, so that a reoccurrence of this (or similar) bug won't pass by accident again.

I understand that the code was intended to work as you say, but better safe than to trust that all future maintainers will figure that out when they're hacking in the middle of the 200+ lines of code across which this undocumented invariant is supposed to hold. Goodness, the code even initializes p, despite that it's set again within a mere dozen lines <wink>.

[effbot]

Okay, I just checked in a new version of DecodeUnicodeEscape (still not 100% happy with it, but it's better than before) /F