Breaking change in jsonschema==3.1.0 for Python 2.7

[logachev]

Recent update 3.1.0 introduces a breaking change for Python 2.7.

Here is a code sample that works with 3.0.0 but is broken with 3.1.0 because of the switch from re to js_regex: #609

from jsonschema import Draft4Validator as Validator
schema = {'type': 'string', 'pattern': r'\d+'}
validator = Validator(schema)
list(validator.iter_errors('123'))

Thrown exception:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python2.7/site-packages/jsonschema/validators.py", line 328, in iter_errors
    for error in errors:
  File "/usr/local/lib/python2.7/site-packages/jsonschema/_validators.py", line 200, in pattern
    not js_regex.compile(patrn).search(instance)
  File "/usr/local/lib/python2.7/site-packages/js_regex/_impl.py", line 39, in compile
    raise TypeError("pattern={!r} must be a unicode string".format(pattern))
TypeError: pattern='\\d+' must be a unicode string

[Julian]

Having a look...

[Julian]

CC @Zac-HD.

[Julian]

OK, I think I don't see how to fix this yet, seems likely that it needs to happen downstream in js-regex, but I at least see how this slipped by, which is the changes to test_validators here: https://github.com/Julian/jsonschema/pull/609/files#diff-523a4ebdb045b5a1613d6711add86c7fR852

Not sure how I missed that during the review, but pretty sure that modification to the tests is our culprit.

[Julian]

OK and now I have some idea of what the issue is :( :(, which seems to me yeah downstream, and is due to the ever-hazardous from __future__ import unicode_literals here plus here.

[Julian]

OK, given that there's 2 separate issues here (#612 and #611) and that this is fairly fundamental functionality, I've temporarily pushed out v3.1.1 reverting the switchover.

We'll add some tests for the two cases here (and anything else we missed), and then there likely will be a v3.1.2 that goes back again to js-regex again.

[Julian]

Looking back at the test changes, not really good that those were mixed in with a rather unrelated test case, it's unsurprising that it wasn't obvious not to change them... so probably this should have been tested much more explicitly anyhow. Will need to add tests specifically here on passing bytes in.

[Zac-HD]

Just to check, do we want to support bytes on Python 3? My understanding was that JS was always at least Unicode BMP, which would not fit in the bytes type...

[Julian]

I am fairly sure on Py3 the json module always returns Unicode yeah. But needs double checking, I always forget.

…

On Wed, Oct 9, 2019, 20:00 Zac Hatfield-Dodds ***@***.***> wrote: Just to check, do we want to support bytes on Python 3? My understanding was that JS was always at least Unicode BMP, which would not fit in the bytes type... — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#611>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AACQQXUMQKGRJ5QOYUGK3TLQNZWATANCNFSM4I7C6EUQ> .

[Julian]

@Zac-HD I think 204c0c6 covers us for this one.

[logachev]

Thanks @Julian for a quick mitigation!

[Julian]

Don't fully thank me until we're re-released :)

(And then thank @Zac-HD for helping contribute something I had 0 desire to ever get into, so real props for that as soon as we get this out the door :)

[Zac-HD]

This is fixed in js-regex == 1.0.1 🎉

(the metacharacters-in-sets thing is trickier, but in progress)