compat: handle SyntaxError when importing simplejson

[mgeisler]

We officially support Python 2.6 to 3.3, but simplejson does not
support Python 3.1 or 3.2:

simplejson/simplejson#66

Importing simplejson on Python 3.2 results in a SyntaxError because
simplejson uses the u'...' syntax (the syntax was not supported in
Python 3.0 to 3.2).

Support for loading simplejson instead of the stdlib json module was
added by #710:

https://github.com/kennethreitz/requests/pull/710

No mention was made of the lack of support for Python 3.2, but it was
mentioned that simplejson can be faster than the stdlib json module.

[Lukasa]

Thanks, this looks good to me! 🍰 I'm not 100% sure it'll get merged, we may drop support for 3.2 soon, but if we don't we'll merge it.

[mgeisler]

I ran into this for a project of mine where we try to support Python 3.2. However, it also affects much more significant projects, in particular pip. I didn't realize this at first, but pip includes a copy of requests and it fails to install packages if simplejson happens to be already installed.

So I would suggest not dropping support for 3.2 before pip does.

[mgeisler]

The error for pip can be provoked like this:

$ virtualenv -p python3.2 venv32
$ source venv32/bin/activate
$ pip install simplejson

This will fail to compile the C extension, but simplejson is still installed as pure Python. This means that requests fail when imported from pip:

$ pip install anything-you-want
Traceback (most recent call last):
  File "/home/mg/tmp/venv32/bin/pip", line 7, in <module>
    from pip import main
  File "/home/mg/tmp/venv32/lib/python3.2/site-packages/pip/__init__.py", line 11, in <module>
    from pip.vcs import git, mercurial, subversion, bazaar  # noqa
  File "/home/mg/tmp/venv32/lib/python3.2/site-packages/pip/vcs/mercurial.py", line 9, in <module>
    from pip.download import path_to_url
  File "/home/mg/tmp/venv32/lib/python3.2/site-packages/pip/download.py", line 22, in <module>
    from pip._vendor import requests, six
  File "/home/mg/tmp/venv32/lib/python3.2/site-packages/pip/_vendor/requests/__init__.py", line 58, in <module>
    from . import utils
  File "/home/mg/tmp/venv32/lib/python3.2/site-packages/pip/_vendor/requests/utils.py", line 25, in <module>
    from .compat import parse_http_list as _parse_list_header
  File "/home/mg/tmp/venv32/lib/python3.2/site-packages/pip/_vendor/requests/compat.py", line 77, in <module>
    import simplejson as json
  File "/home/mg/tmp/venv32/lib/python3.2/site-packages/simplejson/__init__.py", line 114, in <module>
    from .encoder import JSONEncoder, JSONEncoderForHTML
  File "/home/mg/tmp/venv32/lib/python3.2/site-packages/simplejson/encoder.py", line 21
    ESCAPE = re.compile(u'[\\x00-\\x1f\\\\"\\b\\f\\n\\r\\t\u2028\u2029]')
                                                                       ^
SyntaxError: invalid syntax

[Lukasa]

Well, that's a pain in the neck. Seems like a good idea though, and seeing as pip is likely to still want 3.2 I see no reason not to merge this.

[mgeisler]

Cool, thanks for looking at it!

[sigmavirus24]

I could be wrong, but I think I remember @dstufft saying that pip support for 3.2 wasn't a high priority. That aside, I'm not quite sure why we support using simplejson. If @mgeisler hadn't linked to #710 I would have guessed it was to support Python 2.5 (which we no longer do).

[mgeisler]

My first attempt at solving this was to reverse the imports, that is try import json first and then import simplejson as a fallback. But then I discovered that simplejson apparently has some speedups that the author of #710 wanted to take advantage of.

In my own projects (for Python 2.6+) I've always just imported json and let that be good enough.

[sigmavirus24]

Either way y'all are right that it's better to have this than to not. It doesn't increase the complexity of the code and if anything the simplejson maintainers should be scolded for not supporting 3.2 :P. I'm 👍

[dstufft]

3.2 is important, 3.1 is not.

[kennethreitz]

This looks like a great change.