Compatibility with Django 2.2

[ncknuna]

This is basically identical to #610, but for a newer Django version.

Django recently bumped the required mysqlclient version to 1.3.13 (django/django@88619e6), but pymysql's version string only says it's only compatible with 1.3.12 (#623)

Any way we can get at least 1.3.13 support? Like the reporter of that other issue, I don't really know the implications of this change, and I can just make mysqlclient work in the interm, but I do prefer pymysql for ease of use on mac. Thanks so much for all your work supporting this library! :)

[eavictor]

Hi ncknuna,
The problem is not only version.

I believe you already tried modified the version by yourself and still having trouble.

Here is why

mysqlclient returns bytes object
PyMySQL returns str object

Django 2.1.x used to cast every query into "str" like below
https://github.com/django/django/blob/stable/2.1.x/django/db/backends/mysql/operations.py#L134

Django 2.2.x they changed the code by using query.decode instead of force_text method.
https://github.com/django/django/blob/stable/2.2.x/django/db/backends/mysql/operations.py#L140

Could you also help open a ticket on Django's issue tracking system ?
It's relatively easy to fix from Django side.

e.g.

from django.utils.encoding import force_text

def last_executed_query(self, cursor, sql, params):
    # With MySQLdb, cursor objects have an (undocumented) "_executed"
    # attribute where the exact query sent to the database is saved.
    # See MySQLdb/cursors.py in the source distribution.
    query = getattr(cursor, '_executed', None)
    if query is not None:
        if type(query) == bytes:
            query = query.decode(errors='replace')  # mysqlclient
        elif type(query) == str:
            query = query.encode(errors='replace')   # PyMySQL
        else:
            query = force_text(query, errors='replace')  # fallback compatibility ?
    return query

[ncknuna]

@eavictor Ah, got it. Thanks so much for explaining! Issue filed here: https://code.djangoproject.com/ticket/30380

Just curious: if PyMySQL has compatibility with MySQLdb via install_as_MySQLdb(), which of PyMySQL and mysqlclient deviate from the MySQLdb interface here?

[eavictor]

@ncknuna Could you test if the error goes away with django/django#11259 made by felixxm ?

[ncknuna]

I'm not currently blocked since I'm using mysqlclient in the meantime, so I don't think I'll get to this in a while, so if someone else wants to pick this up, that'd be great. Otherwise I'll try to poke at it when I get the chance!

[eavictor]

I will try squeeze some time this weekend upgrade django_backend_test repo and test if fix from Django works.
Then I will create a PR (change MySQLdb version to 1.4.2) if all field-tests are passed.

[eavictor]

Edit:
Add PY2 support

1. Test results:
   Tested with Python 3.7.3 on Windows 10 RS5, MacOS 10.14.4 Mojave, Debian 9,9 with django_backend_test and django:master commit 8c5649b.
   No errors pop up

Note:
There's no backport to django stable/2.2.x branch available.
My Django PR#11318 to stable/2.2.x is closed by felixxm.
Reason: This patch doesn't qualify for a backport.

2. About patch on PyMySQL side:
   My idea of MySQLdb version patch is leave MySQLdb version (1, 3, 12, "final", 0) as default
   By passing an extra argument MySQLdb_version to pretend as any version of MySQLdb.

This can prevent opening same type of issues again in future.

# __init__.py
from __future__ import print_function
import sys

...

version_info = (1, 3, 12, "final", 0)

...

def install_as_MySQLdb(MySQLdb_version=None):
    """
    After this function is called, any application that imports MySQLdb or
    _mysql will unwittingly actually use pymysql.
    """
    if type(MySQLdb_version) == tuple and len(MySQLdb_version) == 5:
        print("WARNING: Use at your own risk !!\nSet MySQLdb version = {}".format(MySQLdb_version))
        global version_info
        version_info = MySQLdb_version
    sys.modules["MySQLdb"] = sys.modules["_mysql"] = sys.modules["pymysql"]

Default behavior of install_as_MySQLdb:

>>> import pymysql
>>> pymysql.install_as_MySQLdb()
>>> pymysql.version_info
(1, 3, 12, 'final', 0)
>>>

Passing extra argument:

>>> import pymysql
>>> pymysql.install_as_MySQLdb(MySQLdb_version=(1, 4, 2, "final", 0))
WARNING: Use at your own risk !!
Set MySQLdb version = (1, 4, 2, 'final', 0)
>>> pymysql.version_info
(1, 4, 2, 'final', 0)
>>>

[methane]

@eavictor Thanks for your testing.
Anyway, I don't want to add such API. Anyone can pymysql.version_info = (1, 4, 2, 'final', 0) at their own risk. No new API is needed for such hack.

[Gauravdarkslayer]

I'm getting this error. Please help.

[mcrowson]

This looks like it will be addressed in the Django 3.0 release which is slated for December 2, 2019.
https://code.djangoproject.com/wiki/Version3.0Roadmap

[mwarkentin]

@mcrowson I went to that page but wasn't able to find anything related to this - any chance you could point to the update you were talking about?

[ace-han]

Django 3.0 has been out there already. However, pymysql is still not working with django==3.0.
Am I missing anything?

$ pip freeze | grep -Ei "(django)|(pymysql)"
Django==3.0
django-braces==1.13.0
django-filter==2.2.0
django-rest-framework-social-oauth2==1.1.0
djangorestframework==3.10.3
djangorestframework-filters==1.0.0.dev0
PyMySQL==0.9.3

$ python manage.py shell
...
raise ImproperlyConfigured('mysqlclient 1.3.13 or newer is required; you have %s.' % Database.__version__)
django.core.exceptions.ImproperlyConfigured: mysqlclient 1.3.13 or newer is required; you have 0.9.3.

[eavictor]

Hello @ace-han

Please read the reply from methane at May/06/2019.
You have to specify version_info manually.

import pymysql
pymysql.version_info = (1, 4, 6, 'final', 0)  # change mysqlclient version
pymysql.pymysql.install_as_MySQLdb()

[pedrogk]

I can confirm that adding these lines to your settings.py (at the beginning of the database section) works in Django 3.0. No need to do any of the other mysterious craft mentioned previously in the thread.

import pymysql
pymysql.version_info = (1, 4, 6, 'final', 0)  # change mysqlclient version
pymysql.pymysql.install_as_MySQLdb()

[methane]

Thank you, but what "I can confirm" means?
Django testsuite passes without any error?

[pedrogk]

I meant that I tried it and it worked. I did not run the Django testsuite.

I was getting the error mentioned in this thread when trying to run my app with Django 2.2 or later and PyMySQL. So, I modified the settings.py file of my Django app to change the mysqlclient version, and it runs properly now.

[Naddiseo]

PyMySQL doesn't work (afaik) in Django 2.2 because the byte handling patches mentioned earlier in the thread weren't backported. However, it seems to work fine in Django 3.0 after setting pymysql.version_info, and at $work I can run manage.py check on our 9 year old codebase without it crashing.

[MiunaChan]

This helped me，thanks

[ajl100b]

I can confirm that adding these lines to your settings.py (at the beginning of the database section) works in Django 3.0. No need to do any of the other mysterious craft mentioned previously in the thread.

import pymysql
pymysql.version_info = (1, 4, 6, 'final', 0)  # change mysqlclient version
pymysql.pymysql.install_as_MySQLdb()

pymysql.pymysql.install_as_MySQLdb()

should be

pymysql.install_as_MySQLdb()