Django's TEST_RUNNER setting is ignored

[barbolani]

Using Django 1.9 it seems that Py.test ignores the TEST_RUNNER specified in Django's settings. Looking in pytest_django.compat, I see that it directly tries to instantiate a DjangoTestRunner instead of using the settings.TEST_RUNNER

At least in Django 1.9 there is django.test.utlils.get_runner which locates and returns the runner class.

[pelme]

This is a misunderstanding, pytest is the test runner, not the thing that is specified in the Django TEST_RUNNER setting. When choosing to use pytest and pytest-django, you are choosing not to use the Django test runner system. :)

The DiscoverRunner in the pytest-django source is used as a workaround since there is no other way to reuse the code within Django to teardown the databases.

This has now been fixed in Django:
django/django@e76981b

And in pytest-django:
9d3f2b2

In the future when we drop support for Django versions prior to 1.11, the reference to DiscoverRunner can finally be removed from the pytest-django sources.

[barbolani]

Thanks for the reply. Besides the bit of self contradiction between "... you are not choosing to use the Django test runner system" and "... there is no other way to reuse the code within Django to teardown the databases" (so I'm choosing not to use it but I'm in fact using it), the fix you point to in pytest-django calls directly the Django methods instead of using the TestRunner. From a customization point of view It is not worse than the previous version, as Django's runner calls also these, but neither is an improvement. And as you correctly point out, it removes an unneeded dependency.

However, my assumption is that the ability to customize the TestRunner was provided by the Django team in order to cater for the many assorted and diverse environments where one may want to develop Django apps and run Django tests. That includes setting up test databases and dealing with bugs that you find when you venture outside the well charted MariaDB/PostgreSQL worlds. And it is very useful to have it.

In my opinion this issue needs to be at least pointed out in the pytest-django documentation, same as the multi_db support. Or at the very minimum put up a note saying that pytest-django is not a 100% compatible replacement for using Django's test framework and tooling. Don't want to make this look detrimental to either pytest or pytest-django, just trying to avoid people wasting time attempting to make pytest-django work in certain environments when it is clearly not designed to do so.