Many of us use Django to build multi-tenant applications where every user only ever gets access to a small, separated fraction of the data in our application, while at the same time having some global functionality that makes separate databases per client infeasible. While Django does a great job protecting us from building SQL injection vulnerabilities and similar errors, Django can't protect us from logic errors and one of the most dangerous types of security issues for multi-tenant applications is that we leak data across tenants.
It's so easy to forget that one .filter call and it's hard to catch these errors
in both manual and automated testing, since you usually do not have a lot of clients
in your development setup. Leaving radical, database-dependent ideas
aside, there aren't many approaches available in the ecosystem to prevent these mistakes
from happening aside from rigorous code review.
We'd like to propose this module as a flexible line of defense. It is meant to have little impact on your day-to-day work, but act as a safeguard in case you build a faulty query.
There's nothing required apart from a simple
pip install django-scopes
This library is tested against Python 3.8-3.11 and Django 3.2-4.2.
Let's assume we have a multi-tenant blog application consisting of the three models Site,
Post, and Comment:
from django.db import models
class Site(models.Model):
name = models.CharField(…)
class Post(models.Model):
site = models.ForeignKey(Site, …)
title = models.CharField(…)
class Comment(models.Model):
post = models.ForeignKey(Post, …)
text = models.CharField(…)In this case, our model Site acts as the tenant for the blog posts and their comments, hence
our application will probably be full of statements like
Post.objects.filter(site=current_site), Comment.objects.filter(post__site=current_site),
or more complex when more flexible permission handling is involved. With django-scopes, we
encourage you to still write these queries with your custom permission-based filters, but
we add a custom model manager that has knowledge about posts and comments being part of a
tenant scope:
from django_scopes import ScopedManager
class Post(models.Model):
site = models.ForeignKey(Site, …)
title = models.CharField(…)
objects = ScopedManager(site='site')
class Comment(models.Model):
post = models.ForeignKey(Post, …)
text = models.CharField(…)
objects = ScopedManager(site='post__site')The keyword argument site defines the name of our scope dimension, while the string
'site' or 'post__site' tells us how we can look up the value for this scope dimension
in ORM queries.
You could have multi-dimensional scopes by passing multiple keyword arguments to
ScopedManager, e.g. ScopedManager(site='post__site', user='author') if that is
relevant to your usecase.
Now, with this custom manager, all queries are banned at first:
>>> Comment.objects.all()
ScopeError: A scope on dimension "site" needs to be active for this query.
The only thing that will work is Comment.objects.none(), which is useful e.g. for Django
generic view definitions.
You can now use our context manager to specifically allow queries to a specific blogging site, e.g.:
from django_scopes import scope
with scope(site=current_site):
Comment.objects.all()This will automatically add a .filter(post__site=current_site) to all of your queries.
Again, we recommend that you still write them explicitly, but it is nice to know to have a
safeguard.
Of course, you can still explicitly enter a non-scoped context to access all the objects in your system:
with scope(site=None):
Comment.objects.all()This also works correctly nested within a previously defined scope. You can also activate multiple values at once:
with scope(site=[site1, site2]):
Comment.objects.all()Sounds cumbersome to put those with statements everywhere? Maybe not at all: You probably
already have a middleware that determines the site (or tenant, in general) for every request
based on URL or logged in user, and you can easily use it there to just automatically wrap
it around all your tenant-specific views.
Functions can opt out of this behavior by using
from django_scopes import scopes_disabled
with scopes_disabled():
…
# OR
@scopes_disabled()
def fun(…):
…Please note that django-scopes is also active during migrations, so if you are writing a
data migration – or have written one in the past! – you'll have to add appropriate scoping
or use the scopes_disabled context.
If you were already using a custom manager class, you can pass it to a ScopedManager with the _manager_class
keyword like this:
from django.db import models
from django.db import models
class SiteManager(models.Manager):
def get_queryset(self):
return super().get_queryset().exclude(name__startswith='test')
class Site(models.Model):
name = models.CharField(…)
objects = ScopedManager(site='site', _manager_class=SiteManager)Assume you've got two models User and Post. Using the examples above, you can ensure that users only ever see their own diary posts. But how about leaking other users to the currently logged in user? If you application doesn't have much (or any) interaction between users, you can scope the user model. Please note that you'll need a custom user model. Which base classes your user and manager work off will very between projects.
class User(AbstractUser):
objects = ScopedManager(user='pk', _manager_class=UserManager)
# (...)Activating the scope comes with a little caveat - you need to use the users primary key, not the whole object:
with scope(user=request.user.pk):
# do something :)With django-scopes, a seemingly innocent query like
Comment.objects.select_for_update().get(pk=3)could cause unexpected locking across your database, since django-scopes will auto-add one or more JOIN statements to the query, and joined tables will also be locked.
One possible fix is of course using scopes_disabled(), around this query.
On most modern databases, there's also a way to specify explicitly which tables you want locked:
Comment.objects.select_for_update(of=("self",)).get(pk=3)You can check if your database supports this feature at runtime using connection.features.has_select_for_update_of.
django-scopes is not compatible with the django admin out of the box, integration requires a custom middleware. (If you write one, please open a PR to include it in this package!)
We want to enforce scoping by default to stay safe, which unfortunately breaks the Django test runner as well as pytest-django. For now, we haven't found a better solution than to monkeypatch it:
from django.test import utils
from django_scopes import scopes_disabled
utils.setup_databases = scopes_disabled()(utils.setup_databases)You can wrap many of your test and fixtures inside scopes_disabled() as well, but we wouldn't advise to do it with all of them: Especially when writing higher-level functional tests, such as tests using Django's test client or tests testing celery tasks, you should make sure that your application code runs as it does in production. Therefore, writing tests for a project using django-scopes often looks like this:
@pytest.mark.django_db
def test_a_view(client):
with scopes_disabled():
u = User.objects.create(...)
client.post('/user/{}/delete'.format(u.pk))
with scopes_disabled():
assert not User.objects.filter(pk=u.pk).exists()If you want to disable scoping or activate a certain scope whenever a specific fixture is used, you can do so in py.test like this:
@pytest.fixture
def site():
s = Site.objects.create(...)
with scope(site=s):
yield sWhen trying to port a project with lots of fixtures, it can be helpful to roll a small py.test plugin in your conftest.py to just globally disable scoping for all fixtures which are not yielding fixtures (like the one above):
@pytest.hookimpl(hookwrapper=True)
def pytest_fixture_setup(fixturedef, request):
if inspect.isgeneratorfunction(fixturedef.func):
yield
else:
with scopes_disabled():
yieldWhen using model forms, Django will automatically generate choice fields on foreign
keys and many-to-many fields. This won't work here, so we supply helper field
classes SafeModelChoiceField and SafeModelMultipleChoiceField that use an
empty queryset instead:
from django.forms import ModelForm
from django_scopes.forms import SafeModelChoiceField
class PostMethodForm(ModelForm):
class Meta:
model = Comment
field_classes = {
'post': SafeModelChoiceField,
}We noticed that django-filter also runs some queries when generating filtersets.
Currently, our best workaround is this:
from django_scopes import scopes_disabled
with scopes_disabled():
class CommentFilter(FilterSet):
…One subtle class of bug that can be introduced by adding django-scopes to your project is if you try to generate unique identifiers in your database with a pattern like this:
def generate_unique_value():
while True:
key = _generate_random_key()
if not Model.objects.filter(key=key).exists():
return keyIf you want keys to be unique across tenants, make sure to wrap such functions with scopes_disabled()!
When using a ModelForm (or class based view) to create or update a model, unexpected IntegrityErrors may occur. ModelForms perform a uniqueness check before actually saving the model. If that check runs in a scoped context, it cannot find conflicting instances, leading to an IntegrityErrors once the actual .save() happens. To combat this, wrap the call in scopes_disabled().
class Site(models.Model):
name = models.CharField(unique=True, …)
# (...)
def validate_unique(self, *args, **kwargs):
with scopes_disabled():
super().validate_unique(*args, **kwargs)If you'd like to read more about the practical use of django-scopes, there is a blog post about its introduction in the pretix project.
Here is a guide on how to write a shell_scoped
django-admin command to provide a scoped Django shell.