-
-
Notifications
You must be signed in to change notification settings - Fork 3.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Index date and ID together on builds #6926
Index date and ID together on builds #6926
Conversation
- The Django admin automatically adds the ID to the ordering in order to get a deterministic sort order. - In our setup, this *greatly* slows down the query since date and ID are not indexed together. - This index may be very slow to add on .org where the Build table has ~6.8M rows.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think this is fine. However, won't this fail if we click on another field that it's not date
in that page, like state
? I suppose that Django will still add id
and we will end up ordering by state
and id
which is again, not indexed.
Another different approach could be to override ModelAdmin.get_ordering
to always remove the id
unless it's the only sorting option. See https://docs.djangoproject.com/en/2.2/ref/contrib/admin/#django.contrib.admin.ModelAdmin.get_ordering
Probably. However, to filter by state we probably want to use the filters not the sorts. Those already work in production.
I can give this a try. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I meant to ship this today -- we could ship it to the admin instance as a hotfix if we wanted to test it prior.
Though I guess its a DB migration, so not easy to deploy to a single server :) |
I tested this and the deterministic build ordering overrides this method. Specifically, even with a method like: def get_ordering(self, request):
return ['-date'] ...the final query generated by the admin ends with |
Actually, the final query when you sort by state ends with |
I tested adding the index in production. It does allow the build admin to load. Filtering works like a charm. However, sorting by other columns can cause the build admin to still time out. Also searching can also trigger timeouts. I still think having this index is better than not having it as this gives an easy look into what builds are in which states across our servers from the admin. |
Currently the production Django admin for Builds times out. This is a fix for that.
This index may be very slow to add on .org where the Build table has 6.8M rows.In testing, it was added in less than a minute.Below is the query plan in production. Virtually all the time is spent on sorting.
I'm not 100% sure this will fix the problem butit should be pretty easy to verify before we roll it out: