[RHCLOUD-36102] - Slow loading of data from roles endpoint perfo… #1354
Conversation
EvanCasey13
changed the title
There was a problem hiding this comment.
Based on the extra query you got from django silk, this seems to be n+1 query issue.
I think you can improve the performance by using prefetch related in the get_role_queryset method. We just need to include the permission lookup and external relation lookup there as well.
rbac/management/role/serializer.py
Outdated
| groups_in_count = serializers.SerializerMethodField() | ||
| groups_in = serializers.SerializerMethodField() |
There was a problem hiding this comment.
The task is to solve the slowness of the endpoint, but not requiring these two new fields to return. Are they required to be added?
There was a problem hiding this comment.
Looking more into it I can probably remove these from the serializer and add them dynamically from policies based on the query param passed
| @@ -324,7 +328,200 @@ def list(self, request, *args, **kwargs): | |||
| } | |||
|
|
|||
| """ | |||
| return super().list(request=request, args=args, kwargs=kwargs) | |||
| public_tenant = Tenant.objects.get(tenant_name="public") | |||
There was a problem hiding this comment.
Might have to coordinate with this: #1368 (comment)
…ynamically when passed in add_fields query param
tests/management/role/test_view.py
Outdated
| @@ -1210,7 +1209,7 @@ def test_list_role_with_additional_fields_principal_success(self): | |||
| self.assertEqual(len(response.data.get("data")), 5) | |||
|
|
|||
| role = response.data.get("data")[0] | |||
|
|
|||
tests/management/role/test_view.py
Outdated
| @@ -1118,6 +1116,7 @@ def test_list_role_with_groups_in_fields_for_admin_scope_success(self): | |||
|
|
|||
| response_data = response.data.get("data") | |||
| for iterRole in response_data: | |||
| print(iterRole.get("groups_in")) | |||
There was a problem hiding this comment.
this is for debugging and can be removed
rbac/management/role/view.py
Outdated
| ) | ||
|
|
||
| # Dynamic annotations | ||
| if "groups_in_count" in add_fields_split: |
There was a problem hiding this comment.
I think the groups_in_count, groups_in, or without are not mutual exclusive, so no if else needed here.
And obtain_groups_in might still needed here? There are some logic about scope and admin has to be included there
…rmance fixes
Link(s) to Jira
https://issues.redhat.com/browse/RHCLOUD-36102
Description of Intent of Change(s)
I used django silk to see how many queries were being executed for each request to the roles endpoint, initially this was 33 queries. This was happening as the role list functionality for the endpoint was using generic listing functionality provided by ListModelMixin from django.
I overwrite this listing functionality and implemented filtering, pagination etc again for the roles endpoint which reduced this from 33 to 3 queries overall.
Local Testing
How can the feature be exercised?
How can the bug be exploited and fix confirmed?
Is any special local setup required?
Checklist
Secure Coding Practices Checklist Link
Secure Coding Practices Checklist