11291 optimize GraphQL queries

[arthanson]

Fixes: #11291

This optimizes the GraphQL queries by automatically putting in prefetch_related to queried fields. Results:

I made a bunch of VLANs and a query that shows the query issue resulting in a 10x or 20x reduction of queries, here are the query counts:

• old code (before filtering at all levels): 985 queries
• current code (with filtering at all levels): 1852 queries
• with query optimizer: 84 queries

Tested with sample queries for nested types, nested filters, fragments (triple dot notation) and they all worked successfully.

Couple Notes on implementation:

There is some interaction between graphene / FilterSets and the optimizer. The optimizer works fine with graphene but without FilterSets and works fine outside of graphene with FilterSets, but combining Graphene, FilterSets and the optimizer produces extra queries. So netbox/netbox/graphql/fields.py if there are no filter params then it doesn't instantiate the FilterSet.

Propose this as a temporary solution as it results in 10x less queries in the non-filtered case and figuring out how to fix the interaction between graphql / filterset will be a big time sink I think so can create a separate issue for that.

There is a separate issue where this code: https://github.com/netbox-community/netbox/blob/develop/netbox/netbox/filtersets.py#L249 is adding in extra queries for custom-fields even if they are not referenced in the query-set. This is what causes the number of queries to double with the filtering at all levels feature that was added in NetBox 3.4. It isn't a bug in that feature, rather an existing bug that was just exposed by this change as it now applies filters at all levels in the schema. If the custom fields issue is fixed the querycount comes back down to 985 with filtering.

In the schmea files the lines:

    def resolve_vrf_list(root, info, **kwargs):
        return gql_query_optimizer(models.VRF.objects.all(), info)

were needed as unfortunately the optimizer code could not be put into fields.py which generically handles getting the schema class and filterset. The reason for this is two fold:

• The optimizer has to be inserted in code in the parent list_resolver initializer after it queries the class but before it passes it down into Django.
• second: fields.py list_resolver gets called for all lists, but you only want the optimizer called for the root resolver so it needs to go into the resolver functions in the schema files.

[v0tti]

I know this is just a draft so sorry if this is premature feedback. There seems to be a problem with accessing assigned objects in ip_address_list (maybe even in more cases but this is what I have run into).

Example query:

{
  ip_address_list(assigned_to_interface: true) {
    assigned_object {
      ... on InterfaceType {
        id
      }
      ... on VMInterfaceType {
        id
      }
    }
  }
}

Results in:

{
  "errors": [
    {
      "message": "name 'store' is not defined",
      "locations": [
        {
          "line": 2,
          "column": 3
        }
      ],
      "path": [
        "ip_address_list"
      ]
    }
  ],
  "data": {
    "ip_address_list": null
  }
}

[arthanson]

@v0tti not premature at all, that really helps. Looking into it now...

[arthanson]

@v0tti should be fixed now, thanks for catching this. Any other real-world testing you can do on this is greatly appreciated!