Create valid documents with authorization filtering

[Geal]

Closes #5648

Problem

To support the router's authorization feature, we have a "transform" query visitor that can be used to modify the query. In authorization, it is used to remove the parts of the query that are not authorized, resulting in null values being inserted in their place in the client response.
That visitor was generating invalid queries for 2 reasons:

• some fragments may not be used anymore because the selection wrapping the fragment spread was removed. Example:

query {
  a {
    ... F
  }
  b
}

fragment F on A {
  c
}

would be tranformed to this if we removed the a field, resulting in an unused fragment:

query {
  b
}

fragment F on A {
  c
}

while we would expect this instead:

query {
  b
}

• variables used in removed selections have to be removed from the query's arguments. Example:

query($arg1:Int, $arg2:String, $arg3:Int) {
  a(arg: $arg1) {
    c @directive(arg: $arg2)
  }
  b(arg: $arg3)
}

would be transformed to this if we removed the a field, resulting in $arg1 and $arg2 being unused in the query:

query($arg1:Int, $arg2:String, $arg1:Int) {
  b(arg: $arg3)
}

while we would expect this instead:

query($arg1:Int) {
  b(arg: $arg3)
}

Notes

To fix this, we have to take special care in how it is implemented, for the following reasons:

• the visitor is supposed to run in one pass, but this runs at odds with managing dependencies between operations (there can be multiple operations per query document) and fragments, and dependencies between fragments (a fragment can require an other through a spread in its selections)
• the visitor is first going through the list of fragments and producing their modified version in the end document, then the list of operations, but maybe after processing operations we would realize that we don't need some of the fragments
• the transform visitor should take care of cleaning up unused fragments, unused variables, but also removing fragment spreads for removed fragments, and do all of that without requiring special code from the Visitor trait implementers
• some of the Visitor implementers, like authorization filters, need to have all the information on fragments at the point of application. As an example, in ... F, if we defined fragment F on Obj with Obj being unauthorized for this query, or the fragment being removed because all of its selections were removed, the visitor must have processed the fragment first. And that also applies when one fragment inserts another fragment

Proposed solution

The visitor now follows this process:

• make a first light pass through the query's fragments, to get the dependencies between fragments, and reorder them, so that the first fragments do not use any other fragments, the next ones only use the previous ones in the list, and so on
• go through the ordered list of fragments:
  • modify their selections if needed
  • remove the fragment entirely if needed
  • record the list of variables it uses in field arguments and directives
  • record the list of fragments it uses
  • store the fragment but do not output it to the final document yet
• go through the list of operations:
  • modify their selections if needed
  • record the list of variables it uses in field arguments and directives
  • record the list of fragments it uses
  • for each fragment it uses:
    • get the list of variables it uses, add to the operation's list
    • get the list of fragments it uses, add to the operation's list, and go through their list of fragments and variables as well
  • filter the operation's variables, only keeping the ones we use
  • output the modified operation to the final document
• output to the final document all of the fragments that are used in at least one query

TODO:

• one test is not passing yet: introspection queries should not be filtered. But maybe that should be handled directly in bridge_query_planner.rs ? Maybe in a follow up PR?

---

Checklist

Complete the checklist (and note appropriate exceptions) before the PR is marked ready-for-review.

• Changes are compatible¹
• Documentation² completed
• Performance impact assessed and acceptable
• Tests added and passing³
  • Unit Tests
  • Integration Tests
  • Manual Tests

Exceptions

Note any exceptions here

Notes

Footnotes

1. It may be appropriate to bring upcoming changes to the attention of other (impacted) groups. Please endeavour to do this before seeking PR approval. The mechanism for doing this will vary considerably, so use your judgement as to how and when to do this. ↩

2. Configuration is an important part of many changes. Where applicable please try to document configuration examples. ↩

3. Tick whichever testing boxes are applicable. If you are adding Manual Tests, please document the manual testing (extensively) in the Exceptions. ↩

[router-perf]

CI performance tests

• connectors-const - Connectors stress test that runs with a constant number of users
• const - Basic stress test that runs with a constant number of users
• demand-control-instrumented - A copy of the step test, but with demand control monitoring and metrics enabled
• demand-control-uninstrumented - A copy of the step test, but with demand control monitoring enabled
• enhanced-signature - Enhanced signature enabled
• events - Stress test for events with a lot of users and deduplication ENABLED
• events_big_cap_high_rate - Stress test for events with a lot of users, deduplication enabled and high rate event with a big queue capacity
• events_big_cap_high_rate_callback - Stress test for events with a lot of users, deduplication enabled and high rate event with a big queue capacity using callback mode
• events_callback - Stress test for events with a lot of users and deduplication ENABLED in callback mode
• events_without_dedup - Stress test for events with a lot of users and deduplication DISABLED
• events_without_dedup_callback - Stress test for events with a lot of users and deduplication DISABLED using callback mode
• extended-reference-mode - Extended reference mode enabled
• large-request - Stress test with a 1 MB request payload
• no-tracing - Basic stress test, no tracing
• reload - Reload test over a long period of time at a constant rate of users
• step-jemalloc-tuning - Clone of the basic stress test for jemalloc tuning
• step-local-metrics - Field stats that are generated from the router rather than FTV1
• step-with-prometheus - A copy of the step test with the Prometheus metrics exporter enabled
• step - Basic stress test that steps up the number of users over time
• xlarge-request - Stress test with 10 MB request payload
• xxlarge-request - Stress test with 100 MB request payload

[Geal]

while working on this I uncovered some issues linked to fragment ordering: when encountering a fragment spread, authorization plugins need the referenced fragment to have been already processed, but this is not always the case if we process them in the order they are defined in. As an example:

query {
 ... F
}

fragment F on Query {
  ... G
}

fragment G on Query {
  op
}

In this query, when we look at the ... G fragment spread, G has not been processed yet, so we don't know which type it applies on, or if we keep that fragment: maybe all of its selection were removed, so we need to remove the fragment and all related spreads.

This is fixed in this PR by first going through the fragments and the spreads in their selections, then ordering the fragments by their dependencies (first processing fragments that do not reference other fragments) and porocessing them, then processing operations.

[BrynCooke]

I ran a few more tests locally and everything worked a treat.

[IvanGoncharov]

Values can have variables deeply nested, and there can be multiple variables per argument.

query Test($a: String, $b: String) {
  foo(arg: { someField: [[$a, "bar", $b]] })
}

It's works for arguments in both fields and directives

[IvanGoncharov]

Directives with variables can be also located on operation definition and fragment definitions.

query Test($a: String) @foo(arg: $a) {
  ...TestFragment
}

fragment TestFragment on Query @bar(arg: $a) {
  __typename
}

Currently, only directives on fields, inline fragments, and fragment spreads are tracked.