Add Support For Automatic Persisted Queries

[pscarey]

Adds customisable support for Automatic Persisted Queries, with default settings compatible with apollo-client.

#110 already has discussion on this. We required support internally, so it's up to maintainers if support in this way is appropriate. Happy to make tweaks based on feedback if required.

Note this builds on #186.

[mcollina]

I'm not convinced by this implementation. How do you plan to scale this horizontally? The queries will be stored only on one peer.

[pscarey]

Interesting point - I think this is how Apollo Server implemented it, which is why I went down this road, then generalised the way the query is delivered.

The argument in favour would be that the cost of hashing a query the first time an instance sees it will still massively reduce the number of times the full query is sent over the wire.

---

Really, this entire interface or setting might be better off merged into the existing persisted queries solution, in a customisable manner, since there is no firm 'Best Practice', such that there is:

Persisted Query Settings:

• isPersistedQuery check if the given GraphQL request/query is persisted or regular.
• getHash get the hash from a GraphQL persisted query request, or falsy if not found.
• getQueryFromHash return the query for a given hash, or falsy if not found.
• getHashForQuery return the hash for a given query, or falsy if not to be stored.
• saveQuery store the query for the given hash.
• onlyPersisted whether to allow persisted queries or not.

Example implementations would be:

Current Persisted Queries (Ahead of time):

{
  isPersistedQuery: (r) => r.persisted,
  getHash: (r) => r.query,
  getQueryFromHash: (hash) => persistedQueries[hash], // Load from memory
  // getHashForQuery and saveQuery are not required as we do not save new queries
}

Apollo Client Compatible Automatic Persisted Queries (Runtime):

{
  isPersistedQuery: (r) => r.extensions.persistedQuery.sha256Hash && r.extensions.persistedQuery.version === 1,
  getHash: (r) => r.extensions.persistedQuery.sha256Hash ,
  getQueryFromHash: (hash) => persistedQueries[hash], // Load from memory
  getHashForQuery: (query) => sha256(query),
  saveQuery: (hash, query) => persistedQueries[hash] = query // Store in memory
}

Cached Apollo Client Compatible Automatic Persisted Queries (Runtime + Shared Cache):

{
  isPersistedQuery: (r) => r.extensions.persistedQuery.sha256Hash && r.extensions.persistedQuery.version === 1,
  getHash: (r) => r.extensions.persistedQuery.sha256Hash ,
  getQueryFromHash: (hash) => redis.get(hash), // Load from redis
  getHashForQuery: (query) => sha256(query),
  saveQuery: (hash, query) => redis.set(hash, query) // Store in redis
}

[mcollina]

My point is that the current interface (ahead of time) cannot be made dynamic without adding support for multiple instances of the server that store the query. Otherwise subsequent requests that hit a different server would not have the cached query.

In the code in this PR the “get” operation is synchronous and it should be made async (and likely moved in a common interface). In your list, the Persisted Query Settings:

• isPersistedQuery check if the given GraphQL request/query is persisted or regular.
• getHash get the hash from a GraphQL persisted query request, or falsy if not found.
• getQueryFromHash return the query for a given hash, or falsy if not found. must be async
• getHashForQuery return the hash for a given query, or falsy if not to be stored.
• saveQuery store the query for the given hash. must be async
• onlyPersisted whether to allow persisted queries or not.

---

I’m also skeptical that adding a delay of a round trip to Redis is going to improve latency much in practice. We are trading the cost of sending a few hundred bytes from the client to fetching those from Redis. Given how HTTP/1.1 keepalive and TCP work, sending a few hundreds bytes more should have no impact.
The risk is to have an ahead-of-line blocking on the redis connection which will make thus caching moot.

Notably this makes a lot of sense if our query is massive.

[pscarey]

I agree with your reasoning that it'd need to be async.

As a general rule, any persisted query requires that the payload reduction savings are greater than the lookup delay. How big the lookup delay is depends on the implementation.

My understanding is that:

Ahead of Time Persisted Queries (AoTPQ):

• Best performance in every use case.
• Hardest to set up, as all queries need to be compiled into hashes ahead of time.
• Maintaining support for old client versions (i.e. old queries) is non trivial and requires a very specific development workflow.

Automatic Persisted Queries (APQ):

• Using local memory will be slow for each of their first query, but then give equal performance compared to AoTPQ afterwards.
• The performance of this is basically equivalent to AoTPQ for long lived servers (i.e. not cloud functions).
• This is also the easiest to set up.
• Works well for unknown query structures, i.e. public APIs.

Cached APQ:

• Remove that first overhead at the cost of a roundtrip to the cache on every request.
• The performance for this would be worse than APQ for long lived servers, but better than APQ for short lived servers (cloud functions).
• The performance of this is heavily caveated on the payload reduction being sufficiently large to counteract the cost of the lookup time. However, it can reasonably be assumed than an in-datacenter Redis connection is much faster than e.g. a 3G mobile connection. Really this means it boils down to the use case, and the performance of the client.

---

Appreciate the responsiveness here, but I also want to flag that in order for us to migrate to fastify-gql, we need to support APQ as otherwise we'll lose backwards compatibility - obviously that doesn't mean that it's something fastify-gql should support - so if it's a no-go, we can fork.

[mcollina]

Could you confirm that you are using "sticky sessions" for APQ? How can you guarantee that the same client goes to the same server otherwise? What i do not understand about APQ is if it's actually feasible in production. It seems very likely that a client talking to a different server might not get the cached one.

I agree with your reasoning that it'd need to be async.

Then we are good, can you update the PR?

---

Appreciate the responsiveness here, but I also want to flag that in order for us to migrate to fastify-gql, we need to support APQ as otherwise we'll lose backwards compatibility - obviously that doesn't mean that it's something fastify-gql should support - so if it's a no-go, we can fork.

I'm cautious about adding feature, I think this is legit and we should add it in one form or another.

[pscarey]

I will do updates to both PR's tomorrow.

Could you confirm that you are using "sticky sessions" for APQ? How can you guarantee that the same client goes to the same server otherwise? What i do not understand about APQ is if it's actually feasible in production. It seems very likely that a client talking to a different server might not get the cached one.

Our use case is as follows:

Client:

• Mobile App running apollo-client with approx 20 unique queries.
• Prior versions of the mobile app are supported, but we haven't tracked legacy queries (i.e. they are no longer in our master branch, but are still valid queries due to no breaking changes).
• apollo-client attempts to issue a query using the persisted queries hash. Expected API responses are as per apollo-link-persisted-query.
• If the client receives the appropriate PersistedQueryNotFound response, then it resends the whole query. This causes the APQ cache to be populated for future requests.

Server:

• 3-4 replicas of our GraphQL API running in K8s.
• Queries are persisted when they are received in full, in local memory.
• After a short amount of usage, the most common queries will all have been persisted.
• After a little while longer, less frequently called queries are persisted.
• The machine continues its lifetime for a long period after, directly replying to persisted queries.

I hope this (very rough) diagram can covey the idea slightly better:

Sticky sessions would ensure that the cache of the machine returning PersistedQueryNotFound is the one populated by the query, but are not strictly necessary, as the queries will still be persisted in a short period of time compared to the lifetime of the machine.

[mcollina]

Thanks, it makes sense now. Please add a short version of that in the README.

[pscarey]

I've reworked the API & tidied things up a lot based on the conversation above.

• The proposed API gives defaults which align with the existing options, support for Apollo's APQ, and room for additional customisation.
• Documentation and types are updated.
• All persisted query tests have been moved into a seperate file.

[mcollina]

Can you rebase this on top of master?

[pscarey]

Have merged the other changes in, diff should be clean.

Thanks.

[pscarey]

Minor fixes have been completed. Don't mind the long process - happy to get the prompt feedback, and I appreciate it's a non trivial set of changes.

Should other casing also be updated?

• Prepared -> prepared
• PreparedOnly -> preparedOnly
• Automatic -> automatic

[mcollina]

persistedQueryProvider maybe?

[mcollina]

@pscarey any updates on this? It would be great to land!

[pscarey]

@mcollina Sorry about the delay - was away for a few days. Will be updating shortly.

[pscarey]

All updated. FYI, we've been running this in production for over a week, and have had no issues.

[pscarey]

Have implemented the feedback, but it looks like some changes from master are breaking CI?

[mcollina]

Yes, we are fixing them today, unfortunately something broke in #147

[mcollina]

I have the reverted the offending commit on master.

[mcollina]

lgtm