apollo-server-plugin-response-cache: cache hint not working

[cherukumilli]

My apollo graphql server setup uses the following:

    "apollo-datasource-rest": "^0.6.9",
    "apollo-server-express": "^2.9.12",
    "apollo-server-plugin-response-cache": "^0.3.8",

My schema:

type Query {
  types: [Type!]! @cacheControl(maxAge: 3600)
}

type Type {
  id: ID!
  label: String!
  active: Boolean!
}

My responseCachePlugin config:

responseCachePlugin({
    sessionId: (requestContext) => (requestContext.request.http.headers.get('sessionId') || null),
  })

Neither the static cache hint like the above nor the dynamic cache hint in the resolver work. i.e., they don't cache the response from the datasource (REST API).

A bunch of other devs reported this issue on Spectrum. One of the Spectrum posts points to the problem:

The property requestContext.overallCachePolicy doesn't get updated by the cache hint, nither the static one in schema or the dynamic one ...
The value is always the cacheControl you configured when creating ApolloServer

github link to the line in the code where there is a potential problem:

apollo-server/packages/apollo-server-plugin-response-cache/src/ApolloServerPluginResponseCache.ts

Line 243 in 1acf62d

const { response, overallCachePolicy } = requestContext;

Spectrum post that details the above issue:
https://spectrum.chat/apollo/apollo-server/apollo-server-plugin-response-cache-is-not-working-properly~e3390cd4-9e3f-4919-88b4-92508e235e38

Other Spectrum posts that talk about this problem:

1. https://spectrum.chat/apollo/apollo-server/cachecontrol-does-not-work-unless-defaultmaxage-is-set~0e4dc3c5-7cdc-40be-b833-0b2d436d1288
2. https://spectrum.chat/apollo/apollo-server/does-rediscache-work-in-dev-mode~18358268-8d11-4b13-a0c9-abddfb6623f3

cc to: @trevor-scheer

[ghost]

I'm seeing exactly this problem also. I was working on adding response caching today and couldn't figure out why I wasn't seeing any caching happening. I'm glad I'm not going mad but also disappointed that response caching is apparently broken :(

[HofmannZ]

Having the same issue, even without using the apollo-server-plugin-response-cache plugin.

[silentlight]

Having the same issue. Any idea how to solve it?

[ksaldana1]

Not seeing any responses get cached with apollo-server-plugin-response-cache, regardless of what I do with the directives and default config settings. I see the headers being set correctly, but that's pretty much it.

[leonardoanalista]

It's disappointing the cache only works if you set a global caching settings:

  const server = new ApolloServer({
    cacheControl: {
       defaultMaxAge: 50,
    },
    plugins: [responseCachePlugin()],
    cache: new RedisCache({
      host: '127.0.0.1',
      namespace: 'apollo-cache:',
    }),

I am using merge-graphql-schemas with Redis and I am unable to get anything cached without the global settings.

I think it is a great idea to @cacheControl in the schema but it would be even better it it works first.

[elie222]

Have a similar problem. Some queries are cached. Others aren't.

Caching doesn't work when I do this:

cacheControl: {
    defaultMaxAge: 30,
  },

It does when I do:

cacheControl: true,

I've been testing cache hits through https://engine.apollographql.com/. Is there an easier way to do this?

[elie222]

The query that was cached fine doesn't use variables. The ones that didn't cache did. The variables didn't change but could this have caused a cache miss?

[Jylth]

The cache hint is actually working but not like expected.
The Max-Age value seems to be calculated from the smallest value found, including the defaultMaxAge.

So if you have a cache hint of 60 but your default is 10, the cache-control header will be set to 10.

this bit of code explains this behavior

[terryf82]

@Jylth agree that the logic is not what you'd expect, but is technically documented correctly (but could probably be improved):

Apollo Server then combines these hints into an overall cache policy for the response. The maxAge of this policy is the minimum maxAge across all fields in your request.

My initial expectation when testing this kind of setup:

cacheControl: {
    defaultMaxAge: 60
},

type Destination {
    name: String! @cacheControl(maxAge: 10)
    distance: Float!
}

would be for Destination.distance to cache for 60 seconds, and Destination.name to cache for 10. But the result is that the entire Destination type caches for the minimum maxAge across all fields, in this case 10.

[leonardoanalista]

It is documented correctly but the result is not something usable. If you can't set a overall cache, the cache doesn't work at all. This is basically the issue. I wonder if someone from Apollo actively looks at the issues?

[jbarefoot]

This is a bug. The overall cache policy is documented correctly, yes, however the documentation is very specific that the default is that things are uncacheable:

By default, root fields (ie, fields on Query and Mutation) and fields returning object and interface types are considered to have a maxAge of 0 (ie, uncacheable) if they don't have a static or dynamic cache hint.

The current behavior is that @cache-control doesn't work, period, unless you also set a defaultMaxAge. That means you have to explicitly annotate every type you have with @cache-control and then set defaultMaxAge to something enormous, because of the overall cache policy logic.

I will say that the overall design approach here is fantastic! This seems like it should be an easy thing to fix frankly.

[leonardoanalista]

Absolutely, I couldn't agree more. It's designed perfectly and we hope it gets fixed. I'll send this thread to the Apollo support team in several channels. Hopefully we get an answer soon.

[glasser]

defaultMaxAge is only supposed to be applied to root fields and object/interface fields (based on my memory and code inspection). The example in the initial issue above correctly applies a hint to the only root field which is also the only object field. So this should work and is a bug if it doesn't.

[glasser]

I implemented apollo-server-plugin-response-cache but my current focus isn't apollo-server. I am happy to look into and likely fix this if somebody can provide a full reproduction recipe: ie, a series of commands probably starting with git clone which allows me to see the bug without having to try to set up a server from scratch and guess all the details of what you tried. (For example, this should include a sample query and some way to tell that it's not cached (perhaps a resolver that returns random values).)

[jbarefoot]

@glasser That's much appreciated, I'll try to find some time soon to create a basic repo that illustrates the issue. In the meantime, I implemented a workaround which I'll post here for others. The workaround is to use a wrapper function around every resolver function, that checks for a cache hint and adds it if it doesn't exist. As long as you remember to use the wrapper on every query resolver, it works fine. Then you set defaultMaxAge to something really high, like a day, so it will never be the minimum in the overall computed cache policy.

const defaultExpiry = 60  // in seconds
const defaultCacheHint = { maxAge: defaultExpiry, scope: 'PUBLIC' };

async function resolverWrapper(info, func) {
  console.log("Cache hint already set to:", info.cacheControl.cacheHint)

  // The hack here is that if cache-control has NOT been set for a type/field/whatever, then you will get { maxAge: 0 } for info.cacheControl.cacheHint i.e. uncached
  // If cache-control HAS been set, then you get { maxAge: <someInt>,  scope: undefined }, i.e. there will be a "scope" key
  // So only set default expiry/TTL if the "scope" key does NOT exist. (Checking for whether the cacheHint is defined is just being defensive)
  if(!info.cacheControl.cacheHint || !("scope" in info.cacheControl.cacheHint)) {
    console.log("Cache hint was NOT set so defaulting to:", defaultCacheHint)
    info.cacheControl.setCacheHint(defaultCacheHint);
  }
  return func()
}

// Use it from a resolver like this:
loadSomeThings: async (_, {fromDate, toDate}, __, info) => resolverWrapper(info, async () => {
  const sql = `select foo, bar, baz, start_date from some_table where start_date between '${fromDate}' and '${toDate}'`
  return await runMySQLQuery( sql )
}),

[glasser]

@jbarefoot How is that better than setting defaultMaxAge to 60?

[jbarefoot]

@glasser It means you can have a short default expiry and override with longer values. The code I posted only sets the expiry to 60 if @cache-control has NOT been set. Setting defaultMaxAge to 60 means if you have e.g. @cache-control(300), it will get ignored and defaultMaxAge of 60 used instead, owing to the "min expiry wins" overall cache policy behavior. The current behavior of defaultMaxAge makes it impossible to set a very short default expiry with selectively longer expiry for objects/fields that you know are safe to cache longer. Frankly I disagree with this, if anything in the query graph has @cache-control set, IMO that should be used and defaultMaxAge ignored. My general expectation is that more granular settings override more general/larger scope (though I'm not sure I would apply that to type vs field @cache-control usage here, I would have to think about it more).

In my experience this is a common way to use a server cache: The default expiry is pretty short or perhaps not cached (whatever is appropriate for the context). For data that is slowly changing, set the expiry case-by-base to longer values.

[glasser]

That is not my understanding of how defaultMaxAge works (though of course, there may be a bug, that's why we're here).

If you set defaultMaxAge to 5 but have a schema like

type Query  {
  someField: Foo @cacheControl(maxAge: 50)
  someOtherField: Bar
}

Then queries against someField will be cached for 50, queries against someOtherField will be cached for 5, and queries that hit both fields will also be cached for 5 because they contain some low-TTL data. Is that not what you're experiencing?

The main feature that I think is missing here (other than features that would make this model more comprehensible/inspectable) is the ability to mark an object/interface field as "inherit parent maxAge" just like scalar fields do.

But I don't understand how your example differs from your example without setting defaultMaxAge (unless this is what helps us narrow down what the bug is!). I am really looking forward to the clear example of this bug given by somebody on this thread!

[jbarefoot]

@glasser Thanks for the reply. I wasn't testing field level caching, so I'm not sure on that. I will try to test it later in my schema and let you know the result. My object example would be like this:

type User @cache-control(maxAge:50)  {
  someField: Foo
  someOtherField: Bar
}

In the above case, with defaultMaxAge=5, if you fetch User by id, it will be cached for 5 seconds, not 50.

the ability to mark an object/interface field as "inherit parent maxAge" just like scalar fields do.

Really I just want it to not use defaultMaxAge, at all, if a cache hint has been set, either via @cache-control or dynamically. To me default means default, i.e. use this if there is no other value. Instead the way it works in my testing is, override cache hint value with defaultMaxAge if it is less than what is currently computed, i.e. defaultMaxAge is the "uber parent" of the object graph. I think that's what is throwing people for a loop here.

And sorry yes let me try to get you an example git repo later this week, I know this is difficult without seeing a concrete thing that doesn't work as expected. :)

[matikucharski]

Then queries against someField will be cached for 50, queries against someOtherField will be cached for 5, and queries that hit both fields will also be cached for 5 because they contain some low-TTL data. Is that not what you're experiencing?

It is how it should work but now someField query is cached also for 5 sec - not for 50 sec. Just tested it.

---

Btw it would be also great to have ability to set default scope on all queries to PRIVATE.
In my app they all should be 'PRIVATE' so I need to mark each individually to be 'PRIVATE' instead of setting it once eg like this:

cacheControl: {
  defaultMaxAge: 5,
  defaultScope: 'PRIVATE'
}

[eetvalve]

My solution (for now) was to make custom @CacheControl decorator (with type-graphql):

import { UseMiddleware } from 'type-graphql';

export function CacheControl(hint: CacheHint) {
  return UseMiddleware(({ context, info }, next) => {
    console.log('Called CacheControl');

    context.cacheOptions = {
        ttl: hint.maxAge,
    };
  
  // replace with setCacheHint when it is working properly again
  // info.cacheControl.setCacheHint(hint);
    return next();
  });
}

@CacheControl({ maxAge: 60 })

Obviously scope: private/public is missing from context.cacheOptions but many times setting ttl is enough.

Tested only with type-graphql -module

[chrislambe]

This was very confusing. I expected to be able to override defaultMaxAge completely (not just providing a lesser value). It's not clear from the documentation that this is the case.

[glasser]

I still do not understand what the complaint is in this bug and will launch into fixing it as soon as somebody presents a self-contained reproduction!

[terryf82]

@glasser I'm not sure exactly what you need for a self-contained reproduction, but I feel my initial comment on expected vs actual behaviour describes the issue most here are experiencing:

My initial expectation when testing this kind of setup:

cacheControl: {
    defaultMaxAge: 60
},

type Destination {
    name: String! @cacheControl(maxAge: 10)
    distance: Float!
}

would be for Destination.distance to cache for 60 seconds, and Destination.name to cache for 10. But the result is that the entire Destination type caches for the minimum maxAge across all fields, in this case 10.

If I can offer anything further that might help, please let me know.

[glasser]

As I said above:

a full reproduction recipe: ie, a series of commands probably starting with git clone which allows me to see the bug without having to try to set up a server from scratch and guess all the details of what you tried. (For example, this should include a sample query and some way to tell that it's not cached (perhaps a resolver that returns random values).)

[terryf82]

@glasser my example is unlikely to be useful to you then, we're using the neo4j-graphql-js package which requires no resolvers to be written, just types defined.

The amount of interest in this thread clearly indicates a problem though, so hopefully someone can move this forward by supplying what you need.

[mattvb91]

Came across the same issue here that my schema cache hints were not being set and no cache headers sent out at all. Fixed it with the suggestion above of defining a defaultMaxAge

 cacheControl: {
    defaultMaxAge: 60
},

This is more of a hack than anything to have any sort of caching headers sent out but will surely cause issues down the line

[sanket-work]

Facing similar issue,
when I use caching at the Apollo server level then everything is cached I can see no DB queries in log after the first request. Caching seems to be working

 cacheControl: {
    defaultMaxAge: 60
},

But when I try the fine grain caching at resolvers I can see the maxAge set in HTTP response but the response doesn't seem to be cached as I can see DB queries on each request

const resolvers = {
  Query: {
    post: (_, { id }, _, info) => {
      info.cacheControl.setCacheHint({ maxAge: 60, scope: 'PUBLIC' });
      return find(posts, { id });// some db call
    }
  }
}

Am I doing anything wrong here?

[glasser]

Please provide a full executable reproduction example, as described in my previous comments on this issue.

[SachaG]

@glasser I might be wrong but I get the impression this is maybe more an issue with unclear documentation than an actual bug? I haven't had time to revisit this issue but now I just learned about datasources and I feel like that's one more caching parameter I need to juggle with… I think a comprehensive blog post or example app that clearly outlines the different caching methods available with Apollo Server and the pros and cons of each one would be helpful here.

[SachaG]

To follow up on my last post, I only just now finally understood that this is about setting cache hints so that Apollo can figure out whether the entire request is cacheable or not; and not telling Apollo to cache individual fields.

In other words, if I have 9 cached fields and 1 uncacheable fields in my request I falsely expected after reading the documentation that those 9 fields would be cached and not trigger database, API, etc. requests. Whereas the truth (as I now understand it, I might still be wrong?) is that none of my 10 fields will be cached because the entire request is not cacheable.

On the other hand, if I used Datasources for those 9 fields then those database, API, etc. requests would be cached independently of whether the full GraphQL can be cached or not.

Again sorry if I'm wrong and just making things more confusing, but I suspect other people might've misunderstood this as well.

[glasser]

@SachaG So there's two separate things.

Apollo Server has a flexible way of assigning cache hints on a per-field basis.

AS then uses that data to power several features. Right now, the only features that makes use of this data are full-response features, such as the full response cache (https://www.apollographql.com/docs/apollo-server/performance/caching/#saving-full-responses-to-a-cache) and HTTP response header calculations.

One could imagine building out a partial response cache as well! In a lot of cases this will require a way for the server to know how to "jump" deep into a query to re-query just the changing data. We have not built that and I do not believe it is on any particular roadmap, but it is an imaginable way to make use of the detailed per-field cache data.

Datasources is basically unrelated. It's a fancy word for "we built a caching HTTP client in Node, maybe that's useful for you when writing your resolvers". Also note that there are two different things called "data sources" in AS (the caching HTTP client thing and the interface that Apollo Gateway uses to talk to implementing services) that are also unrelated.

[SachaG]

One could imagine building out a partial response cache as well! In a lot of cases this will require a way for the server to know how to "jump" deep into a query to re-query just the changing data. We have not built that and I do not believe it is on any particular roadmap, but it is an imaginable way to make use of the detailed per-field cache data.

Yes, I saw this issue about that: #4042

Datasources is basically unrelated. It's a fancy word for "we built a caching HTTP client in Node, maybe that's useful for you when writing your resolvers". Also note that there are two different things called "data sources" in AS (the caching HTTP client thing and the interface that Apollo Gateway uses to talk to implementing services) that are also unrelated.

Good to know, thanks for the added details!

[aonamrata]

Hi @SachaG

About this part 👇

On the other hand, if I used Datasources for those 9 fields then those database, API, etc. requests would be cached independently of whether the full GraphQL can be cached or not.

Do you know how I can implement this or what package are you referring to? I am using REST datasource so if I can do this, then I don't need the field/type level cache control. This would behave the same for 90% of my cases. FYI, not ttl as that is only for a request so not really useful across requests.

[SachaG]

Hi @SachaG

About this part 👇

On the other hand, if I used Datasources for those 9 fields then those database, API, etc. requests would be cached independently of whether the full GraphQL can be cached or not.

Do you know how I can implement this or what package are you referring to? I am using REST datasource so if I can do this, then I don't need the field/type level cache control. This would behave the same for 90% of my cases. FYI, not ttl as that is only for a request so not really useful across requests.

I don't know any more than what's in the docs and what @glasser mentioned, sorry. I haven't implemented them myself yet.

[avatsaev]

I only want to be able to cache things that have explicit cache hints, without defining the defaultMaxAge.
cache hints i define in resolvers don't work without defaultMaxAge, and when I define it, apollo server automatically caches things I don't want it to cache (even without hints) which results in unexpected behaviors on the front end.

How do I only cache things that have explicit hints, and disable automatic root object caching? It is not possible to do that?

[mattvb91]

Applying this diff to your repo and running your curl command, the header is set to 100.

diff --git a/index.js b/index.js
index 4533fd6..eda16c5 100644
--- a/index.js
+++ b/index.js
@@ -1,7 +1,7 @@
 const { ApolloServer, gql } = require("apollo-server");
 
 const typeDefs = gql`
-  type Book { #@cacheControl(maxAge: 100)
+  type Book @cacheControl(maxAge: 100) {
     title: String
     author: String
   }
@@ -31,10 +31,6 @@ const resolvers = {
 const server = new ApolloServer({
   typeDefs,
   resolvers,
-  cacheControl: {
-    defaultMaxAge: 1000,
-  },
-  // cacheControl: true,
 });
 
 // The `listen` method launches a web server.

The point is that passing true for cacheControl enables a backwards-compatibility mode that you don't want, but just leaving it out (or specifying an object with other options in it) gives the result you want.

Definitely a lesson here about the naming of boolean options: we should have been more thoughtful and named it includeCacheControlExtension or cacheControl: {includeExtension}.

@glasser I have played around with this sample now and it does indeed work with your diff. However as soon as I make the following change it stops working and I think thats where the confusion comes from for a lot of people:

const typeDefs = gql`
  type Book @cacheControl(maxAge: 100) { 
    title: String
    author: String
    nestedObject: Object     #<--- this nestedObject will break it 
  }

  type Query {
    books: [Book]
  }

  type Object {
    test: Boolean
  }
`;

const books = [
  {
    title: "Harry Potter and the Chamber of Secrets",
    author: "J.K. Rowling",
    nestedObject: {
      test: true
    }
  },
  {
    title: "Jurassic Park",
    author: "Michael Crichton",
    nestedObject: {
      test: true
    }
  },
];

My assumption would be that the Book type would return maxAge of 100 in that case. However that does not happen until I also add @cacheControl(maxAge: 100) to my Object type:

  type Object @cacheControl(maxAge: 100)  {
    test: Boolean
  }

Only then does the cache-control: max-age=100, public get sent for Book. Can you confirm this is expected behaviour?

Edit:

Ok this is pretty embarrasing but here we go...
I was testing inside the GraphQL playground the whole time and viewing the requests through dev console network tab... Now I had assumed that the playground would automatically set the correct header for me which is:
{ "Accept": "application/json", "Content-Type": "application/json" }

Once I have that in my headers in GraphQL Playground everything seems to be working fine... Only noticed this when curling the same queries and seeing a difference in headers..

I suggest everyone recheck their responses through curl and compare.

Edit 2: Opening the playground in incognito seems to send the correct caching headers. Not sure how I managed to get it into such a weird state. So looks like caching is indeed working correctly for me now

Final Edit: Back to square one no idea why or how its sometimes working and sometimes not. As soon as I change my structure slightly it stops working. Need to investigate further, there is some structure change that breaks the caching

[mattvb91]

@glasser I have a repo with reproduction for you: https://github.com/mattvb91/apollo-cache-issue-reproduction

My assumption is caching will not work top level unless all sub nested objects have a caching hint on them.

[glasser]

@mattvb91 I'll dig in on your reproduction, but I do think you're observing documented behavior. https://www.apollographql.com/docs/apollo-server/performance/caching/#setting-a-default-maxage says:

By default, root fields (ie, fields on Query and Mutation) and fields returning object and interface types are considered to have a maxAge of 0 (ie, uncacheable) if they don't have a static or dynamic cache hint.

nestedObject is a field returning an object type is its default maxAge is 0.

Whether this design for @cacheControl's defaults is a good idea is a different question, but I'm pretty sure this is working exactly as intended?

[glasser]

Yes, looking at your reproduction, this is working as intended. The theory when designing this feature was that object fields are more likely to be "looking up a different thing with different cacheability" than scalar fields. I'm not sure that this area of the product is under enough active development for it to be likely to change the semantics at this point.

[mattvb91]

Ah i think i may have gotten that confused then in thinking the removal of defaultMaxAge would allow the max age to be taken from the parent then.

Yea it does seem to be more a discussion of wether that is a good default behaviour. Thanks

[macrozone]

i also struggle with this problem.

We want to define some named root queries that should be cached (the full response), but the default should be no-cache.

There is no combination of setting the defaultMaxAge and individual cache hints that seem to be correct. defaultMaxAge in fact is the only thing that seems to have an impact.

If someone could point me int the right direction, i would be very glad.

[macrozone]

I also noticed that only the first query has the right cacheHint:

• i define a defaultMaxAge of 3600 (without it, it does not work at all)
• on a field i add a cachehint of 60 (1min)
• i run the query twice and inspect both the headers and whether my db request is performed)

first request responses with cache-control: max-age=60, public
second request responses with cache-control: max-age=3600, public

it seems like apollo server caches the cache hints as well!

Edit:

its clear. Not setting defaultMaxAge --> caching DOES NOT WORK AT ALL
you have to set defaultMaxAge to something huge, because it defines the maximum caching time, not the default caching time, when no other cache hint is available.

But setting cacheHints dynamically for every resolver also does not work properly. As mentioned above it only works shortly. After further investigation it seems that it works for the given cachehint time, after that it falls back to the global setting.

Something is totally wrong in the cache logic and its not only the flawed design of defaultMaxAge

Edit2: its really confusing to debug.

But it seems clear that resolver-level cache hints get lost when it tries to use the cache.

Edit3: it had something to do how we set cachehints dynamically, i think i solved it.

But the one flaw remains: defaultMaxAge is the maximum TTL that you can set. if you omit it, no resolver will get cached

[ankuradhey]

I tried without defaultMaxAge and caching is working for me. I noticed that if we are not using cache control hints properly then it looks like it is not working. Make sure that cache hint is applied to a field and its nested non scalar fields as well then only caching will work. Otherwise when it finds nested non scalar field is missing cache control so it tries to use defaultMaxAge and which is 0 by default. Hence, it looks like caching is not working.

Some points to be noted

• Resolver level caching policy takes higher precedence than schema-object level
• Lowest cache expiration takes highest precedence
• Responses containing GraphQL errors or no data are never cached

[timsuchanek]

In case you want to save the effort of setting up Redis, getting the config working etc,
you may wanna have a look at https://graphcdn.io.
I'm building that right now, especially to make caching easier, as I struggled with it myself before.

[koborg]

I tried without defaultMaxAge and caching is working for me. I noticed that if we are not using cache control hints properly then it looks like it is not working. Make sure that cache hint is applied to a field and its nested non scalar fields as well then only caching will work. Otherwise when it finds nested non scalar field is missing cache control so it tries to use defaultMaxAge and which is 0 by default. Hence, it looks like caching is not working.

Some points to be noted

• Resolver level caching policy takes higher precedence than schema-object level
• Lowest cache expiration takes highest precedence
• Responses containing GraphQL errors or no data are never cached

HUGE THANKS ankuradhey!!!
Your comment lightened up my implementation!
So... if we want defaultMaxAge: 0, than we need to explicitly define the 'cacheControl(maxAge: XXX)' directive to all non scalar fields of the respected query or mutation. Once I did that, the all the fields marked with the directive started to respect the the defined maxAge and all the other fields (non marked) remained with 0 caching time.

[glasser]

Here's an implementation of something that's similar to the inherit concept I discussed above: #5247
I'm interested in feedback from folks who were interested in this issue!

[glasser]

I think the main concern in this issue is addressed in #5247.