FusionCache dependency injection with configurable service lifetime?

[rjrudman]

Would it be possible to have the service lifetime configuration for AddFusionCache configurable? Currently it's hard-coded to be a singleton:

IFusionCacheBuilder builder = new FusionCacheBuilder(cacheName, services);

if (cacheName == FusionCacheOptions.DefaultCacheName)
{
	services.TryAddSingleton<IFusionCache>(serviceProvider =>
	{
		return builder.Build(serviceProvider);
	});
}
else
{
	services.AddSingleton<LazyNamedCache>(serviceProvider =>
	{
		return new LazyNamedCache(builder.CacheName, () => builder.Build(serviceProvider));
	});
}

We also can't easily create our own extension to do so, as the FusionCacheBuilder is internal.

The reason we'd like this functionality is to be able to instantiate one IMemoryCache per scope - in particular, one per AWS lambda invocation. This is because:

• The execution environment's memory may be shared across invocations
• The execution environment may be paused between invocations

This means that the memory cache may become stale, and we cannot use the backplane to resolve this as the environment may be paused during an update (and additionally, we'd like to skip the backplane anyway to reduce overhead - it is acceptable to have items become stale during a single invocation). We also need to be mindful about memory usage, and the items in the cache between invocations probably do not overlap.

We have a few options at the moment, but none are ideal:

• We could disable the memory cache, but it's not recommended in the documentation, and we'd like to have full stampeding protection.
• We could configure the dependency injection to provide a particular memory cache which we manually wipe at each run - but that's a bit fiddly and extra overhead we'd like to avoid if possible. Also unsure about the implications of doing this
• We could set the memory cache expiry duration to very low, with a longer distributed cache duration, but setting this too low would mean we hit the distributed cache too often, and too high would mean we're over caching entries we don't actually need.
• Our best bet that I can see is probably to manually register the cache ourselves without the AddFusionCache extension: services.AddScoped<IFusionCache>(sp => new FusionCache(new FusionCacheOptions { ... })). Though then we'd need to be sure to keep in sync with the extension method (ie calling AddOptions, AddFusionCacheProvider), as well as handling the named caches

Is this something that could be feasibly implemented? Is there another way to have an isolated IMemoryCache used for a particular scope?

[jodydonetti]

Hi @rjrudman and thanks for using FusionCache!

Would it be possible to have the service lifetime configuration for AddFusionCache configurable?
[...]
The reason we'd like this functionality is to be able to instantiate one IMemoryCache per scope - in particular, one per AWS lambda invocation. This is because:

• The execution environment's memory may be shared across invocations
• The execution environment may be paused between invocations

This is really strange, never heard of this.
I don't know AWS Lambdas, so let me understand better: in that environment a scoped service is not per-request but per-execution-environment, and that can only be used by one request at a time but also reused between different requests?
Did I got it right?

Also, what does it mean "The execution environment may be paused between invocations"? In which sense "paused"? What happens when it is "un-paused"?

Do you have an example of what happen with singleton when pausing/unpausing?

I'd like to know more, and if you can provide some links (blog posts, tutorials, etc) that would be great.

• We could disable the memory cache, but it's not recommended in the documentation, and we'd like to have full stampeding protection.

Agree, try to avoid this.

• We could configure the dependency injection to provide a particular memory cache which we manually wipe at each run - but that's a bit fiddly and extra overhead we'd like to avoid if possible. Also unsure about the implications of doing this

Again, agree: it feels "smelly", even though I would need to look more into the details of this approach.

• We could set the memory cache expiry duration to very low, with a longer distributed cache duration, but setting this too low would mean we hit the distributed cache too often, and too high would mean we're over caching entries we don't actually need.

I was about to suggest low memory duration + higher distributed duration, but first I'd like to understand more about singleton vs scoped with execution pause/unpause.

• Our best bet that I can see is probably to manually register the cache ourselves without the AddFusionCache extension: services.AddScoped<IFusionCache>(sp => new FusionCache(new FusionCacheOptions { ... })). Though then we'd need to be sure to keep in sync with the extension method (ie calling AddOptions, AddFusionCacheProvider), as well as handling the named caches

About named caches, I just released v1.2.0 which also contains support for keyed services 😉

Is this something that could be feasibly implemented? Is there another way to have an isolated IMemoryCache used for a particular scope?

Let me think about it, will let you know, but please give me some more info as said above.

Thanks!

[rjrudman]

Thanks for the quick response!

Would it be possible to have the service lifetime configuration for AddFusionCache configurable?
[...]
The reason we'd like this functionality is to be able to instantiate one IMemoryCache per scope - in particular, one per AWS lambda invocation. This is because:

• The execution environment's memory may be shared across invocations
• The execution environment may be paused between invocations

This is really strange, never heard of this. I don't know AWS Lambdas, so let me understand better: in that environment a scoped service is not per-request but per-execution-environment, and that can only be used by one request at a time but also reused between different requests? Did I got it right?

Sorry, I should have been more clear here. Registering the cache as scoped alleviates the problems caused by lambdas sharing resources - it's not that scoped services are shared.

A super simplistic example that may demonstrate the problem is, we have our lambda implementation:

public class MyLambdaHandler
{
  private IServiceProvider serviceProvider;
  public MyLambdaHandler() 
  {
    // Configure DI
    serviceProvider = services.UseFusionCache().Build();
  }

  public async Task Handle() {
    using (var scope = serviceProvider.CreateScope()) {
      // If IFusionCache is registered as a 'Singleton', we get the same instance every invocation. 
      // We want a new one per scope
      var cache = scope.GetRequiredService<IFusionCache>()
    }
  }
}

And AWS invokes our function like:

var handler = new MyLambdaHandler();
while(!terminate) {
	handler.Handle();
	
	// Process frozen here until next request, and then it continues
}

So here, every time we run scope.GetRequiredService<IFusionCache>, we get the same instance because it's registered as a singleton. And because runs can be far apart, the memory cache is likely to get out of sync.

The details of when an environment is terminated between runs is not publicly available, but usually the environment is used multiple times. The fact that the process is frozen only impacts us not being able to subscribe to a backplane to keep the memory cache in sync. But, even so, we'd still like to not use the backplane in lambdas

Also, what does it mean "The execution environment may be paused between invocations"? In which sense "paused"? What happens when it is "un-paused"?

Do you have an example of what happen with singleton when pausing/unpausing?

I'd like to know more, and if you can provide some links (blog posts, tutorials, etc) that would be great.

I am not familiar with exactly how AWS freezes/thaws environments, but I assume it literally pauses the process. Here's the AWS documentation: https://docs.aws.amazon.com/lambda/latest/dg/lambda-runtime-environment.html - particularly under the 'Shutdown phase' heading (emphasis mine):

After the function and all extensions have completed, Lambda maintains the execution environment for some time in anticipation of another function invocation. However, Lambda terminates execution environments every few hours to allow for runtime updates and maintenance—even for functions that are invoked continuously. You should not assume that the execution environment will persist indefinitely. For more information, see Implementing statelessness in functions.
When the function is invoked again, Lambda thaws the environment for reuse. Reusing the execution environment has the following implications:
Objects declared outside of the function's handler method remain initialized, providing additional optimization when the function is invoked again. For example, if your Lambda function establishes a database connection, instead of reestablishing the connection, the original connection is used in subsequent invocations. We recommend adding logic in your code to check if a connection exists before creating a new one.

Also, the keyed services support looks great!

[jodydonetti]

I'm still in the process of looking at the documentation you linked, but I started thinking about other things like the internals of building a FusionCache instance inside the DI-related methods, or the relation to the handling of multiple named caches, disposale lifestyle, etc.
This just to say that there's a lot of stuff that can go wrong, and in a subtle way...

[rjrudman]

Hi @jodydonetti - just following up on this, wondering if there's been an progress here?

Thinking about it, the situation isn't actually different from having a console application doing this:

while (true)
{
    using var cache = new FusionCache(new FusionCacheOptions());
    
    // Do something with cache
    await Task.Delay(1000);
}        

Does creating (and disposing) multiple caches in this fashion cause any issues?

If not, then perhaps the solution in my situation may be registering it like:

serviceCollection.AddScoped<IFusionCache>(_ => new FusionCache(new FusionCacheOptions()));

or

serviceCollection.AddScoped(_ => new FusionCacheOptions());
serviceCollection.AddScoped<IFusionCache, FusionCache>();

[jodydonetti]

Hi @rjrudman

Thinking about it, the situation isn't actually different from having a console application doing this:

while (true)
{
    using var cache = new FusionCache(new FusionCacheOptions());
    
    // Do something with cache
    await Task.Delay(1000);
}        

Does creating (and disposing) multiple caches in this fashion cause any issues?

Nope, no issues!

I tried running this very code locally and after 5 min this is the situation:

So I would say go on with it.

If not, then perhaps the solution in my situation may be registering it like:

serviceCollection.AddScoped<IFusionCache>(_ => new FusionCache(new FusionCacheOptions()));

I would go with this, it's better to register the service with the IFusionCache interface since it will be more in line with how it isnormally done.
Also, that is in fact how it is done behind the scene, I mean register the interface and pass a lambda that creates a FusionCache instance.

Let me know how it went!