State group caches keep full state for too long, bloating the cache size.

[erikjohnston]

We have a complicated set up for caching state look ups to handle both filtered and full state look ups. However, the eviction strategy is sub-optimal: we keep all cached values for a given state group until we stop querying for any keys in that state group.

This means that if we pull out and cache the full state for a state group, we will keep the full state in the cache even if we only query one type / state key pair periodically.

Ideally the cache would see that nothing has asked for the full state in a while and evict it.

[erikjohnston]

Perhaps the solution here is to have the dictionary cache backed by two separate LruCaches. One for specific keys, and one for the "full" dict. When we ask for specific keys we hit the first cache, if its not there we hit the second (without updating the access time!) and add any result back into the first cache.

This should allow the cached "full" dict to be expired if we don't need the full dict.