Throughput
Below, we measure throughput for four scenarios:
- Read (100% cache hits)
- Read + Write
- Update
- Evict (100% cache miss)
Data was collected on an Intel Xeon W-2133 CPU 3.60GHz, 1 CPU, 12 logical and 6 physical cores, running on .NET 6.0. The MemoryCache used here is from Microsoft.Extensions.Caching.Memory version 6.0.1.
In each test, the cache key and value are integers. This makes key hashing and lookup very cheap. To compute new values we simply store the key, therefore a cache miss is practically zero cost. We therefore measure the throughput of the cache in the absence of any other computation.
In this test, we generate 2000 samples of 500 keys with a Zipfian distribution (s = 0.86). Caches have size 500. From N concurrent threads, fetch the sample keys in sequence (each thread is using the same input keys). The principal scalability limit in concurrent applications is the exclusive resource lock. As the number of threads increases, ConcurrentLru and ConcurrentLfu significantly outperform both MemoryCache and an LRU implemented with a short lived exclusive lock used to synchronize the linked list data structure.
Similar to read, we generate 2000 samples of 500 keys with a Zipfian distribution (s = 0.86). But now caches have size 50, meaning the cache can only hold 10% of all of the values. From N concurrent threads, fetch the sample keys in sequence (each thread is using the same input keys). Some of the requests will be cache misses, and the cache must evict items to preserve bounded size.
