Cache file in local file system

[waynexia]

Target

Provides a local file system cache layer over object store

Design

Add an ObjectStore implementor CachedStore, which is made up of:

• Local Store: a local file system cache
• Remote Store: a remote OSS service
• LRU Manager: a cache eviction policy (LRU) to keep the local store size under control.

Read

On serving a read request, the CachedStore will first check if the object is in the local store. If not, it will fetch it from the remote store and store it in the local store. To simplify the implementation, all the read sources returned are based on LocalStore. Ascii art below:

Workflow

(read)
       Serve I/O requests
              ▲ │
            4 │ │1
              │ │
   ┌──────────┴─▼─────────┐
   │                      │
   │  Object Store Cache  │
   │                      │
   └─┬─▲────────────┬─▲───┘
  2.1│ │3.1      2.2│ │3.2
     │ │            │ │
 ┌───▼─┴──┐      ┌──▼─┴─────┐
 │Local FS│      │Remote OSS│
 └────────┘      └──────────┘

Write

For write requests, we will write the content to both underlying stores.

Restart

To suit some deploy scenarios that aren't stateless, ObjectStore will try to load all existing entries from LocalStore to LRU Manager.

Purge

Both read and write operations may trigger purge on LocalStore. The purge policy is defined by the LRU Manager. It will get a delete list from LRU Manager and delete the list from LocalStore.

To ensure the total size of LocalStore is always less than the threshold, CachedStore will first purge enough space for the incoming new objects.

Other features

Config

Except for the basic configurations like the underlying provider of LocalStore and RemoteStore, threshold of LocalStore etc., we can also configure other behaviors like whether to cache write request, how to act on restart or cache evict policy.

Async Cache

It is doable to feed the data from RemoteStore directly to the user and write LocalStore asynchronously.