Variable length values

FASTER supports variable-length values. The exact nature of the support depends on whether you are using the C++ or C# version of FASTER. These are described next.

In C++

If you update (i.e., Upsert() or Rmw()) an existing variable-length value to a new value that's no larger than the existing value, FASTER will update the value in place. Otherwise, FASTER will mark the existing value as read-only, and perform a copy-on-write/read-copy-update to a new value, of the appropriate size. See example unit tests "InMemFaster / UpsertRead_ResizeValue_Concurrent" and "InMemFaster / Rmw_ResizeValue_Concurrent".

To create a variable-length value, your Upsert or RMW context should:

1. Define a value_size() method that returns the actual size of your value. You should store this size inside your value's header, but you can define this however you like. In the example unit tests, value_size() returns "sizeof(value_t) + length_", where length_ is a 4-byte integer stored in value_t's header.
2. Define its PutAtomic() / RmwAtomic() lambda to return "false" when the value cannot be updated in place (e.g, because the update requires more space), and also mark the record as read-only or "replaced." The lambda should also return false if some other thread previously marked the record as read-only/"replaced."

When the FASTER KV store sees PutAtomic() / RmwAtomic() return false, it retries the operation as a copy-on-write/read-copy-update.

If your variable-length value never changes in size, you can just define an appropriate value_size() method, and have your PutAtomic() / RmwAtomic() always return "true." The value_size() method is what FASTER uses to allocate space for a record, for initial updates and copy-on-write/read-copy-update.

In C#

C# types can either be blittable or non-blittable. All blittable types are structs (or native types like int), but not all structs are blittable. C# class types are non-blittable. Blittable structs are fixed-size by definition, and are supported by FASTER in C# by default at the highest level of performance. Non-blittable types can be variable-sized (e.g., strings) and are stored on the heap in C#. In order to support non-blittable types with secondary storage, the C# version of FASTER uses a special storage mechanism called an object log, described next.

FASTER's hybrid log in memory, for non-blittable keys and/or values of type T, contains a logical pointer to a paged in-memory array of type T. Operations on objects in the mutable region use the logical pointer to dereference the object and perform operations on it. In particular, if the object size changes (e.g., we modify a string), we would replace the array entry with the new object of the larger size. The .NET GC naturally recycles the older object.

When the object entry in the hybrid log reaches the read-only region and is ready to be flushed to disk, the objects are serialized and written to a separate log file called the object log. The main hybrid log page, just before flushing to disk, is also updated to point to the appropriate location on this object log (instead of to the array). If we later need to read this key-value pair, we load the hybrid log entry as usual, and use the object log address to retrieve the object from storage and deserialize it to main memory as a C# object.

Note that performance will be lower if you are using non-blittable objects, due to the (space and access time) overheads for such objects. An example of using C# class objects with FASTER is present at FASTER/cs/playground/ManagedSample3/.