[C#] Add ConditionalCopyToTail (#807)

* Locking fixes
- CompletePending* must also check to lock non-auxiliary records
- Ephemeral locking must look for lock evictions as well
- InternalLock must check for SpinWaitUntilRecordIsClosed
- Replace oneMiss handling in InternalLock
- EphemeralSUnlock must follow unlock failures
- LockTable should have its own bufferPool
- Ensure OnPagesClosed does eviction scans in address order

* More locking fixes:
- Fix missing unlock of a Tombstoned record in InternalContinuePendingRead
- Add more Asserts where ManualLocking requires a locked record
- Update and remove obsolete comments

* - Remove key-present optimization for SpinWaitUntilRecordIsClosed
- Better checking on following records transferred from LockTable in InternalLock(unlock)
- Add some additional comments

* Fix hash bucket overflow pointer during recovery
Fix debug assert logic

* WIP on locktable implemented via mainhash overflow buckets

* unstaged files missed in previous push

* Updates to EphemeralOnly locking for transfer-from-RO/RC

* Change LockTableTests to OverflowBucketlockTableTests

* Remove RecordInfo.Tentative, .IsIntermediate*

* More locking WIP: session.NeedKeyLockCode, test cleanup

* WIP locking: Tests build

* WIP on mainhash locking:
- Add KeyCode creation and sorting and methods public and internal
- Remove unnecessary IsLocked (it does not make work with EphemeralOnly and SessionControlled may map to multiple keys)
- More tests run now

* Most UTs run

* More UTs succeed

* Still some UT failures

* - use XLocks as Tentative indicator for insertions/splices at log tail and readcache insertions
- improve readcache consistency checking
- clear HasInMemoryLock when HeadAddress goes above srcRecord address following BlockAllocate
- rename LockingMode.SessionControlled -> Mixed and other renaming for clarity
- remove some unused methods
- fix locking sequences in InternalContinuePendingR(ead|MW)

* More fixes
- Missing 'ref' on recordInfo var in FindAndUnlockTransferredRecord
- Improve record allocation checking for below-HA in-memory source and RecordInfo lock clearing if so
- Add LockCode to HashEntryInfo in DEBUG

* Remove SpinWaitUntilRecordIsClosed; it was only used in ReadCache and in the relevant case we just need to wait for ReadCacheEvict to complete.

* - MixedModeTentativeOrClosed is not needed
- Add/Improve some ToString()s
- Remove unused RecSrc.ClearSrc()
- Rename to HasRecordInfoLock
- Move ICPRead HandleImmediateRetryStatus outside the lock scope
- Consoldiate EphemeralSUnlock(AfterPendingIO)

* Streamline locking and FindInMemory calls in InternalXxx

* - Change names to pendingContext.Initial*Address and improve setting of these in InternalXxx and handling of these in CompletePending*() and TryFindRecordInMemory.
    - Also consistentize related param names for readcache and log search to minAddress.

* Clean up pending RMW call to CreateNewRecordRMW, per prior changes that make the inputSrc unnecessary.

* - Fix stale readcache assignment to recSrc.Log
- make searches after pendingIO completion non-inclusive of minAddress
- Some ToString() and naming changes for consistency
- Remove RecordInfo.IsValidUpdateOrLockSource (superseded by direct call to .IsClosed)

* Allow length specification on KeyCode array operations

* - Fix check for lock having escaped to eviction zone following BlockAllocate
- Add some comments for SafeHeadAddress et al.

* - Make sure the readcache bit is set when needed on addresses put into stackCtx.newLogicalAddress.
- Rename to HandleNewRecordOnException for clarity

* Update remote project

* - Make sure FindAndUnlockTransferredRecord can go to SafeHeadAddress
- Add non-ref overload of GetLockCode

* Tweaks for benchmark with Mixed locking

* Add ILockableKey.CompareLockCodes

* Change LockCode to just be hashcode, since we may grow the hash table.

* - Make CPR consistency check work without conflict with the key latches in the HashBuckets.
- Change some names for clarity.
- Pending read completion should return values from any in-memory record, as they are the latest

* Remove duplicate opCtx param and removed sessionCtx (aka currentCtx, aka ctx) from calls where FasterSession was also passed (in favor of (new) FasterSession.SessionCtx), where the called method has a type constraint of "where FasterSession : IFasterSession<Key, Value, Input, Output, Context>". Not yet done: converting "whereFasterSession : IFasterSession" constraints to generic to remove the duplicate ctx parameter.

* Fix DoLockOp to compare bucket indexes, not KeyCodes

* Finish removal of duplicate sessionCtx param from calls where FasterSession was also passed (in favor of (new) FasterSession.SessionCtx), where the called method has a type constraint of "where FasterSession : IFasterSession" (non-generic). This cannot be done in Synchronization (the state machines) due to their use of NullFasterSession.

* Fix key duplication in readcache and log:
- VerifyInMemoryAddresses should not refresh hei if no splice point is present
- FindRecordInMemory checks pendingContext.InitialHash
- Move CompleteCopyToTail in ITCTT into the non-readcache section
- TryFindRecordInMemory needs to check pc.InitialEntryIndex's readcache bit
- Updaters need to scan down to hlog.ReadOnlyAddress to ensure the "key is not at or below LatestLogicalAddress" logic holds
- ReadCacheCheckTailAfterSplice needs to check hei.*Address readcache bit

* Replace use of SafeHeadAddress with SpinWaitUntilClosed in FindInReadCache

* Remove LockingMode.EphemeralOnly
add --hashpack arg to FASTER.benchmark

* Tweaks for the Standard locking change

* fix remote to use LockingMode.Standard

* Add DisposeTest comments and more Asserts

* Use calls on RecordSource to make it easier to verify correct matching of log and physicalAddress

* Temporarily disable LocalMemoryDevice tests for investigation

* test removal of DisposeTests

* - Fix for CTT to address SameKeyInsertAndCTTTest failure
- Add missing arg parsing for MemOnlyCache
- Re-enable DeviceType.LocalMemory in tests

* WIP on ConditionalInsert

* [C#] MemOnlyCache sample: improved size tracking, including read cache (#800)

* Improved cache size tracking, including read cache

* Rename MemOnlyCache to ResizableCacheStore.

* update doc

* fixes

* Updates

* ClearBitsForDiskImages should unseal record as well (Q: why was it sealed to begin with?)

* Correct clearing of index bits during recovery

* Remove RecordInfo sealing

* WIP Ephemeral locking at IFunctions-level only, part 1

* Remove unused SessionType enumeration; consolidate duplicate IFasterSession implementations

* - Add EphemeralLocking  readcache detach/reattach logic
- Remove unused ReadFlags.ResetModifiedBit

* Rename a couple files

* re-add RecordInfo.Seal for Ephemeral locking; remove  unused (RMW|Upsert)Action.NeedMoreSpace (this is the default 'false' behavior)

* Remove single-key overloads of Lock() and Unlock()

* - Update a too-aggressive assert.
- CreateLogDevice errors on Windows if path > MAX_PATH minus room for segment#.

* Improve previous MAX_PATH fix

* Chnge Read of an expired to just return NOTFOUND | Expired rather than tombstoning

* - Replace ReadFlags with ReadCopyOptions
- More WIP on ConditionalInsert

* RETRY_LATER rather than trying to re-find splice point directly in VerifyInMemoryAddresses

* More WIP on ConditionalInsert

* WIP on ConditionalCopyToTail

* change filename

* - Fix Compact/ConditionalCopyToTail
- Make TryCopyToReadCache bool
- Tighten up DetachAndReattachReadCacheChain
- Other minor tweaks

* fix merge

---------

cs/samples/ReadAddress/VersionedReadApp.cs

@@ -151,7 +151,7 @@ private static void ScanStore(FasterKV<Key, Value> store, int keyValue)
             var output = default(Value);
             var input = default(Value);
             var key = new Key(keyValue);
-            ReadOptions readOptions = new() { ReadFlags = ReadFlags.DisableReadCache};
+            ReadOptions readOptions = new() { CopyOptions = ReadCopyOptions.None };
             for (int lap = 9; /* tested in loop */; --lap)
             {
                 var status = session.Read(ref key, ref input, ref output, ref readOptions, out var recordMetadata, serialNo: maxLap + 1);
@@ -185,7 +185,7 @@ private static async Task ScanStoreAsync(FasterKV<Key, Value> store, int keyValu
             var input = default(Value);
             var key = new Key(keyValue);
             RecordMetadata recordMetadata = default;
-            ReadOptions readOptions = new() { ReadFlags = ReadFlags.DisableReadCache };
+            ReadOptions readOptions = new() { CopyOptions = ReadCopyOptions.None };
             for (int lap = 9; /* tested in loop */; --lap)
             {
                 var readAsyncResult = await session.ReadAsync(ref key, ref input, ref readOptions, default, serialNo: maxLap + 1, cancellationToken: cancellationToken);

cs/samples/ResizableCacheStore/Program.cs

@@ -342,7 +342,7 @@ static void Main(string[] args)
             {
                 LogDevice = log, ObjectLogDevice = objectLog,
                 MutableFraction = 0.9, // 10% of memory log is "read-only region"
-                ReadFlags = UseReadCTT ? ReadFlags.CopyReadsToTail : ReadFlags.None, // whether reads in read-only region are copied to tail
+                ReadCopyOptions = UseReadCTT ? new(ReadCopyFrom.AllImmutable, ReadCopyTo.MainLog) : ReadCopyOptions.None, // whether reads in read-only region are copied to tail
                 PageSizeBits = PageSizeBits,
                 MemorySizeBits = MemorySizeBits
             };

cs/src/core/Async/ReadAsync.cs

@@ -105,9 +105,7 @@ internal ReadAsyncResult(FasterKV<Key, Value> fasterKV, IFasterSession<Key, Valu
         internal ValueTask<ReadAsyncResult<Input, Output, Context>> ReadAsync<Input, Output, Context>(IFasterSession<Key, Value, Input, Output, Context> fasterSession,
             ref Key key, ref Input input, ref ReadOptions readOptions, Context context, long serialNo, CancellationToken token, bool noKey = false)
         {
-            var pcontext = new PendingContext<Input, Output, Context> { IsAsync = true };
-            var operationFlags = PendingContext<Input, Output, Context>.GetOperationFlags(MergeReadFlags(fasterSession.Ctx.ReadFlags, readOptions.ReadFlags), noKey);
-            pcontext.SetOperationFlags(operationFlags, readOptions.StopAddress);
+            var pcontext = new PendingContext<Input, Output, Context>(fasterSession.Ctx.ReadCopyOptions, ref readOptions, isAsync: true, noKey: true);
             var diskRequest = default(AsyncIOContext<Key, Value>);
 
             fasterSession.UnsafeResumeThread();

cs/src/core/ClientSession/ClientSession.cs

@@ -663,10 +663,8 @@ internal bool UnsafeCompletePending<FasterSession>(FasterSession fasterSession,
             var result = fht.InternalCompletePending(fasterSession, wait, requestedOutputs);
             if (spinWaitForCommit)
             {
-                if (wait != true)
-                {
+                if (!wait)
                     throw new FasterException("Can spin-wait for commit (checkpoint completion) only if wait is true");
-                }
                 do
                 {
                     fht.InternalCompletePending(fasterSession, wait, requestedOutputs);
@@ -861,14 +859,13 @@ public long Compact<CompactionFunctions>(ref Input input, ref Output output, lon
         /// <param name="output"></param>
         /// <param name="desiredValue"></param>
         /// <param name="untilAddress">Lower-bound address (addresses are searched from tail (high) to head (low); do not search for "future records" earlier than this)</param>
-        /// <param name="actualAddress">Actual address of existing key record</param>
         [MethodImpl(MethodImplOptions.AggressiveInlining)]
-        internal OperationStatus CompactionCopyToTail(ref Key key, ref Input input, ref Value desiredValue, ref Output output, long untilAddress, long actualAddress)
+        internal Status CompactionCopyToTail(ref Key key, ref Input input, ref Value desiredValue, ref Output output, long untilAddress)
         {
             UnsafeResumeThread();
             try
             {
-                return fht.InternalCopyToTailForCompaction<Input, Output, Context, InternalFasterSession>(ref key, ref input, ref desiredValue, ref output, untilAddress, actualAddress, FasterSession);
+                return fht.CompactionConditionalCopyToTail<Input, Output, Context, InternalFasterSession>(FasterSession, ref key, ref input, ref desiredValue, ref output, untilAddress);
             }
             finally
             {