Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
[C#] Change locking to use FasterBase hash buckets (#799)
* Locking fixes
- CompletePending* must also check to lock non-auxiliary records
- Ephemeral locking must look for lock evictions as well
- InternalLock must check for SpinWaitUntilRecordIsClosed
- Replace oneMiss handling in InternalLock
- EphemeralSUnlock must follow unlock failures
- LockTable should have its own bufferPool
- Ensure OnPagesClosed does eviction scans in address order
* More locking fixes:
- Fix missing unlock of a Tombstoned record in InternalContinuePendingRead
- Add more Asserts where ManualLocking requires a locked record
- Update and remove obsolete comments
* - Remove key-present optimization for SpinWaitUntilRecordIsClosed
- Better checking on following records transferred from LockTable in InternalLock(unlock)
- Add some additional comments
* Fix hash bucket overflow pointer during recovery
Fix debug assert logic
* WIP on locktable implemented via mainhash overflow buckets
* unstaged files missed in previous push
* Updates to EphemeralOnly locking for transfer-from-RO/RC
* Change LockTableTests to OverflowBucketlockTableTests
* Remove RecordInfo.Tentative, .IsIntermediate*
* More locking WIP: session.NeedKeyLockCode, test cleanup
* WIP locking: Tests build
* WIP on mainhash locking:
- Add KeyCode creation and sorting and methods public and internal
- Remove unnecessary IsLocked (it does not make work with EphemeralOnly and SessionControlled may map to multiple keys)
- More tests run now
* Most UTs run
* More UTs succeed
* Still some UT failures
* - use XLocks as Tentative indicator for insertions/splices at log tail and readcache insertions
- improve readcache consistency checking
- clear HasInMemoryLock when HeadAddress goes above srcRecord address following BlockAllocate
- rename LockingMode.SessionControlled -> Mixed and other renaming for clarity
- remove some unused methods
- fix locking sequences in InternalContinuePendingR(ead|MW)
* More fixes
- Missing 'ref' on recordInfo var in FindAndUnlockTransferredRecord
- Improve record allocation checking for below-HA in-memory source and RecordInfo lock clearing if so
- Add LockCode to HashEntryInfo in DEBUG
* Remove SpinWaitUntilRecordIsClosed; it was only used in ReadCache and in the relevant case we just need to wait for ReadCacheEvict to complete.
* - MixedModeTentativeOrClosed is not needed
- Add/Improve some ToString()s
- Remove unused RecSrc.ClearSrc()
- Rename to HasRecordInfoLock
- Move ICPRead HandleImmediateRetryStatus outside the lock scope
- Consoldiate EphemeralSUnlock(AfterPendingIO)
* Streamline locking and FindInMemory calls in InternalXxx
* - Change names to pendingContext.Initial*Address and improve setting of these in InternalXxx and handling of these in CompletePending*() and TryFindRecordInMemory.
- Also consistentize related param names for readcache and log search to minAddress.
* Clean up pending RMW call to CreateNewRecordRMW, per prior changes that make the inputSrc unnecessary.
* - Fix stale readcache assignment to recSrc.Log
- make searches after pendingIO completion non-inclusive of minAddress
- Some ToString() and naming changes for consistency
- Remove RecordInfo.IsValidUpdateOrLockSource (superseded by direct call to .IsClosed)
* Allow length specification on KeyCode array operations
* - Fix check for lock having escaped to eviction zone following BlockAllocate
- Add some comments for SafeHeadAddress et al.
* - Make sure the readcache bit is set when needed on addresses put into stackCtx.newLogicalAddress.
- Rename to HandleNewRecordOnException for clarity
* Update remote project
* - Make sure FindAndUnlockTransferredRecord can go to SafeHeadAddress
- Add non-ref overload of GetLockCode
* Tweaks for benchmark with Mixed locking
* Add ILockableKey.CompareLockCodes
* Change LockCode to just be hashcode, since we may grow the hash table.
* - Make CPR consistency check work without conflict with the key latches in the HashBuckets.
- Change some names for clarity.
- Pending read completion should return values from any in-memory record, as they are the latest
* Remove duplicate opCtx param and removed sessionCtx (aka currentCtx, aka ctx) from calls where FasterSession was also passed (in favor of (new) FasterSession.SessionCtx), where the called method has a type constraint of "where FasterSession : IFasterSession<Key, Value, Input, Output, Context>". Not yet done: converting "whereFasterSession : IFasterSession" constraints to generic to remove the duplicate ctx parameter.
* Fix DoLockOp to compare bucket indexes, not KeyCodes
* Finish removal of duplicate sessionCtx param from calls where FasterSession was also passed (in favor of (new) FasterSession.SessionCtx), where the called method has a type constraint of "where FasterSession : IFasterSession" (non-generic). This cannot be done in Synchronization (the state machines) due to their use of NullFasterSession.
* Fix key duplication in readcache and log:
- VerifyInMemoryAddresses should not refresh hei if no splice point is present
- FindRecordInMemory checks pendingContext.InitialHash
- Move CompleteCopyToTail in ITCTT into the non-readcache section
- TryFindRecordInMemory needs to check pc.InitialEntryIndex's readcache bit
- Updaters need to scan down to hlog.ReadOnlyAddress to ensure the "key is not at or below LatestLogicalAddress" logic holds
- ReadCacheCheckTailAfterSplice needs to check hei.*Address readcache bit
* Replace use of SafeHeadAddress with SpinWaitUntilClosed in FindInReadCache
* Remove LockingMode.EphemeralOnly
add --hashpack arg to FASTER.benchmark
* Tweaks for the Standard locking change
* fix remote to use LockingMode.Standard
* Add DisposeTest comments and more Asserts
* Use calls on RecordSource to make it easier to verify correct matching of log and physicalAddress
* Temporarily disable LocalMemoryDevice tests for investigation
* test removal of DisposeTests
* - Fix for CTT to address SameKeyInsertAndCTTTest failure
- Add missing arg parsing for MemOnlyCache
- Re-enable DeviceType.LocalMemory in tests
* [C#] MemOnlyCache sample: improved size tracking, including read cache (#800)
* Improved cache size tracking, including read cache
* Rename MemOnlyCache to ResizableCacheStore.
* update doc
* fixes
* Updates
* ClearBitsForDiskImages should unseal record as well (Q: why was it sealed to begin with?)
* Correct clearing of index bits during recovery
* Remove RecordInfo sealing
* WIP Ephemeral locking at IFunctions-level only, part 1
* Remove unused SessionType enumeration; consolidate duplicate IFasterSession implementations
* - Add EphemeralLocking readcache detach/reattach logic
- Remove unused ReadFlags.ResetModifiedBit
* Rename a couple files
* re-add RecordInfo.Seal for Ephemeral locking; remove unused (RMW|Upsert)Action.NeedMoreSpace (this is the default 'false' behavior)
* Remove single-key overloads of Lock() and Unlock()
* - Update a too-aggressive assert.
- CreateLogDevice errors on Windows if path > MAX_PATH minus room for segment#.
* Improve previous MAX_PATH fix
* Chnge Read of an expired to just return NOTFOUND | Expired rather than tombstoning
* RETRY_LATER rather than trying to re-find splice point directly in VerifyInMemoryAddresses
---------
Co-authored-by: Badrish Chandramouli <badrishc@microsoft.com>- Loading branch information
