[C#] Cannot delete fasterlog from disk if the log was disposed with a commit request in flight

[hiteshmadan]

I recently upgraded to FASTER's latest nuget version and the following testcase started failing.

The testcase initiates a write, initiates a commit (with spinWait=false), disposes off the log and the device, and then tries to delete the log from disk. This can easily happen in my app because the app has API's to create/delete logs and to append to logs. The delete and append can be done concurrently.

The delete fails with System.IO.IOException: The process cannot access the file 'commit.0.0' because it is being used by another process.

I figured out that there is some code in fasterlog that's holding an active reference to the log-commit file even after dispose.

Note that if I remove the call to log.Commit, or do the commit with spinWait=true, then the delete does not fail.

Note that even if i wait for the commit to complete (with spinWait=false) - by checking that the ValueTask returned by TryEnqueue has succeeded/failed - I still cannot delete the log.

[TestMethod]
public void TestDisposeReleasesFileLocksWithInprogressCommit()
{
    DirectoryInfo di = Directory.CreateDirectory(this.TestContext.TestRunDirectory + $"\\log");
    IDevice device = Devices.CreateLogDevice(this.TestContext.TestRunDirectory + $"\\log\\testDisposeReleasesFileLocksWithInprogressCommit.log", preallocateFile: true, deleteOnClose: false);
    FasterLog fasterLog = new FasterLog(new FasterLogSettings { LogDevice = device, LogChecksum = LogChecksumType.PerEntry });
    fasterLog.TryEnqueue(new byte[100], out long beginAddress).Should().BeTrue();
    fasterLog.Commit(spinWait: false);
    fasterLog.Dispose();
    device.Dispose();
    di.Delete(recursive: true);
}

[badrishc]

Commit initiates a write to disk, so the delete would fail until that write completes. The following seems to work:

        [Test]
        public void TestDisposeReleasesFileLocksWithInprogressCommit()
        {
            var commitPath = TestContext.CurrentContext.TestDirectory + "/" + TestContext.CurrentContext.Test.Name + "/";
            DirectoryInfo di = Directory.CreateDirectory(commitPath);

            IDevice device = Devices.CreateLogDevice(commitPath + "fasterlog.log", preallocateFile: true, deleteOnClose: false);
            FasterLog fasterLog = new FasterLog(new FasterLogSettings { LogDevice = device, LogChecksum = LogChecksumType.PerEntry });
            Assert.IsTrue(fasterLog.TryEnqueue(new byte[100], out long beginAddress));
            fasterLog.Commit(spinWait: false);
            fasterLog.WaitForCommitAsync().GetAwaiter().GetResult();
            fasterLog.Dispose();
            device.Dispose();
            di.Delete(recursive: true);
        }

Note that even if i wait for the commit to complete (with spinWait=false) - by checking that the ValueTask returned by TryEnqueue has succeeded/failed - I still cannot delete the log.

TryEnqueue does not return a ValueTask. If you meant EnqueueAsync then note that this task completes when the record has been successfully enqueued in memory, not when it is committed to disk:

        /// <summary>
        /// Enqueue entry to log in memory (async) - completes after entry is 
        /// appended to memory, NOT committed to storage.
        /// </summary>
        /// <param name="entry">Entry to enqueue</param>
        /// <param name="token">Cancellation token</param>
        /// <returns></returns>
        public ValueTask<long> EnqueueAsync(byte[] entry, CancellationToken token = default)

[hiteshmadan]

Note that even if i wait for the commit to complete (with spinWait=false) - by checking that the ValueTask returned by TryEnqueue has succeeded/failed - I still cannot delete the log.

I misspoke here, i meant to say "by checking that the ValueTask returned by WaitForCommitAsync has faulted".

Here is something that fails (delete fails with IOException) even if I wait for the commit task to fail. Adding a 10 second delay before deleting the subscription also doesn't free up the file lock, so it looks unlikely there's a race condition in how the testcase is structured ?

[TestMethod]
public async Task TestDisposeReleasesFileLocksWithInprogressCommit()
{
    DirectoryInfo di = Directory.CreateDirectory(this.TestContext.TestRunDirectory + $"\\log");
    IDevice device = Devices.CreateLogDevice(this.TestContext.TestRunDirectory + $"\\log\\testDisposeReleasesFileLocksWithInprogressCommit.log", preallocateFile: true, deleteOnClose: false);
    FasterLog fasterLog = new FasterLog(new FasterLogSettings { LogDevice = device, LogChecksum = LogChecksumType.PerEntry });
    fasterLog.TryEnqueue(new byte[100], out long beginAddress).Should().BeTrue();
    fasterLog.Commit(spinWait: false);
    ValueTask waitingWriter = fasterLog.WaitForCommitAsync(beginAddress + 1);
    fasterLog.Dispose();
    device.Dispose();
    while (!waitingWriter.IsCompleted)
    {
        await Task.Yield();
    }
    waitingWriter.IsFaulted.Should().BeTrue();
    await Task.Delay(10_000);

    di.Delete(recursive: true);
}