unit tests: Fix GlobalAndWBMSetupDelay (#589)

The problem was that calling Flush with FlushOptions.wait = true,
doesn't wait for the memory to be freed which also happens during the
flush process.
So the fix is adding a sync point which delays the assertion of checking
the WBM memory usage to after the flush cleared the memtable memory.

HISTORY.md

@@ -5,6 +5,9 @@
 ### Enhancements
 * db_bench: add estimate-table-readers-mem benchmark which prints these stats.
 
+### Bug Fixes
+* unit tests: fix GlobalWriteControllerTest.GlobalAndWBMSetupDelay by waiting for the memtable memory release.
+
 ## Fig v2.5.0 (06/14/2023)
 Based on RocksDB 8.1.1
 

db/global_write_controller_test.cc

@@ -360,6 +360,12 @@ TEST_F(GlobalWriteControllerTest, GlobalAndWBMSetupDelay) {
   OpenDBsAndSetUp(num_dbs, options, true);
   WriteOptions wo;
 
+  SyncPoint::GetInstance()->DisableProcessing();
+  SyncPoint::GetInstance()->LoadDependency(
+      {{"DBImpl::BackgroundCallFlush:ContextCleanedUp",
+        "GlobalAndWBMSetupDelay:WaitForMemFree"}});
+  SyncPoint::GetInstance()->EnableProcessing();
+
   // verify that theres no delay
   for (int i = 0; i < num_dbs; i++) {
     ASSERT_FALSE(IsDbWriteDelayed(dbimpls_[i]));
@@ -388,6 +394,15 @@ TEST_F(GlobalWriteControllerTest, GlobalAndWBMSetupDelay) {
 
   // clear the memory usage
   ASSERT_OK(dbs_[0]->Flush(FlushOptions()));
+
+  // The Flush waits for imm()->NumNotFlushed() == 0 which happens in
+  // MemTableListVersion::Remove inside FlushJob::Run. However, the WBM memory
+  // is only freed after FlushJob::Run() ends in job_context.Clean() under
+  // DBImpl::BackgroundCallFlush right after PurgeObsoleteFiles. So the Flush
+  // call can return before the memory is actually freed thats why we need wait
+  // until the memory is actually freed in job_context.Clean().
+  TEST_SYNC_POINT("GlobalAndWBMSetupDelay:WaitForMemFree");
+
   // there should only be 2k per memtable left
   ASSERT_TRUE(options.write_buffer_manager->memory_usage() < 5_kb);