flush_job: do not roll back memtable flush on CF drop and DB shutdown (…

…#126)

Rolling back in these cases is meaningless because the memtables will be
freed shortly afterwards, so making them available for flush again is useless
at best.

Additionally, in atomic flush, memtables that were successfully flushed
aren't rolled back if a CF is dropped or a DB shutdown was requested (the
shutdown case is the more relevant one for the comparison here because it
aborts the flush, whereas the in the CF drop case it's simply ignored at
flush result installation time). In this regard this change simply matches
the behaviour there.

Lastly, this might be needed for the WriteBufferManager changes in #113,
since we plan to use the information about memory that can be flushed in
the future for triggering flushes based on immutable memory as well, and
rolling back flushes causes the memtable memory to become ready for flush
again for a brief period of time until it is dropped, which might wrongly
affect the WBM decisions.

db/flush_job.cc

@@ -286,16 +286,12 @@ Status FlushJob::Run(LogsWithPrepTracker* prep_tracker, FileMetaData* file_meta,
     s = WriteLevel0Table();
   }
 
-  if (s.ok() && cfd_->IsDropped()) {
-    s = Status::ColumnFamilyDropped("Column family dropped during compaction");
-  }
-  if ((s.ok() || s.IsColumnFamilyDropped()) &&
-      shutting_down_->load(std::memory_order_acquire)) {
-    s = Status::ShutdownInProgress("Database shutdown");
-  }
-
   if (!s.ok()) {
     cfd_->imm()->RollbackMemtableFlush(mems_, meta_.fd.GetNumber());
+  } else if (shutting_down_->load(std::memory_order_acquire)) {
+    s = Status::ShutdownInProgress("Database shutdown");
+  } else if (cfd_->IsDropped()) {
+    s = Status::ColumnFamilyDropped("Column family dropped during flush");
   } else if (write_manifest_) {
     TEST_SYNC_POINT("FlushJob::InstallResults");
     // Replace immutable memtable with the generated Table