feat: concurrent writes in MirroringBufferedMutator

[mwalkiewicz]

This change is 

[dopiera]

Reviewed 3 of 14 files at r3, all commit messages.
Reviewable status: 3 of 14 files reviewed, 7 unresolved discussions (waiting on @dopiera and @mwalkiewicz)

---

bigtable-hbase-mirroring-client-1.x-parent/bigtable-hbase-mirroring-client-1.x/src/main/java/com/google/cloud/bigtable/mirroring/hbase1_x/bufferedmutator/ConcurrentMirroringBufferedMutator.java, line 189 at r3 (raw file):

    ListenableFuture<Void> secondaryFlushFinished = scheduleSecondaryFlush();

    final AtomicBoolean firstFinished = new AtomicBoolean(false);

This works, but why not use a standard solution, i.e. Futures.allAsList?

---

bigtable-hbase-mirroring-client-1.x-parent/bigtable-hbase-mirroring-client-1.x/src/main/java/com/google/cloud/bigtable/mirroring/hbase1_x/bufferedmutator/MirroringBufferedMutator.java, line 53 at r3 (raw file):

 * <p>We want to perform a secondary write only if we are certain that it was successfully applied
 * on primary database.

That is no longer true, is it?

---

bigtable-hbase-mirroring-client-1.x-parent/bigtable-hbase-mirroring-client-1.x/src/main/java/com/google/cloud/bigtable/mirroring/hbase1_x/bufferedmutator/MirroringBufferedMutator.java, line 107 at r3 (raw file):

  protected final BufferedMutator primaryBufferedMutator;
  protected final BufferedMutator secondaryBufferedMutator;
  protected final FlowController flowController;

As we discussed, flowControler is specific the sequential version.

---

bigtable-hbase-mirroring-client-1.x-parent/bigtable-hbase-mirroring-client-1.x/src/main/java/com/google/cloud/bigtable/mirroring/hbase1_x/bufferedmutator/MirroringBufferedMutator.java, line 109 at r3 (raw file):

  protected final FlowController flowController;
  protected final ListeningExecutorService executorService;
  protected final SecondaryWriteErrorConsumer secondaryWriteErrorConsumer;

The secondaryWriteErrorConsumer should likely be different between implementations, right? In in sequential we put the errors into the faillog, in concurrent we need to wait for them. I think you only want to provide this member in sequential (async) mode, right?

---

bigtable-hbase-mirroring-client-1.x-parent/bigtable-hbase-mirroring-client-1.x/src/main/java/com/google/cloud/bigtable/mirroring/hbase1_x/bufferedmutator/MirroringBufferedMutator.java, line 148 at r3 (raw file):

SecondaryWriteErrorConsumer secondaryWriteErrorConsumer,

As mentioned before - this is specific to the sequential version, right?

---

bigtable-hbase-mirroring-client-1.x-parent/bigtable-hbase-mirroring-client-1.x/src/main/java/com/google/cloud/bigtable/mirroring/hbase1_x/bufferedmutator/MirroringBufferedMutator.java, line 159 at r3 (raw file):

              throws RetriesExhaustedWithDetailsException {
            handlePrimaryException(e);
            userListener.onException(e, bufferedMutator);

In concurrent (sync), this is no longer the case, right?

---

bigtable-hbase-mirroring-client-1.x-parent/bigtable-hbase-mirroring-client-1.x/src/main/java/com/google/cloud/bigtable/mirroring/hbase1_x/bufferedmutator/MirroringBufferedMutator.java, line 342 at r3 (raw file):

  }

  protected static class FlushFutures {

Depending on the context, you're using the secondaryFlushFinished future to indicate what its name suggests or to indicate that both flushes have completed. Why not Introduce a third future?

[mwalkiewicz]

Reviewable status: 3 of 15 files reviewed, 7 unresolved discussions (waiting on @dopiera)

---

bigtable-hbase-mirroring-client-1.x-parent/bigtable-hbase-mirroring-client-1.x/src/main/java/com/google/cloud/bigtable/mirroring/hbase1_x/bufferedmutator/ConcurrentMirroringBufferedMutator.java, line 189 at r3 (raw file):

Previously, dopiera (Marek Dopiera) wrote…

This works, but why not use a standard solution, i.e. Futures.allAsList?

Futures.allAsList is @Beta, I don't know any stable method that is able to do it.

---

bigtable-hbase-mirroring-client-1.x-parent/bigtable-hbase-mirroring-client-1.x/src/main/java/com/google/cloud/bigtable/mirroring/hbase1_x/bufferedmutator/MirroringBufferedMutator.java, line 53 at r3 (raw file):

Previously, dopiera (Marek Dopiera) wrote…

 * <p>We want to perform a secondary write only if we are certain that it was successfully applied
 * on primary database.

That is no longer true, is it?

Done.

---

bigtable-hbase-mirroring-client-1.x-parent/bigtable-hbase-mirroring-client-1.x/src/main/java/com/google/cloud/bigtable/mirroring/hbase1_x/bufferedmutator/MirroringBufferedMutator.java, line 107 at r3 (raw file):

Previously, dopiera (Marek Dopiera) wrote…

As we discussed, flowControler is specific the sequential version.

Done.

---

bigtable-hbase-mirroring-client-1.x-parent/bigtable-hbase-mirroring-client-1.x/src/main/java/com/google/cloud/bigtable/mirroring/hbase1_x/bufferedmutator/MirroringBufferedMutator.java, line 109 at r3 (raw file):

Previously, dopiera (Marek Dopiera) wrote…

The secondaryWriteErrorConsumer should likely be different between implementations, right? In in sequential we put the errors into the faillog, in concurrent we need to wait for them. I think you only want to provide this member in sequential (async) mode, right?

There shouldn't be any secondaryWriteErrorConsumer in concurrent version, you are right, TBD.

---

bigtable-hbase-mirroring-client-1.x-parent/bigtable-hbase-mirroring-client-1.x/src/main/java/com/google/cloud/bigtable/mirroring/hbase1_x/bufferedmutator/MirroringBufferedMutator.java, line 148 at r3 (raw file):

Previously, dopiera (Marek Dopiera) wrote…

SecondaryWriteErrorConsumer secondaryWriteErrorConsumer,

As mentioned before - this is specific to the sequential version, right?

Done.

---

bigtable-hbase-mirroring-client-1.x-parent/bigtable-hbase-mirroring-client-1.x/src/main/java/com/google/cloud/bigtable/mirroring/hbase1_x/bufferedmutator/MirroringBufferedMutator.java, line 159 at r3 (raw file):

Previously, dopiera (Marek Dopiera) wrote…

In concurrent (sync), this is no longer the case, right?

Done.

---

bigtable-hbase-mirroring-client-1.x-parent/bigtable-hbase-mirroring-client-1.x/src/main/java/com/google/cloud/bigtable/mirroring/hbase1_x/bufferedmutator/MirroringBufferedMutator.java, line 342 at r3 (raw file):

Previously, dopiera (Marek Dopiera) wrote…

Depending on the context, you're using the secondaryFlushFinished future to indicate what its name suggests or to indicate that both flushes have completed. Why not Introduce a third future?

Done.

[mwalkiewicz]

Reviewable status: 3 of 15 files reviewed, 7 unresolved discussions (waiting on @dopiera)

---

bigtable-hbase-mirroring-client-1.x-parent/bigtable-hbase-mirroring-client-1.x/src/main/java/com/google/cloud/bigtable/mirroring/hbase1_x/bufferedmutator/MirroringBufferedMutator.java, line 109 at r3 (raw file):

Previously, mwalkiewicz (Mateusz Walkiewicz) wrote…

There shouldn't be any secondaryWriteErrorConsumer in concurrent version, you are right, TBD.

Done.

[dopiera]

Reviewable status: 3 of 15 files reviewed, all discussions resolved (waiting on @dopiera)

---

bigtable-hbase-mirroring-client-1.x-parent/bigtable-hbase-mirroring-client-1.x/src/main/java/com/google/cloud/bigtable/mirroring/hbase1_x/bufferedmutator/MirroringBufferedMutator.java, line 107 at r3 (raw file):

Previously, mwalkiewicz (Mateusz Walkiewicz) wrote…

Done.

But you left it here.

[mwalkiewicz]

Reviewable status: 3 of 15 files reviewed, all discussions resolved (waiting on @dopiera)

---

bigtable-hbase-mirroring-client-1.x-parent/bigtable-hbase-mirroring-client-1.x/src/main/java/com/google/cloud/bigtable/mirroring/hbase1_x/bufferedmutator/MirroringBufferedMutator.java, line 107 at r3 (raw file):

Previously, dopiera (Marek Dopiera) wrote…

But you left it here.

Done.

[dopiera]

Reviewable status: 3 of 15 files reviewed, all discussions resolved (waiting on @dopiera)