feat: Client-side timestamping

[mwalkiewicz]

This change is 

[dopiera]

Reviewable status: 0 of 38 files reviewed, 7 unresolved discussions (waiting on @dopiera and @mwalkiewicz)

---

quickstart.md, line 135 at r1 (raw file):

## Client-side timestamping
HBase and Bigtable assign row version (timestamp) based on server-side time for mutations without version assigned by the client. The Mirroring Client issues writes to underlying databases a few milliseconds apart and performing mutations without version assigned on the client side will cause inconsistencies between databases. To mitigate some of those issues a client-side timestamping is available in the Mirroring Client. When client-side timestamping is enabled the Mirroring Client will automatically add a timestamp based on client's local time to every `Put` object passed to the Mirroring Client. Client-side timestamps assigned by `Table`s and `BufferedMutator`s created by one `Connection` are always increasing, even if system clock is moved backwards, for example by NTP or manually by the user.

That probably goes without saying, but I think we should mention that this concerns only the mutations which do not have that timestamp explicitly set by the user.

Code quote:

to every `Put` object passed to the Mirroring Client

---

quickstart.md, line 164 at r1 (raw file):

## Caveats
### Timestamps
Be aware that client-side timestamping modifies only `Put`s - `Delete`s, `Increment`s and `Append`s are not affected by this setting and will cause inconsistencies between databases.

Can you also add a sentence about the consequences of the time differences between the client servers?

---

quickstart.md, line 137 at r2 (raw file):

HBase and Bigtable assign row version (timestamp) based on server-side time for mutations without version assigned by the client. The Mirroring Client issues writes to underlying databases a few milliseconds apart and performing mutations without version assigned on the client side will cause inconsistencies between databases. To mitigate some of those issues a client-side timestamping is available in the Mirroring Client. When client-side timestamping is enabled the Mirroring Client will automatically add a timestamp based on client's local time to every `Put` object passed to the Mirroring Client. Client-side timestamps assigned by `Table`s and `BufferedMutator`s created by one `Connection` are always increasing, even if system clock is moved backwards, for example by NTP or manually by the user.
Be aware that client-side timestamping modifies only `Put`s - `Delete`s, `Increment`s and `Append`s are not affected by this setting and will cause inconsistencies between databases.
To enable client-side timestamping set `google.bigtable.mirroring.enable-default-client-side-timestamps` to `true`.

As discussed F2F, let's make it a string with 3 valid values: "disabled", "inplace" and "copy".

---

bigtable-hbase-mirroring-client-1.x-parent/bigtable-hbase-mirroring-client-1.x/src/main/java/com/google/cloud/bigtable/mirroring/hbase1_x/MirroringConnection.java, line 95 at r1 (raw file):

  protected final boolean waitForSecondaryWrites;
  protected final Timestamper timestamper;

Could you please add a description the all the timestamper fields along these lines:

// Depending on configuration, ensures that mutations have an explicitly set timestamp.

---

bigtable-hbase-mirroring-client-1.x-parent/bigtable-hbase-mirroring-client-1.x/src/main/java/com/google/cloud/bigtable/mirroring/hbase1_x/MirroringOptions.java, line 160 at r1 (raw file):

    this.enableDefaultClientSideTimestamps =
        configuration.getBoolean(MIRRORING_ENABLE_DEFAULT_CLIENT_SIDE_TIMESTAMPS, false);

I think we should set this to inplace. We need to pick a less bad value and I'd make the judgement call that time drifting on the servers will be a smaller issue than diverging databases.

---

bigtable-hbase-mirroring-client-1.x-parent/bigtable-hbase-mirroring-client-1.x/src/main/java/com/google/cloud/bigtable/mirroring/hbase1_x/utils/timestamper/MonotonicTimestamper.java, line 80 at r2 (raw file):

          }
        } catch (IOException e) {
          // IOException is thrown when `cell` does not implement `SettableTimestamp` and if it

assert false;?

---

bigtable-hbase-mirroring-client-1.x-parent/bigtable-hbase-mirroring-client-1.x/src/main/java/com/google/cloud/bigtable/mirroring/hbase1_x/utils/timestamper/Timestamper.java, line 23 at r2 (raw file):

import org.apache.hadoop.hbase.client.RowMutations;

public interface Timestamper {

Please add a description of what it is

[mwalkiewicz]

Reviewable status: 0 of 45 files reviewed, 7 unresolved discussions (waiting on @dopiera and @mwalkiewicz)

---

quickstart.md, line 164 at r1 (raw file):

Previously, dopiera (Marek Dopiera) wrote…

Can you also add a sentence about the consequences of the time differences between the client servers?

Done.

---

quickstart.md, line 137 at r2 (raw file):

Previously, dopiera (Marek Dopiera) wrote…

As discussed F2F, let's make it a string with 3 valid values: "disabled", "inplace" and "copy".

Done.

---

bigtable-hbase-mirroring-client-1.x-parent/bigtable-hbase-mirroring-client-1.x/src/main/java/com/google/cloud/bigtable/mirroring/hbase1_x/MirroringConnection.java, line 95 at r1 (raw file):

Previously, dopiera (Marek Dopiera) wrote…

Could you please add a description the all the timestamper fields along these lines:

// Depending on configuration, ensures that mutations have an explicitly set timestamp.

Done.

---

bigtable-hbase-mirroring-client-1.x-parent/bigtable-hbase-mirroring-client-1.x/src/main/java/com/google/cloud/bigtable/mirroring/hbase1_x/MirroringOptions.java, line 160 at r1 (raw file):

Previously, dopiera (Marek Dopiera) wrote…

I think we should set this to inplace. We need to pick a less bad value and I'd make the judgement call that time drifting on the servers will be a smaller issue than diverging databases.

Done.

---

bigtable-hbase-mirroring-client-1.x-parent/bigtable-hbase-mirroring-client-1.x/src/main/java/com/google/cloud/bigtable/mirroring/hbase1_x/utils/timestamper/MonotonicTimestamper.java, line 80 at r2 (raw file):

Previously, dopiera (Marek Dopiera) wrote…

assert false;?

Done. I'll throw a new RuntimeException(e) because assert would be rejected in the review.

[dopiera]

Reviewed 9 of 38 files at r1, 2 of 5 files at r2, 27 of 32 files at r3, 5 of 7 files at r4, all commit messages.
Reviewable status: 43 of 45 files reviewed, 4 unresolved discussions (waiting on @dopiera and @mwalkiewicz)

---

bigtable-hbase-mirroring-client-1.x-parent/bigtable-hbase-mirroring-client-1.x/src/main/java/com/google/cloud/bigtable/mirroring/hbase1_x/utils/MirroringConfigurationHelper.java, line 287 at r4 (raw file):

  /**
   * Enables timestamping {@link org.apache.hadoop.hbase.client.Put}s without timestamp set based on
   * client's host local time. Client-side timestamps assigned by {@link Table}s and {@link

Nit: local time is the time you see on your clock, i.e. time zone is needed to interpret it. Timestamp is the duration since epoch in UTC, hence it's not local.

---

bigtable-hbase-mirroring-client-1.x-parent/bigtable-hbase-mirroring-client-1.x/src/main/java/com/google/cloud/bigtable/mirroring/hbase1_x/utils/timestamper/Timestamper.java, line 23 at r2 (raw file):

Previously, dopiera (Marek Dopiera) wrote…

Please add a description of what it is

Ping ;)

[mwalkiewicz]

Reviewable status: 39 of 45 files reviewed, 3 unresolved discussions (waiting on @dopiera)

---

bigtable-hbase-mirroring-client-1.x-parent/bigtable-hbase-mirroring-client-1.x/src/main/java/com/google/cloud/bigtable/mirroring/hbase1_x/utils/timestamper/Timestamper.java, line 23 at r2 (raw file):

Previously, dopiera (Marek Dopiera) wrote…

Ping ;)

Done, sorry.

[dopiera]

Reviewable status: 39 of 45 files reviewed, all discussions resolved (waiting on @dopiera)