[CDCSDK] Add guards around updating invalid checkpoints during snapshot phase #17972
Assignees
Labels
Comments
yugabyte-ci
added
kind/bug
adithya-kb
pushed a commit
that referenced
this issue
Jun 28, 2023
…g snapshot phase Summary: We need to add guards around setting invalid checkpoints , which can arise in some corner cases: 1. After the snapshot is done, the connector sends a snasphot_done_key, the connector sends a fixed checkpoint term and index (0.0) in this request , we need to ignore the term and index fields in this request 2. If explicit checkpointing is used, and there was no data to be streamed from the the tablet, we can get an explicit checkpoint as -1.-1 , we should add a guard to stop setting this as a checkpoint Jira: DB-7038 Test Plan: Existing ctests Reviewers: vkushwaha, srangavajjula, skumar Reviewed By: srangavajjula Subscribers: ycdcxcluster Differential Revision: https://phorge.dev.yugabyte.com/D26526
adithya-kb
pushed a commit
that referenced
this issue
Jul 6, 2023
…eckpoints during snapshot phase Summary: Original commit: 54ecf5d / D26526 To support resuming of snapshot on connectors using a stream created in explicit mode, we will add snapshot_op_id and snapshot_time to each snapshot record. Additionally, when we initiate the snapshot process for colocated table, we need to copy over the checkpoint details from the first table's snapshot row to the colocated streaming row. To ensure we are doing this only for the first table, we were looking at the "snapshot_time" field of the streaming row , now instead we check to see if the checkpoint of the streaming row in 0.0. This is needed as a consequence of change: https://phorge.dev.yugabyte.com/D26526, where we stopped updating the checkpoint of the streaming row when a snapshot is done. Jira: DB-7038 Test Plan: ybd --cxx-test integration-tests_cdcsdk_snapshot-test --gtest_filter CDCSDKYsqlTest.TestCheckpointUpdatedDuringSnapshot Reviewers: vkushwaha, aagarwal Reviewed By: vkushwaha Subscribers: ycdcxcluster Differential Revision: https://phorge.dev.yugabyte.com/D26664
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Jira Link: DB-7038
We need to add guards around setting invalid checkpoints , which can arise in some corner cases:
The text was updated successfully, but these errors were encountered: