Skip to content

Commit

Permalink
[#3357][#3358][colocation] Create table tombstone
Browse files Browse the repository at this point in the history
Summary:

There is no mechanism to delete the data for a colocated table.  For
`DROP TABLE`, metadata changes ensure that the dropped table is no
longer reachable; however, the data remains in the colocated tablet.
For `TRUNCATE TABLE`, all tables in the colocated tablet are truncated.

Use the concept of a table-level tombstone to delete colocated table
data for both `DROP TABLE` and `TRUNCATE TABLE`.  Split the work into
four steps:

1. Prevent `TRUNCATE TABLE` on a colocated table from sending
   `TruncateTablet` RPCs
1. For both `DROP TABLE` and `TRUNCATE TABLE`, send write DML requests
   to create table-level tombstones
1. Detect table-level tombstone for read path
1. Detect table-level tombstone for compactions

Perform the first two steps.  Additionally, heavily modify the
`yb_feature_colocation` PostgreSQL regress test to include `TRUNCATE`
statements, test `DROP` cases more comprehensively, and use more
descriptive table and index names.

Call tree:

* [ ] `YBCDropIndex` | `YBCDropTable` | `YBCTruncateTable`
  * [x] `YBCPgNewTruncateColocated`
    * [x] `PgApiImpl::NewTruncateColocated`
      * [x] `PgTruncateColocated::PgTruncateColocated`
      * [ ] `PgDmlWrite::Prepare`
        * [x] `PgTruncateColocated::AllocWriteRequest`
          * [x] `PgTableDesc::NewPgsqlTruncateColocated`
            * [x] `YBTable::NewPgsqlTruncateColocated`
              * [x] `YBPgsqlWriteOp::NewTruncateColocated`
                * [ ] `NewYBPgsqlWriteOp`
  * [x] `YBCPgDmlBindTable`
    * [x] `PgApiImpl::DmlBindTable`
      * [x] `PgDml::BindTable`
  * [ ] `YBCPgDmlExecWriteOp`
    * [ ] `PgApiImpl::DmlExecWriteOp`
      * [ ] `PgDmlWrite::Exec`
        * [ ] `PgDocOp::Execute`
          * ...
            * [ ] `TabletServiceImpl::Write`
              * ...
                * [ ] `PgsqlWriteOperation::Apply`
                  * [x] `PgsqlWriteOperation::ApplyTruncateColocated`

Unused functions:

* [x] `PgApiImpl::ExecTruncateColocated`
* [x] `PgsqlWriteRequestPB::PGSQL_TRUNCATE_COLOCATED`
* [x] `YBCPgExecTruncateColocated`

Style:

* Fix typo `s/contants/constants/` in `pggate.h` and `ybc_pggate.h`
* Fix typo in comments regarding `BindColumn`
* Remove unused function `YBPgsqlWriteOp::NewUpsert`

Legend:

* [x] Newly added
* [ ] Already existed

Test Plan:

* `./yb_build.sh --java-test org.yb.pgsql.TestPgRegressBetaFeatures`

Reviewers: neha

Reviewed By: neha

Subscribers: yql, bogdan

Differential Revision: https://phabricator.dev.yugabyte.com/D7800
  • Loading branch information
Jason Kim committed Jan 22, 2020
1 parent 8606194 commit 6286e50
Show file tree
Hide file tree
Showing 25 changed files with 671 additions and 193 deletions.
36 changes: 36 additions & 0 deletions src/postgres/src/backend/commands/ybccmds.c
Original file line number Diff line number Diff line change
Expand Up @@ -535,6 +535,15 @@ YBCDropTable(Oid relationId)
{
YBCPgStatement handle;

/* Create table-level tombstone */
/* TODO(jason): do this only for colocated tables (issue #3387) */
HandleYBStatus(YBCPgNewTruncateColocated(MyDatabaseId, relationId, false, &handle));
HandleYBStmtStatus(YBCPgDmlBindTable(handle), handle);
int rows_affected_count = 0;
HandleYBStmtStatus(YBCPgDmlExecWriteOp(handle, &rows_affected_count),
handle);
HandleYBStatus(YBCPgDeleteStatement(handle));
/* Drop the table */
HandleYBStatus(YBCPgNewDropTable(MyDatabaseId,
relationId,
false, /* if_exists */
Expand All @@ -549,9 +558,18 @@ YBCTruncateTable(Relation rel) {
Oid relationId = RelationGetRelid(rel);

/* Truncate the base table */
/* TODO(jason): do this only for non-colocated tables (issue #3387) */
HandleYBStatus(YBCPgNewTruncateTable(MyDatabaseId, relationId, &handle));
HandleYBStmtStatus(YBCPgExecTruncateTable(handle), handle);
HandleYBStatus(YBCPgDeleteStatement(handle));
/* Create table-level tombstone */
/* TODO(jason): do this only for colocated tables (issue #3387) */
HandleYBStatus(YBCPgNewTruncateColocated(MyDatabaseId, relationId, false, &handle));
HandleYBStmtStatus(YBCPgDmlBindTable(handle), handle);
int rows_affected_count = 0;
HandleYBStmtStatus(YBCPgDmlExecWriteOp(handle, &rows_affected_count),
handle);
HandleYBStatus(YBCPgDeleteStatement(handle));

if (!rel->rd_rel->relhasindex)
return;
Expand All @@ -567,9 +585,18 @@ YBCTruncateTable(Relation rel) {
if (indexId == rel->rd_pkindex)
continue;

/* Truncate the index table */
HandleYBStatus(YBCPgNewTruncateTable(MyDatabaseId, indexId, &handle));
HandleYBStmtStatus(YBCPgExecTruncateTable(handle), handle);
HandleYBStatus(YBCPgDeleteStatement(handle));
/* Create table-level tombstone */
/* TODO(jason): do this only for colocated tables (issue #3387) */
HandleYBStatus(YBCPgNewTruncateColocated(MyDatabaseId, relationId, false, &handle));
HandleYBStmtStatus(YBCPgDmlBindTable(handle), handle);
int rows_affected_count = 0;
HandleYBStmtStatus(YBCPgDmlExecWriteOp(handle, &rows_affected_count),
handle);
HandleYBStatus(YBCPgDeleteStatement(handle));
}

list_free(indexlist);
Expand Down Expand Up @@ -832,6 +859,15 @@ YBCDropIndex(Oid relationId)
{
YBCPgStatement handle;

/* Create table-level tombstone */
/* TODO(jason): do this only for colocated tables (issue #3387) */
HandleYBStatus(YBCPgNewTruncateColocated(MyDatabaseId, relationId, false, &handle));
HandleYBStmtStatus(YBCPgDmlBindTable(handle), handle);
int rows_affected_count = 0;
HandleYBStmtStatus(YBCPgDmlExecWriteOp(handle, &rows_affected_count),
handle);
HandleYBStatus(YBCPgDeleteStatement(handle));
/* Drop the index table */
HandleYBStatus(YBCPgNewDropIndex(MyDatabaseId,
relationId,
false, /* if_exists */
Expand Down
Loading

0 comments on commit 6286e50

Please sign in to comment.