Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(redshift): tables were dropped on table name change #24308

Merged
merged 56 commits into from
Dec 7, 2023
Merged
Show file tree
Hide file tree
Changes from 53 commits
Commits
Show all changes
56 commits
Select commit Hold shift + click to select a range
6b3a91c
buildup
Rizxcviii Feb 21, 2023
61c09aa
initial commit: test, initial work
Rizxcviii Feb 21, 2023
11c42ec
Merge branch 'main' into feature/persist-table-name-change
Rizxcviii Feb 22, 2023
6899734
reverting, updating resource id
Rizxcviii Feb 22, 2023
125d1f5
updating test
Rizxcviii Feb 22, 2023
5448bf9
more explicit with test
Rizxcviii Feb 22, 2023
96a5e8d
catching error when table name is changed
Rizxcviii Feb 23, 2023
93d7459
test to delete table name
Rizxcviii Feb 24, 2023
464847c
modification: new physicalResourceId creates new resource
Rizxcviii Feb 27, 2023
0c9d55d
modification: storing Data key in return object, to keep track of pre…
Rizxcviii Feb 28, 2023
adb99cc
modification
Rizxcviii Feb 28, 2023
43ad1cb
modification: update to delete scenario
Rizxcviii Mar 1, 2023
cf36699
modification: new resources are not created on table name change
Rizxcviii Mar 1, 2023
c444707
addition: test case for Data not existing
Rizxcviii Mar 1, 2023
da2f78f
modification: bugfix for failing test
Rizxcviii Mar 1, 2023
ac30602
integ test
Rizxcviii Mar 1, 2023
940942a
deleting table in integ test
Rizxcviii Mar 2, 2023
1724d38
creating new tables from stack id
Rizxcviii Mar 2, 2023
c8a3165
reverting replace
Rizxcviii Mar 2, 2023
6143c67
reverting file
Rizxcviii Mar 2, 2023
77b9ec1
reworking priviliges for this PR
Rizxcviii Mar 2, 2023
0f332df
modification: using stack id rather than request id, is the same as m…
Rizxcviii Mar 2, 2023
baacd96
Merge branch 'main' into feature/persist-table-name-change
Rizxcviii Mar 3, 2023
f5301d3
modification: deletion for both legacy and current tables
Rizxcviii Mar 6, 2023
312fc64
modification: this should not be called
Rizxcviii Mar 7, 2023
8b31212
addition: default actions
Rizxcviii Mar 7, 2023
021bd6d
addition: test for table id appended
Rizxcviii Mar 8, 2023
c61a7a3
integ test for change to table
Rizxcviii Mar 8, 2023
d06549e
modification: updating integ to remove table name change
Rizxcviii Mar 8, 2023
cf044b6
Merge branch 'main' into feature/persist-table-name-change
Rizxcviii Mar 9, 2023
e646409
integ test
Rizxcviii Mar 17, 2023
e9beee4
modification: test update for new cdk tests
Rizxcviii Mar 20, 2023
adfd101
Merge branch 'main' into feature/persist-table-name-change
Rizxcviii Oct 6, 2023
942e503
removing old files
Rizxcviii Oct 6, 2023
dbca263
Merge branch 'main' into feature/persist-table-name-change
Rizxcviii Oct 6, 2023
0dbd0d0
removing table id, not needed
Rizxcviii Oct 6, 2023
6ecf510
Merge branch 'main' into feature/persist-table-name-change
Rizxcviii Oct 28, 2023
842bfe2
removing tableid
Rizxcviii Nov 1, 2023
d2917e0
fixing test
Rizxcviii Nov 1, 2023
a879bc7
removing try, catch. Breaking change instead
Rizxcviii Nov 1, 2023
e2b6814
test
Rizxcviii Nov 1, 2023
61d608c
deleting only table name, not resource id
Rizxcviii Nov 1, 2023
11fe979
Merge branch 'main' into feature/persist-table-name-change
Rizxcviii Nov 1, 2023
0099b2e
removing diffAssets, was causing failure
Rizxcviii Nov 1, 2023
c539e47
moving tablename to top of handler
Rizxcviii Nov 9, 2023
3d9714b
using `newTableName`
Rizxcviii Nov 9, 2023
71a49f6
Merge branch 'main' into feature/persist-table-name-change
Rizxcviii Nov 9, 2023
1ee3968
Merge branch 'main' into feature/persist-table-name-change
scanlonp Nov 14, 2023
47443ee
disabling functionality for older tables
Rizxcviii Nov 20, 2023
ef50f34
revert
Rizxcviii Nov 20, 2023
4f4d609
Merge branch 'main' into feature/persist-table-name-change
Rizxcviii Nov 20, 2023
c571c8c
using old table name if needed
Rizxcviii Nov 20, 2023
1a3f408
linting
Rizxcviii Nov 20, 2023
f0f1ec8
changing table name README
Rizxcviii Dec 7, 2023
5019bb8
Merge branch 'main' into feature/persist-table-name-change
scanlonp Dec 7, 2023
eaa7285
Merge branch 'main' into feature/persist-table-name-change
mergify[bot] Dec 7, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -1,34 +1,39 @@
/* eslint-disable-next-line import/no-unresolved */
import * as AWSLambda from 'aws-lambda';
import { Column } from '../../table';
import { executeStatement } from './redshift-data';
import { ClusterProps, TableAndClusterProps, TableSortStyle } from './types';
import { areColumnsEqual, getDistKeyColumn, getSortKeyColumns } from './util';
import { Column } from '../../table';
import { areColumnsEqual, getDistKeyColumn, getSortKeyColumns, makePhysicalId } from './util';

export async function handler(props: TableAndClusterProps, event: AWSLambda.CloudFormationCustomResourceEvent) {
const tableNamePrefix = props.tableName.prefix;
const tableNameSuffix = props.tableName.generateSuffix === 'true' ? `${event.RequestId.substring(0, 8)}` : '';
const getTableNameSuffix = (generateSuffix: string) => generateSuffix === 'true' ? `${event.StackId.substring(event.StackId.length - 12)}` : '';
const tableColumns = props.tableColumns;
const tableAndClusterProps = props;
const useColumnIds = props.useColumnIds;
let tableName = tableNamePrefix + getTableNameSuffix(props.tableName.generateSuffix);

if (event.RequestType === 'Create') {
const tableName = await createTable(tableNamePrefix, tableNameSuffix, tableColumns, tableAndClusterProps);
return { PhysicalResourceId: tableName };
tableName = await createTable(tableNamePrefix, getTableNameSuffix(props.tableName.generateSuffix), tableColumns, tableAndClusterProps);
return { PhysicalResourceId: makePhysicalId(tableNamePrefix, tableAndClusterProps, event.StackId.substring(event.StackId.length - 12)) };
} else if (event.RequestType === 'Delete') {
await dropTable(event.PhysicalResourceId, tableAndClusterProps);
await dropTable(
event.PhysicalResourceId.includes(event.StackId.substring(event.StackId.length - 12)) ? tableName : event.PhysicalResourceId,
tableAndClusterProps,
);
return;
} else if (event.RequestType === 'Update') {
const tableName = await updateTable(
event.PhysicalResourceId,
const oldTableName = event.OldResourceProperties.tableName.prefix + getTableNameSuffix(event.OldResourceProperties.tableName.generateSuffix);
tableName = await updateTable(
event.PhysicalResourceId.includes(event.StackId.substring(event.StackId.length - 12)) ? oldTableName : event.PhysicalResourceId,
tableNamePrefix,
tableNameSuffix,
getTableNameSuffix(props.tableName.generateSuffix),
tableColumns,
useColumnIds,
tableAndClusterProps,
event.OldResourceProperties as TableAndClusterProps,
);
return { PhysicalResourceId: tableName };
return { PhysicalResourceId: event.PhysicalResourceId };
} else {
/* eslint-disable-next-line dot-notation */
throw new Error(`Unrecognized event type: ${event['RequestType']}`);
Expand Down Expand Up @@ -89,17 +94,13 @@ async function updateTable(
oldResourceProperties: TableAndClusterProps,
): Promise<string> {
const alterationStatements: string[] = [];
const newTableName = tableNamePrefix + tableNameSuffix;

const oldClusterProps = oldResourceProperties;
if (tableAndClusterProps.clusterName !== oldClusterProps.clusterName || tableAndClusterProps.databaseName !== oldClusterProps.databaseName) {
return createTable(tableNamePrefix, tableNameSuffix, tableColumns, tableAndClusterProps);
}

const oldTableNamePrefix = oldResourceProperties.tableName.prefix;
if (tableNamePrefix !== oldTableNamePrefix) {
return createTable(tableNamePrefix, tableNameSuffix, tableColumns, tableAndClusterProps);
}

const oldTableColumns = oldResourceProperties.tableColumns;
const columnDeletions = oldTableColumns.filter(oldColumn => (
tableColumns.every(column => {
Expand Down Expand Up @@ -208,6 +209,12 @@ async function updateTable(

await Promise.all(alterationStatements.map(statement => executeStatement(statement, tableAndClusterProps)));

const oldTableNamePrefix = oldResourceProperties.tableName.prefix;
if (tableNamePrefix !== oldTableNamePrefix) {
await executeStatement(`ALTER TABLE ${tableName} RENAME TO ${newTableName}`, tableAndClusterProps);
return tableNamePrefix + tableNameSuffix;
}

return tableName;
}

Expand Down
Loading