Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

CH Sorting Key: Introduce sorting key expression #2215

Closed
wants to merge 1 commit into from
Closed

CH Sorting Key: Introduce sorting key expression #2215

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 5 additions & 1 deletion flow/connectors/clickhouse/normalize.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -233,7 +233,11 @@ func getOrderedOrderByColumns(

orderbyColumns := make([]string, len(orderby))
for idx, col := range orderby {
orderbyColumns[idx] = getColName(colNameMap, col.SourceName)
sortingKeyName := col.SourceName
if col.SortingKeyExpression != nil && *col.SortingKeyExpression != "" {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

checking for both defeats the purpose of an optional string
either switch to just string or ensure you don't send empty string from upstream

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I prefer not using optional

sortingKeyName = *col.SortingKeyExpression
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lack of validation means this opens up using ; to run an arbitrary query. Hard to say what our threat model is, but we should explicitly get the okay that our threat model assumes anyone with access to ui has access to run arbitrary queries with any clickhouse peer

}
orderbyColumns[idx] = getColName(colNameMap, sortingKeyName)
}

return orderbyColumns
Expand Down
1 change: 1 addition & 0 deletions protos/flow.proto
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@ message ColumnSetting {
string destination_type = 3;
int32 ordering = 4;
bool nullable_enabled = 5;
optional string sorting_key_expression = 6;
}

message TableMapping {
Expand Down
Loading