Fix incorrect $bucket for mismatch bucket queries #11885
Conversation
oraclechang
force-pushed
the
support_mismatch_bucket_query2
branch
from
3580a93 to
f78d3a1
Compare
| assertUpdate( | ||
| "CREATE TABLE test_mismatch_bucketing32\n" + | ||
| "WITH (bucket_count = 32, bucketed_by = ARRAY['key32']) AS\n" + | ||
| "SELECT custkey key32, comment value32 FROM orders", |
There was a problem hiding this comment.
Is this testable with smaller tables?
Like nation with bucketing on nationkey?
| assertQuery(withMismatchOptimization, query, "SELECT 130361"); | ||
| assertQuery(withoutMismatchOptimization, query, "SELECT 130361"); | ||
| } | ||
| finally { |
There was a problem hiding this comment.
nit: the try-finally isn't really needed in this class, since all the test data is ephemeral anyway
There was a problem hiding this comment.
Removed the try-finally
| 15000); | ||
|
|
||
| Session withMismatchOptimization = Session.builder(getSession()) | ||
| .setSystemProperty(COLOCATED_JOIN, "true") |
| .build(); | ||
|
|
||
| @Language("SQL") String query = "SELECT count(*) AS count\n" + | ||
| "FROM (\n" + |
There was a problem hiding this comment.
nit: \n are redundant, replace with spaces so that it's more readable
oraclechang
force-pushed
the
support_mismatch_bucket_query2
branch
2 times, most recently
from
3ce8645 to
185deca
Compare
|
Hi @arhimondr, Can you please have a look? |
| @@ -219,6 +219,14 @@ public ConnectorSplitSource getSplits( | |||
| // sort partitions | |||
| partitions = Ordering.natural().onResultOf(HivePartition::getPartitionId).reverse().sortedCopy(partitions); | |||
|
|
|||
| if (bucketHandle.isPresent()) { | |||
| if (bucketHandle.get().getReadBucketCount() > bucketHandle.get().getTableBucketCount()) { | |||
| throw new TrinoException( | |||
There was a problem hiding this comment.
Could you please open a follow up PR dropping the HivePartitioningHandle#maxCompatibleBucketCount. Currently it is effectively unused as number of read buckets higher than number of table buckets is no longer supported.
|
@oraclechang Could you please rebase and resubmit to make sure it compiles on Trunk. I will merge once the build is green. |
oraclechang
force-pushed
the
support_mismatch_bucket_query2
branch
from
185deca to
9adf088
Compare
Comparable change: prestodb/presto#12429
This change allows executing queries with the following conditions. Currently, these queries will raise an exception.
A more specific example can be seen in the
testMismatchedBucketWithBucketPredicate()test.