Skip to content

Commit 7f75e58

Browse files
2010YOUY01alamb
2010YOUY01
and
alamb
authored
perf: Fix NLJ slow join with condition array_has (apache#18161)
## Which issue does this PR close? <!-- We generally require a GitHub issue to be filed for all bug fixes and enhancements and this helps us generate change logs for our releases. You can link an issue to this PR using the GitHub syntax. For example `Closes #123` indicates that this PR will close issue #123. --> - Closes apache#18070 ## Rationale for this change <!-- Why are you proposing this change? If this is already explained clearly in the issue then this section is not needed. Explaining clearly why changes are proposed helps reviewers understand your changes and offer better suggestions for fixes. --> See the above issue and its comment apache#18070 (comment) ## What changes are included in this PR? <!-- There is no need to duplicate the description in the issue here but it is sometimes worth providing a summary of the individual changes in this PR. --> In nested loop join, when the join column includes `List(Utf8View)`, use `take()` instead of `to_array_of_size()` to avoid deep copying the utf8 buffers inside `Utf8View` array. This is the quick fix, avoiding deep copy inside `to_array_of_size()` is a bit tricky. Here is `ListArray`'s physical layout: https://arrow.apache.org/rust/arrow/array/struct.GenericListArray.html If multiple elements is pointing to the same list range, the underlying payload can't be reused.So the potential fix in `to_array_of_size` can only avoids copying the inner-inner utf8view array buffers, but can't avoid copying the inner array (i.e. views are still copied), and deep copying for other primitive types also can't be avoided. Seems this can be better solved when `ListView` type is ready 🤔 ### Benchmark I tried query 1 in apache#18070, but only used 3 randomly sampled `places` parquet file. 49.0.0: 4s 50.0.0: stuck > 1 minute PR: 4s Now the performance are similar, I suspect the most time is spend evaluating the expensive `array_has` so the optimization in apache#16996 can't help much. ## Are these changes tested? <!-- We typically require tests for all PRs in order to: 1. Prevent the code from being accidentally broken by subsequent changes 2. Serve as another way to document the expected behavior of the code If tests are not included in your PR, please explain why (for example, are they covered by existing tests)? --> Existing tests ## Are there any user-facing changes? <!-- If there are user-facing changes then we may require documentation to be updated before approving the PR. --> No <!-- If there are any breaking changes to public APIs, please add the `api change` label. --> --------- Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
1 parent 7c215ed commit 7f75e58

File tree

2 files changed

+92
-6
lines changed

2 files changed

+92
-6
lines changed

‎datafusion/physical-plan/src/joins/nested_loop_join.rs‎

Lines changed: 29 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -48,11 +48,15 @@ use crate::{
4848

4949
use arrow::array::{
5050
new_null_array, Array, BooleanArray, BooleanBufferBuilder, RecordBatchOptions,
51+
UInt64Array,
5152
};
5253
use arrow::buffer::BooleanBuffer;
53-
use arrow::compute::{concat_batches, filter, filter_record_batch, not, BatchCoalescer};
54+
use arrow::compute::{
55+
concat_batches, filter, filter_record_batch, not, take, BatchCoalescer,
56+
};
5457
use arrow::datatypes::{Schema, SchemaRef};
5558
use arrow::record_batch::RecordBatch;
59+
use arrow_schema::DataType;
5660
use datafusion_common::cast::as_boolean_array;
5761
use datafusion_common::{
5862
arrow_err, internal_datafusion_err, internal_err, project_schema,
@@ -1661,11 +1665,30 @@ fn build_row_join_batch(
16611665
// Broadcast the single build-side row to match the filtered
16621666
// probe-side batch length
16631667
let original_left_array = build_side_batch.column(column_index.index);
1664-
let scalar_value = ScalarValue::try_from_array(
1665-
original_left_array.as_ref(),
1666-
build_side_index,
1667-
)?;
1668-
scalar_value.to_array_of_size(filtered_probe_batch.num_rows())?
1668+
// Avoid using `ScalarValue::to_array_of_size()` for `List(Utf8View)` to avoid
1669+
// deep copies for buffers inside `Utf8View` array. See below for details.
1670+
// https://github.com/apache/datafusion/issues/18159
1671+
//
1672+
// In other cases, `to_array_of_size()` is faster.
1673+
match original_left_array.data_type() {
1674+
DataType::List(field) | DataType::LargeList(field)
1675+
if field.data_type() == &DataType::Utf8View =>
1676+
{
1677+
let indices_iter = std::iter::repeat_n(
1678+
build_side_index as u64,
1679+
filtered_probe_batch.num_rows(),
1680+
);
1681+
let indices_array = UInt64Array::from_iter_values(indices_iter);
1682+
take(original_left_array.as_ref(), &indices_array, None)?
1683+
}
1684+
_ => {
1685+
let scalar_value = ScalarValue::try_from_array(
1686+
original_left_array.as_ref(),
1687+
build_side_index,
1688+
)?;
1689+
scalar_value.to_array_of_size(filtered_probe_batch.num_rows())?
1690+
}
1691+
}
16691692
} else {
16701693
// Take the filtered probe-side column using compute::take
16711694
Arc::clone(filtered_probe_batch.column(column_index.index))

‎datafusion/sqllogictest/test_files/join_lists.slt‎

Lines changed: 63 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,63 @@
1+
# Licensed to the Apache Software Foundation (ASF) under one
2+
# or more contributor license agreements. See the NOTICE file
3+
# distributed with this work for additional information
4+
# regarding copyright ownership. The ASF licenses this file
5+
# to you under the Apache License, Version 2.0 (the
6+
# "License"); you may not use this file except in compliance
7+
# with the License. You may obtain a copy of the License at
8+
9+
# http://www.apache.org/licenses/LICENSE-2.0
10+
11+
# Unless required by applicable law or agreed to in writing,
12+
# software distributed under the License is distributed on an
13+
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
14+
# KIND, either express or implied. See the License for the
15+
# specific language governing permissions and limitations
16+
# under the License.
17+
18+
19+
20+
## Ensure test coverage for NLJ using joining on LISTS
21+
22+
## Reproducer for https://github.com/apache/datafusion/issues/18070
23+
24+
statement ok
25+
CREATE TABLE categories_raw
26+
AS SELECT arrow_cast('cat_' || value, 'Utf8View') AS category_id FROM generate_series(1, 5);
27+
28+
statement ok
29+
CREATE TABLE places
30+
AS SELECT column1 as id, column2 as fsq_category_ids, column3 as date_refreshed
31+
FROM VALUES
32+
(1, ['cat_1', 'cat_2', 'cat_3'], DATE '2023-05-10'),
33+
(2, ['cat_4', 'cat_5'], DATE '2021-12-01'),
34+
(3, ['cat_6', 'cat_7', 'cat_8', 'cat_9'], DATE '2024-01-15'); --> NOTE these categories do not exist in categories_raw
35+
36+
37+
query I
38+
WITH categories_arr AS (
39+
SELECT array_agg(category_id) AS category_ids FROM categories_raw LIMIT 500
40+
)
41+
SELECT COUNT(*)
42+
FROM places p
43+
WHERE array_has_any(p.fsq_category_ids, (SELECT category_ids FROM categories_arr));
44+
----
45+
2
46+
47+
query I
48+
WITH categories_arr AS (
49+
SELECT array_agg(category_id) AS category_ids FROM categories_raw LIMIT 500
50+
)
51+
SELECT COUNT(*)
52+
FROM places p
53+
WHERE id <> 1 AND array_has_any(p.fsq_category_ids, (SELECT category_ids FROM categories_arr));
54+
----
55+
1
56+
57+
# cleanup
58+
statement ok
59+
DROP TABLE categories_raw;
60+
61+
statement ok
62+
DROP TABLE places;
63+

0 commit comments

Comments
 (0)