Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[POC] Fuse operations in equal_rows_arr #13607

Draft
wants to merge 10 commits into
base: main
Choose a base branch
from

Conversation

LeslieKid
Copy link
Contributor

Which issue does this PR close?

Closes #12131 .

Rationale for this change

What changes are included in this PR?

Are these changes tested?

Are there any user-facing changes?

@LeslieKid
Copy link
Contributor Author

LeslieKid commented Nov 29, 2024

Major changes currently:

  • Compare arrays with indices (in a for loop) without take+eq.
  • Update a single boolean buffer instead of create a new one every time.

I am working on:

  • Compare nulls and values separately.
  • Avoid using Filter.
  • Support more types.
  • Add benchmarks for join on more data types. The tpch benchmarks mainly focus on primitive types.

@LeslieKid
Copy link
Contributor Author

The current benchmark results on my computer are:

--------------------
Benchmark tpch_mem_sf1.json
--------------------
┏━━━━━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Query        ┃     main ┃ hash-join ┃        Change ┃
┡━━━━━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ QQuery 1     │  88.79ms │   85.61ms │     no change │
│ QQuery 2     │  13.24ms │   13.14ms │     no change │
│ QQuery 3     │  21.88ms │   22.28ms │     no change │
│ QQuery 4     │  13.12ms │   13.36ms │     no change │
│ QQuery 5     │  42.65ms │   40.32ms │ +1.06x faster │
│ QQuery 6     │   4.79ms │    4.77ms │     no change │
│ QQuery 7     │  81.89ms │   80.93ms │     no change │
│ QQuery 8     │  16.44ms │   16.56ms │     no change │
│ QQuery 9     │  45.76ms │   45.13ms │     no change │
│ QQuery 10    │  41.66ms │   41.02ms │     no change │
│ QQuery 11    │   5.65ms │    6.05ms │  1.07x slower │
│ QQuery 12    │  26.32ms │   26.35ms │     no change │
│ QQuery 13    │  18.90ms │   19.65ms │     no change │
│ QQuery 14    │   5.07ms │    5.09ms │     no change │
│ QQuery 15    │  11.67ms │   11.69ms │     no change │
│ QQuery 16    │  12.68ms │   12.57ms │     no change │
│ QQuery 17    │  63.97ms │   63.23ms │     no change │
│ QQuery 18    │ 148.73ms │  148.57ms │     no change │
│ QQuery 19    │  23.53ms │   23.29ms │     no change │
│ QQuery 20    │  23.16ms │   23.25ms │     no change │
│ QQuery 21    │ 101.53ms │   99.36ms │     no change │
│ QQuery 22    │  15.41ms │   15.11ms │     no change │
└──────────────┴──────────┴───────────┴───────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━┓
┃ Benchmark Summary        ┃          ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━┩
│ Total Time (main)        │ 826.83ms │
│ Total Time (hash-join)   │ 817.33ms │
│ Average Time (main)      │  37.58ms │
│ Average Time (hash-join) │  37.15ms │
│ Queries Faster           │        1 │
│ Queries Slower           │        1 │
│ Queries with No Change   │       20 │
└──────────────────────────┴──────────┘
--------------------
Benchmark tpch_sf1.json
--------------------
┏━━━━━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Query        ┃     main ┃ hash-join ┃        Change ┃
┡━━━━━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ QQuery 1     │ 115.67ms │  113.55ms │     no change │
│ QQuery 2     │  20.18ms │   20.85ms │     no change │
│ QQuery 3     │  40.90ms │   42.72ms │     no change │
│ QQuery 4     │  27.69ms │   26.85ms │     no change │
│ QQuery 5     │  65.46ms │   65.07ms │     no change │
│ QQuery 6     │  19.83ms │   20.36ms │     no change │
│ QQuery 7     │  92.16ms │   87.85ms │     no change │
│ QQuery 8     │  59.73ms │   57.81ms │     no change │
│ QQuery 9     │  80.80ms │   80.41ms │     no change │
│ QQuery 10    │  71.76ms │   68.15ms │ +1.05x faster │
│ QQuery 11    │  13.70ms │   14.08ms │     no change │
│ QQuery 12    │  41.34ms │   40.55ms │     no change │
│ QQuery 13    │  41.06ms │   39.47ms │     no change │
│ QQuery 14    │  30.83ms │   29.78ms │     no change │
│ QQuery 15    │  49.08ms │   50.25ms │     no change │
│ QQuery 16    │  14.89ms │   15.39ms │     no change │
│ QQuery 17    │ 105.27ms │  102.18ms │     no change │
│ QQuery 18    │ 140.74ms │  137.79ms │     no change │
│ QQuery 19    │  54.41ms │   51.36ms │ +1.06x faster │
│ QQuery 20    │  46.00ms │   43.53ms │ +1.06x faster │
│ QQuery 21    │ 108.36ms │  108.11ms │     no change │
│ QQuery 22    │  17.36ms │   17.62ms │     no change │
└──────────────┴──────────┴───────────┴───────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃ Benchmark Summary        ┃           ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ Total Time (main)        │ 1257.23ms │
│ Total Time (hash-join)   │ 1233.75ms │
│ Average Time (main)      │   57.15ms │
│ Average Time (hash-join) │   56.08ms │
│ Queries Faster           │         3 │
│ Queries Slower           │         0 │
│ Queries with No Change   │        19 │
└──────────────────────────┴───────────┘

I'm not sure if the test results on my computer are accurate. Can you help for running the benchmarks on your machines? @Dandandan @Rachelint

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
physical-expr Physical Expressions
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Fuse operations in equal_rows_arr
1 participant