Skip to content

Commit

Permalink
feat: emitting partial join results in HashJoinStream (#8020)
Browse files Browse the repository at this point in the history
* HashJoin partial batch emitting

* batch splitting tests

* stream state & extended tests

* fmt & clippy warns fixed

* Apply suggestions from code review

Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>

* review comments

* ported join limited output

* comments & formatting

* Reuse hashes buffer

* Apply suggestions from code review

Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>

* fixed metrics and updated comment

* precalculate hashes & remove iterators

* draft: preparing for review

---------

Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
  • Loading branch information
korowa and alamb authored Jan 22, 2024
1 parent 38d5f75 commit f2e6701
Show file tree
Hide file tree
Showing 7 changed files with 736 additions and 284 deletions.
1 change: 1 addition & 0 deletions datafusion/physical-plan/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -61,6 +61,7 @@ uuid = { version = "^1.2", features = ["v4"] }

[dev-dependencies]
rstest = { workspace = true }
rstest_reuse = "0.6.0"
termtree = "0.4.1"
tokio = { version = "1.28", features = [
"macros",
Expand Down
Loading

0 comments on commit f2e6701

Please sign in to comment.