perf(553): Optimize incremental join evaluation #557
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Centril
reviewed
Nov 14, 2023
joshua-spacetime
force-pushed
the
joshua/perf/553/incremental-join-eval
branch
2 times, most recently
from
12b5690 to
dee576c
Compare
Centril
reviewed
Nov 14, 2023
joshua-spacetime
force-pushed
the
joshua/perf/553/incremental-join-eval
branch
2 times, most recently
from
284764d to
65a5588
Compare
joshua-spacetime
force-pushed
the
joshua/perf/553/incremental-join-eval
branch
from
65a5588 to
1b3b402
Compare
Fixes #553. Before this change, we would use the same plan for both query and incremental eval. This is problematic for index joins. In particular, table sizes are drastically different under incremental eval. After this change, joins are reordered for incremental eval.
joshua-spacetime
force-pushed
the
joshua/perf/553/incremental-join-eval
branch
from
1b3b402 to
13b9bc7
Compare
mamcx
approved these changes
Nov 15, 2023
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fixes #553.
Before this change, we would use the same query plan on both subscription initialization as well as incremental evaluation.
This is problematic for left index joins that read from the right table and probe the index of the left table.
In particular, updates to the left table will use the same query plan, which means they will read the entire right table.
After this change, updates to the left table will only read those updates and probe the index of the right table.
Description of Changes
Please describe your change, mention any related tickets, and so on here.
API and ABI breaking changes
If this is an API or ABI breaking change, please apply the
corresponding GitHub label.
Expected complexity level and risk
How complicated do you think these changes are? Grade on a scale from 1 to 5,
where 1 is a trivial change, and 5 is a deep-reaching and complex change.
This complexity rating applies not only to the complexity apparent in the diff,
but also to its interactions with existing and future code.
If you answered more than a 2, explain what is complex about the PR,
and what other components it interacts with in potentially concerning ways.