Skip to content
This repository has been archived by the owner on Jan 28, 2021. It is now read-only.

Partitions: Rules to apply Exchange operator into query trees #313

Closed
ajnavarro opened this issue Aug 1, 2018 · 3 comments
Closed

Partitions: Rules to apply Exchange operator into query trees #313

ajnavarro opened this issue Aug 1, 2018 · 3 comments
Assignees
Labels
enhancement New feature or request
Milestone

Comments

@ajnavarro
Copy link
Contributor

The following actions should be implemented as analyzer rules to be able to parallelize queries:

  • If parallelization is > 1, we can introduce an Exchange node, if not, the rule is doing nothing.

  • Exchange nodes should be added on top of much nodes as possible.

  • We stop adding nodes inside Exchange node when we found a binary node or the last node with no parents. We must also stop adding nodes inside Exchange node if we find an Aggregation. In the future, we can parallelize that kind of nodes too.

  • We cannot add Exchange nodes on top of other Exchange nodes, but we can have several Exchange nodes on siblings (see example graph).

  • Exchange nodes must finish always in a table.

@ajnavarro ajnavarro added the enhancement New feature or request label Aug 1, 2018
@ajnavarro ajnavarro added this to the partitions milestone Aug 1, 2018
@mcarmonaa mcarmonaa changed the title Partitions: Rules to apply Exange operator into query trees Partitions: Rules to apply Exchange operator into query trees Aug 1, 2018
@erizocosmico
Copy link
Contributor

erizocosmico commented Aug 13, 2018

Depends on #312 and #309

@erizocosmico erizocosmico added the blocked Some other issue is blocking this label Aug 13, 2018
@erizocosmico erizocosmico self-assigned this Aug 21, 2018
@erizocosmico
Copy link
Contributor

@ajnavarro parallelism is configurable or will we use a hardcoded value?

@ajnavarro
Copy link
Contributor Author

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants