feat(optimizer): column pruning framework #420
Conversation
Signed-off-by: st1page <1245835950@qq.com>
Signed-off-by: st1page <1245835950@qq.com>
| @@ -42,6 +44,8 @@ impl Optimizer { | |||
| } | |||
| let hep_optimizer = HeuristicOptimizer { rules }; | |||
| plan = hep_optimizer.optimize(plan); | |||
| let out_types_num = plan.out_types().len(); | |||
| plan = plan.prune_col(BitSet::from_iter(0..out_types_num)); | |||
There was a problem hiding this comment.
I think we already have bitvec, and there's no need to import another bit-vec-like crate...
There was a problem hiding this comment.
Here we might need a collection which is more like a "Set", Maybe we will change it to fixedbitset
| }) | ||
| }) | ||
| .collect(); | ||
| LogicalProjection::new(exprs, self.clone_as_plan_ref()).into_plan_ref() |
There was a problem hiding this comment.
Simply doing projection won't save I/Os. Will we have plan rules to merge LogicalProjection with TableScan, and merge LogicalProjection with LogicalProjection?
There was a problem hiding this comment.
Here we simply doing projection to maintain the correct behavior of the fn prune_col. in future, every PlanNode should implement its own prune_col method, which can call its inputs' prune_col recursively. in the end, when call the prune_col of the TableScan Node, it will remove the columns not required.
add
prune_colmethod onPlanNode, every logical node should implementfn prune_col(&self, required_cols: BitSet) -> PlanRef, which will optimize itself and output only the required columns. Now there is a default implementation which is insert aLogialProjectionon the top of the Node.ref #388