Skip to content

Create Intermediate Representation Between arroyo-sql::SqlOperator and arroyo-datastream::Program #66

@jacksonrnewhouse

Description

@jacksonrnewhouse

Currently this is translated in a single call to get_program_from_operator().

However, this code path is overloaded, responsible for code generation, creation of the operator graph, edge construction and several in-lined optimizations. This results in unclear, complex code.

For this issue, the task is to define another representation, SqlPlanGraph which can be more directly translated to the Program.

Before

[SqlOperator] - (graph construction, code gen, optimization) -> [Program].

After

[SqlOperator] -(graph construction) -> [SqlPlanGraph] - (optimizations) -> [SqlPlanGraph] - (code gen) -> [Program]

Metadata

Metadata

Labels

sqlRelated to the DataFusion SQL integration

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions