Skip to content

[FEATURE] Avoid using PIT search for simple query #3879

@qianheng-aws

Description

@qianheng-aws

Is your feature request related to a problem?
For some simple queries whose plan or optimized plan is only a scan operator, we should avoid PIT search if the expected retrieved rows count(i.e. QUERY_SIZE_LIMIT) is less than the max_result_window.

Examples for simple query:

source = table

// the filter and project can be push down
source = table | where ... | fields ...

What solution would you like?
Push down QUERY_SIZE_LIMIT into the final single scan operator.

What alternatives have you considered?
Add a limit operator on the top of the original plan and rely on Calcite optimizer to do push down. It has advantage that it will benefit some case where the plan is Project-Scan because of SortProjectTransposeRule, but the side effect is that it will change the plan by adding a limit operator.

Do you have any additional context?
Add any other context or screenshots about the feature request here.

Metadata

Metadata

Assignees

No one assigned

    Labels

    calcitecalcite migration releatedenhancementNew feature or requestpushdownpushdown related issues

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions