Skip to content

Cannot Count(Expr:Wildcard) with DataFrame API #5473

@Jefffrey

Description

@Jefffrey

Describe the bug
A clear and concise description of what the bug is.

Cannot do count(Expr::Wildcard) aggregate with DataFrame API

To Reproduce
Steps to reproduce the behavior:

    ctx.table("alltypes_plain")
        .await?
        .aggregate(vec![], vec![count(Expr::Wildcard)])?
        .explain(false, false)?
        .show()
        .await?;

Throws error:

Error: Internal("Wildcard expressions are not valid in a logical query plan")

Expected behavior
A clear and concise description of what you expected to happen.

Should be like when doing SQL:

    ctx.sql("select count(*) from alltypes_plain")
        .await?
        .explain(false, false)?
        .show()
        .await?;

Gives plan:

+---------------+---------------------------------------------------+
| plan_type     | plan                                              |
+---------------+---------------------------------------------------+
| logical_plan  | Aggregate: groupBy=[[]], aggr=[[COUNT(UInt8(1))]] |
|               |   TableScan: alltypes_plain projection=[id]       |
| physical_plan | ProjectionExec: expr=[7300 as COUNT(UInt8(1))]    |
|               |   EmptyExec: produce_one_row=true                 |
|               |                                                   |
+---------------+---------------------------------------------------+

Where count(*) has been transformed to count(1) essentially.

Additional context
Add any other context about the problem here.

Expansion substitute:

https://github.com/apache/arrow-datafusion/blob/e9852074bacd8c891d84eba38b3417aa16a2d18c/datafusion/expr/src/utils.rs#L43-L45

See SQL substitute function:

https://github.com/apache/arrow-datafusion/blob/e9852074bacd8c891d84eba38b3417aa16a2d18c/datafusion/sql/src/expr/function.rs#L193-L218

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions