Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Panic when the number of partitions of the pipeline that throws the exception is inconsistent with the number of partitions output by the query #4096

Closed
yukkit opened this issue Nov 3, 2022 · 1 comment · Fixed by #4097
Labels
bug Something isn't working

Comments

@yukkit
Copy link
Contributor

yukkit commented Nov 3, 2022

Describe the bug

I am the developer of CnosDB, when we are doing integration testing, some SQL causes panic.

such as:

WITH gp AS (SELECT id FROM table1 GROUP BY id) 
SELECT
    COUNT(CAST(CAST(gp.id || '' AS TIMESTAMP) AS BIGINT)) 
FROM 
    gp;

To Reproduce

  1. physical plan
Plan: ProjectionExec: expr=[COUNT(gp.id || Utf8(""))@0 as COUNT(gp.id || Utf8(""))]
  AggregateExec: mode=Final, gby=[], aggr=[COUNT(gp.id || Utf8(""))]
    CoalescePartitionsExec
      AggregateExec: mode=Partial, gby=[], aggr=[COUNT(gp.id || Utf8(""))]
        ProjectionExec: expr=[id@0 as id]
          AggregateExec: mode=FinalPartitioned, gby=[id@0 as id], aggr=[]
            CoalesceBatchesExec: target_batch_size=4096
              RepartitionExec: partitioning=Hash([Column { name: "id", index: 0 }], 4)
                AggregateExec: mode=Partial, gby=[id@0 as id], aggr=[]
                  RepartitionExec: partitioning=RoundRobinBatch(4)
                    MemoryExec: partitions=2, partition_sizes=[20, 20]
  1. result
thread 'df-worker-1' panicked at 'index out of bounds: the len is 1 but the index is 2', datafusion/core/src/scheduler/task.rs:306:17
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
thread 'df-worker-2' panicked at 'index out of bounds: the len is 1 but the index is 1', datafusion/core/src/scheduler/task.rs:306:17
thread 'df-worker-3' panicked at 'index out of bounds: the len is 1 but the index is 3', datafusion/core/src/scheduler/task.rs:306:17
thread 'scheduler::tests::test_runtime_err' panicked at 'called `Result::unwrap()` on an `Err` value: ExternalError(ArrowError(ExternalError(ArrowError(CastError("Error parsing '2' as timestamp")))))', datafusion/core/src/scheduler/mod.rs:480:64
[2022-11-03T08:53:53Z ERROR datafusion::scheduler] worker 3 panicked with: index out of bounds: the len is 1 but the index is 3
[2022-11-03T08:53:53Z ERROR datafusion::scheduler] worker 2 panicked with: index out of bounds: the len is 1 but the index is 1

Expected behavior
no panic

Additional context

I am trying to fix this

@yukkit yukkit added the bug Something isn't working label Nov 3, 2022
@yukkit
Copy link
Contributor Author

yukkit commented Nov 3, 2022

I use the Scheduler to execute execution plan.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant