[CI] Investigate stack overflow errors on TPC-DS queries

[comphead]

Is your feature request related to a problem or challenge?

In Datafusion CI builds we spontaneously hit the Rust stack overflow errors on TPC-DS queries. The most notorious is

thread 'tpcds_physical_q54' has overflowed its stack

As a workaround the RUST_MIN_STACK = 3000000 applied for CI builds, but it might be a bigger problem behind that

Describe the solution you'd like

Investigate the reason for hitting stack overflow errors

Describe alternatives you've considered

No response

Additional context

No response

[Jefffrey]

Leaving note here, that part of this work should involve removing/cleaning up occurrences of RUST_MIN_STACK setting from CI files, e.g. https://github.com/apache/arrow-datafusion/blob/2b218be67a6c412629530b812836a6cec76efc32/.github/actions/setup-rust-runtime/action.yaml#L39

Also here's a PR that dealt with the issue in the past, in case it might be helpful: #4779

[mustafasrepo]

For the query tpcds_physical_q64 It seems that overflow occurs in create_intial_plan stage of planner (It is the place, where LogicalPlan is converted to the PhysicalPlan. Not during any physical rule or logical plan creation). Examining code structure, it seems that recursive futures are constructed in the create_initial_plan method of the LogicalPlan. I am not familiar with its implications, However the problem might be related to this behavior. Writing this in case it is helpful for others.