extended_test (with memory limit tracking) are commented out

[alamb]

Describe the bug

• CI tests were failing on main: Extended tests are (still) failing on main #14576

I commented out the extended_test in

• Disable extended tests (extended_tests) that are failing on runner #14604

This test was added by @2010YOUY01 in

• Test: Validate memory limit for sort queries to extended test #14142

It started failing after

• Add Cargo.lock #14483

There is more detail in this comment

• Disable extended tests (extended_tests) that are failing on runner #14604 (comment)

To Reproduce

No response

Expected behavior

I would like the extended_test to be fixed so it runs regularly or is removed

Having a commented out / unrun test in the codebase is not adding any value

Additional context

No response

[2010YOUY01]

This test is failing because it runs out of disk space, I've checked the remaining disk space after Github's CI runner has setup the rust toolchain and before running any test: It's 12GB disk space left.
The failing extended_test requires 11G, and the normal test requires 7G according to my local experiments. This disk space budget is quite tight IMO.

I'm trying to find is there any good way to free up more disk space, It's quite hard since now CI jobs are running inside the container.

datafusion/.github/workflows/extended.yml

Lines 62 to 63 in 50c3547

	# container:
	# image: amd64/rust

I'm wondering is it possible not to run inside docker, and manually setup rust dependency, and this way we can delete some unused files and free up disk space.

[alamb]

I'm wondering is it possible not to run inside docker, and manually setup rust dependency, and this way we can delete some unused files and free up disk space.

Yeah, and figuring out how to reduce the disk space usage in general might be a good idea too (maybe we can make a separate binary for extended tests, for example 🤔 )

[Omega359]

We already run with the CI profile which reduces binary size iirc.

[2010YOUY01]

I'm wondering is it possible not to run inside docker, and manually setup rust dependency, and this way we can delete some unused files and free up disk space.

Yeah, and figuring out how to reduce the disk space usage in general might be a good idea too (maybe we can make a separate binary for extended tests, for example 🤔 )

This is a good idea to let extended tests run in a separate integration test binary.
I did a PR to fix the CI by freeing up disk spaces, which I think can also help in the long run, so perhaps this separating binary can be done as a follow up PR.