Skip to content

Conversation

alamb
Copy link
Contributor

@alamb alamb commented Feb 11, 2025

Which issue does this PR close?

Rationale for this change

These tests are failing on main due to out of disk space (see #14576)

Here is an example: https://github.com/apache/datafusion/actions/runs/13261520752/job/37019301867
Screenshot 2025-02-11 at 6 49 44 AM

What changes are included in this PR?

  1. Disable failing tests

Are these changes tested?

No

Are there any user-facing changes?

Hopefully we'll get the tests passing again on main

@github-actions github-actions bot added the development-process Related to development process of DataFusion label Feb 11, 2025
# # Run extended tests (with feature 'extended_tests')
# # Disabling as it is running out of disk space
# # see https://github.com/apache/datafusion/issues/14576
# linux-test-extended:
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This test was added by @2010YOUY01 in

It started failing after

I suspect something about the test is using too much disk space

Note when I ran

cargo test --profile ci --exclude datafusion-examples --exclude datafusion-benchmarks --workspace --lib --tests --bins --features avro,json,backtrace,extended_tests

Locally, my target directory was 4GB

However, after extended_tests ran, the target directory was 14GB

$ du -s -h target
 14G	target
 14G	target

I suspect what is happening is that sort mem validation call is somehow recompiling the entire workspace

let output = Command::new("cargo")
.arg("test")
.arg("--package")
.arg("datafusion")
.arg("--test")
.arg("core_integration")
.arg("--features")
.arg("extended_tests")
.arg("--")
.arg(&test_path)
.arg("--exact")
.arg("--nocapture")
.env("DATAFUSION_TEST_MEM_LIMIT_VALIDATION", "1")
.output()
.expect("Failed to execute test command");

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the extended-tests is the one which runs the sqllogic tests?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nvm, you commented the other part of extended tests

@alamb alamb marked this pull request as ready for review February 11, 2025 11:54
@Omega359
Copy link
Contributor

lgtm

Copy link
Contributor

@comphead comphead left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm thanks @alamb

@comphead comphead merged commit cb5d42e into apache:main Feb 11, 2025
25 checks passed
@alamb alamb deleted the alamb/fix_etended branch February 11, 2025 20:26
@alamb
Copy link
Contributor Author

alamb commented Feb 11, 2025

Thanks @Omega359 and @comphead

@alamb
Copy link
Contributor Author

alamb commented Feb 12, 2025

It seems to have worked:
cb5d42e

Screenshot 2025-02-11 at 7 58 15 PM

middleparrot

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

development-process Related to development process of DataFusion

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants