Skip to content

[Core] Deflake test_put_out_of_disk #56442

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
jjyao merged 23 commits into from
Nov 25, 2025
Merged

[Core] Deflake test_put_out_of_disk #56442

jjyao merged 23 commits into from
Nov 25, 2025

Conversation

@jjyao
Copy link
Collaborator

@jjyao jjyao commented Sep 11, 2025

Why are these changes needed?

Related issue number

Checks

  • I've signed off every commit(by using the -s flag, i.e., git commit -s) in this PR.
  • I've run scripts/format.sh to lint the changes in this PR.
  • I've included any doc changes needed for https://docs.ray.io/en/master/.
    • I've added any new APIs to the API Reference. For example, if I added a
      method in Tune, I've added it in doc/source/tune/api/ under the
      corresponding .rst file.
  • I've made sure the tests are passing. Note that there might be a few flaky tests, see the recent failures at https://flakey-tests.ray.io/
  • Testing Strategy
    • Unit tests
    • Release tests
    • This PR is not tested :(

jjyao added 17 commits September 10, 2025 22:08
@jjyao
up
62bccfc 
Signed-off-by: Jiajun Yao <jeromeyjj@gmail.com>
@jjyao
Merge branch 'master' of github.com:ray-project/ray into jjyao/bisecttt
6da3384
@jjyao
debug
83dad27 
Signed-off-by: Jiajun Yao <jeromeyjj@gmail.com>
@jjyao
debug
a36edd0 
Signed-off-by: Jiajun Yao <jeromeyjj@gmail.com>
@jjyao
up
a9564d3 
Signed-off-by: Jiajun Yao <jeromeyjj@gmail.com>
@jjyao
up
6520d39 
Signed-off-by: Jiajun Yao <jeromeyjj@gmail.com>
@jjyao
up
89ffd57 
Signed-off-by: Jiajun Yao <jeromeyjj@gmail.com>
@jjyao
up
9f6fec3 
Signed-off-by: Jiajun Yao <jeromeyjj@gmail.com>
@jjyao
up
4918b62 
Signed-off-by: Jiajun Yao <jeromeyjj@gmail.com>
@jjyao
up
97005dd 
Signed-off-by: Jiajun Yao <jeromeyjj@gmail.com>
@jjyao
up
63e4c7e 
Signed-off-by: Jiajun Yao <jeromeyjj@gmail.com>
@jjyao
pu
a49af03 
Signed-off-by: Jiajun Yao <jeromeyjj@gmail.com>
@jjyao
up
29c8456 
Signed-off-by: Jiajun Yao <jeromeyjj@gmail.com>
@jjyao
up
bd855c5 
Signed-off-by: Jiajun Yao <jeromeyjj@gmail.com>
@jjyao
up
54bf744 
Signed-off-by: Jiajun Yao <jeromeyjj@gmail.com>
@jjyao
up
90f3dd5 
Signed-off-by: Jiajun Yao <jeromeyjj@gmail.com>
@jjyao
revert
3c8bbdc 
Signed-off-by: Jiajun Yao <jeromeyjj@gmail.com>
@github-actions
Copy link

github-actions bot commented Oct 17, 2025

This pull request has been automatically marked as stale because it has not had
any activity for 14 days. It will be closed in another 14 days if no further activity occurs.
Thank you for your contributions.

You can always ask for help on our discussion forum or Ray's public slack channel.

If you'd like to keep this open, just leave any comment, and the stale label will be removed.

@github-actions github-actions bot added the stale The issue is stale. It will be closed within 7 days unless there are further conversation label Oct 17, 2025
@github-actions github-actions bot added unstale A PR that has been marked unstale. It will not get marked stale again if this label is on it. and removed stale The issue is stale. It will be closed within 7 days unless there are further conversation labels Oct 26, 2025
@jjyao jjyao added the go add ONLY when ready to merge, run all tests label Nov 24, 2025
jjyao added 2 commits November 24, 2025 15:41
@jjyao
up
b37eb8a 
Signed-off-by: Jiajun Yao <jeromeyjj@gmail.com>
@jjyao
Deflake test_put_out_of_disk
4ed73dd 
Signed-off-by: Jiajun Yao <jeromeyjj@gmail.com>
@jjyao jjyao changed the title Bisect [Core] Deflake test_put_out_of_disk Nov 25, 2025
jjyao
jjyao commented Nov 25, 2025
View reviewed changes
python/ray/tests/test_out_of_disk_space.py

assert get_current_usage() < local_fs_capacity_threshold
time.sleep(1)
wait_for_condition(lambda: get_current_usage() < local_fs_capacity_threshold)
Copy link
Collaborator Author

@jjyao jjyao Nov 25, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On the CI machine, the get_current_usage() is not immediately updated after the tmp_file is deleted for some reason so I'm using wait_for_condition() here to give it some time to update it.

Copy link
Contributor

@Sparks0219 Sparks0219 Nov 25, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you leave a comment on get_current_usage that it could be potentially flaky due to file size not being immediately updated + should be used with wait_for_condition?

Copy link
Collaborator Author

@jjyao jjyao Nov 25, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added a comment.

Copy link
Contributor

@Sparks0219 Sparks0219 Nov 25, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cool thanks

@jjyao jjyao marked this pull request as ready for review November 25, 2025 18:57
@jjyao jjyao requested a review from a team as a code owner November 25, 2025 18:57
@jjyao jjyao removed the unstale A PR that has been marked unstale. It will not get marked stale again if this label is on it. label Nov 25, 2025
@ray-gardener ray-gardener bot added the core Issues that should be addressed in Ray Core label Nov 25, 2025
@jjyao
up
2a709b3 
Signed-off-by: Jiajun Yao <jeromeyjj@gmail.com>
@jjyao jjyao requested a review from Sparks0219 November 25, 2025 20:40
Sparks0219
Sparks0219 approved these changes Nov 25, 2025
View reviewed changes
Copy link
Contributor

@Sparks0219 Sparks0219 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 🚢

python/ray/tests/test_out_of_disk_space.py

assert get_current_usage() < local_fs_capacity_threshold
time.sleep(1)
wait_for_condition(lambda: get_current_usage() < local_fs_capacity_threshold)
Copy link
Contributor

@Sparks0219 Sparks0219 Nov 25, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cool thanks

@jjyao jjyao merged commit 3d0f004 into ray-project:master Nov 25, 2025
5 checks passed
@jjyao jjyao deleted the jjyao/bisecttt branch November 25, 2025 21:54
KaisennHu pushed a commit to KaisennHu/ray that referenced this pull request Nov 26, 2025
@jjyao @KaisennHu
[Core] Deflake test_put_out_of_disk (ray-project#56442)
d4d20f5 
Signed-off-by: Jiajun Yao <jeromeyjj@gmail.com>
SheldonTsen pushed a commit to SheldonTsen/ray that referenced this pull request Dec 1, 2025
@jjyao @SheldonTsen
[Core] Deflake test_put_out_of_disk (ray-project#56442)
4e7e347 
Signed-off-by: Jiajun Yao <jeromeyjj@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@Sparks0219 Sparks0219 Sparks0219 approved these changes

Assignees

No one assigned

Labels

core Issues that should be addressed in Ray Core go add ONLY when ready to merge, run all tests

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@jjyao @Sparks0219