Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[CI] linux://python/ray/tests:test_placement_group is failing/flaky on master. #38726

Closed
rkooo567 opened this issue Aug 22, 2023 · 2 comments
Closed

[CI] linux://python/ray/tests:test_placement_group is failing/flaky on master. #38726

rkooo567 opened this issue Aug 22, 2023 · 2 comments
Labels
core Issues that should be addressed in Ray Core core-client ray client related issues flaky-tracker Issue created via Flaky Test Tracker https://flaky-tests.ray.io/ P1 Issue that should be fixed within a few weeks ray 2.10

Comments

@rkooo567
Copy link
Contributor

....
Generated from flaky test tracker. Please do not edit the signature in this section.
DataCaseName-linux://python/ray/tests:test_placement_group-END
....
@rkooo567 rkooo567 added P1 Issue that should be fixed within a few weeks core Issues that should be addressed in Ray Core flaky-tracker Issue created via Flaky Test Tracker https://flaky-tests.ray.io/ labels Aug 22, 2023
@rkooo567 rkooo567 added P0 Issues that should be fixed in short order ray 2.8 and removed P1 Issue that should be fixed within a few weeks labels Oct 4, 2023
@vitsai
Copy link
Contributor

vitsai commented Oct 18, 2023

Took a quick look at this. ray.get() times out due to lock contention during auto-init when ray client connection is enabled. So, for some reason, ray.init is taking longer than 3 minutes.

One specific failure here (test_placement_ready): https://buildkite.com/ray-project/oss-ci-build-branch/builds/6484

@jjyao jjyao added the release-blocker P0 Issue that blocks the release label Oct 21, 2023
@rkooo567
Copy link
Contributor Author

Is there a PR now? If ^ is the root cause, I think we dont need to mark it as a blocker (let's fix it as p0 in 29)

@vitsai vitsai removed the release-blocker P0 Issue that blocks the release label Oct 24, 2023
@rkooo567 rkooo567 added ray 2.9 Issues targeting Ray 2.9 release (~Q4 CY2023) and removed ray 2.8 labels Oct 30, 2023
@jjyao jjyao unassigned vitsai Nov 8, 2023
@anyscalesam anyscalesam added core-client ray client related issues P1 Issue that should be fixed within a few weeks and removed P0 Issues that should be fixed in short order labels Nov 13, 2023
@rickyyx rickyyx mentioned this issue Nov 16, 2023
Merged
8 tasks
@rkooo567 rkooo567 added ray 2.10 and removed ray 2.9 Issues targeting Ray 2.9 release (~Q4 CY2023) labels Dec 12, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
core Issues that should be addressed in Ray Core core-client ray client related issues flaky-tracker Issue created via Flaky Test Tracker https://flaky-tests.ray.io/ P1 Issue that should be fixed within a few weeks ray 2.10
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@jjyao @vitsai @rkooo567 @anyscalesam