Skip to content

Conversation

@bveeramani
Copy link
Member

Description

test_backpressure_e2e occasionally fails without any traceback or warning message:

[2025-11-24T21:42:12Z] ==================== Test output for //python/ray/data:test_backpressure_e2e:
--
[2025-11-24T21:42:12Z] /opt/miniforge/lib/python3.12/site-packages/paramiko/pkey.py:82: CryptographyDeprecationWarning: TripleDES has been moved to cryptography.hazmat.decrepit.ciphers.algorithms.TripleDES and will be removed from cryptography.hazmat.primitives.ciphers.algorithms in 48.0.0.
[2025-11-24T21:42:12Z]   "cipher": algorithms.TripleDES,
[2025-11-24T21:42:12Z] /opt/miniforge/lib/python3.12/site-packages/paramiko/transport.py:253: CryptographyDeprecationWarning: TripleDES has been moved to cryptography.hazmat.decrepit.ciphers.algorithms.TripleDES and will be removed from cryptography.hazmat.primitives.ciphers.algorithms in 48.0.0.
[2025-11-24T21:42:12Z]   "class": algorithms.TripleDES,
[2025-11-24T21:42:12Z] ============================= test session starts ==============================
[2025-11-24T21:42:12Z] platform linux -- Python 3.12.9, pytest-7.4.4, pluggy-1.3.0 -- /opt/miniforge/bin/python3
[2025-11-24T21:42:12Z] cachedir: .pytest_cache
[2025-11-24T21:42:12Z] rootdir: /root/.cache/bazel/_bazel_root/1df605deb6d24fc8068f6e25793ec703/execroot/io_ray
[2025-11-24T21:42:12Z] configfile: pytest.ini
[2025-11-24T21:42:12Z] plugins: repeat-0.9.3, anyio-3.7.1, fugue-0.8.7, aiohttp-1.1.0, asyncio-0.17.2, docker-tools-3.1.3, forked-1.4.0, pytest_httpserver-1.1.3, lazy-fixtures-1.1.2, mock-3.14.0, remotedata-0.3.2, rerunfailures-11.1.2, sphinx-0.5.1.dev0, sugar-0.9.5, timeout-2.1.0, typeguard-2.13.3
[2025-11-24T21:42:12Z] asyncio: mode=Mode.AUTO
[2025-11-24T21:42:12Z] timeout: 180.0s
[2025-11-24T21:42:12Z] timeout method: signal
[2025-11-24T21:42:12Z] timeout func_only: False
[2025-11-24T21:42:12Z] collecting ... collected 12 items
[2025-11-24T21:42:12Z]
[2025-11-24T21:42:12Z] python/ray/data/tests/test_backpressure_e2e.py::test_large_e2e_backpressure_no_spilling PASSED [  8%]
[2025-11-24T21:42:12Z] python/ray/data/tests/test_backpressure_e2e.py::test_no_deadlock_on_small_cluster_resources[False-3-500] PASSED [ 16%]
[2025-11-24T21:42:12Z] python/ray/data/tests/test_backpressure_e2e.py::test_no_deadlock_on_small_cluster_resources[False-4-100] PASSED [ 25%]
[2025-11-24T21:42:12Z] python/ray/data/tests/test_backpressure_e2e.py::test_no_deadlock_on_small_cluster_resources[False-3-100] PASSED [ 33%]
[2025-11-24T21:42:12Z] python/ray/data/tests/test_backpressure_e2e.py::test_no_deadlock_on_small_cluster_resources[True-3-500] PASSED [ 41%]
[2025-11-24T21:42:12Z] python/ray/data/tests/test_backpressure_e2e.py::test_no_deadlock_on_small_cluster_resources[True-4-100] PASSED [ 50%]
[2025-11-24T21:42:12Z] python/ray/data/tests/test_backpressure_e2e.py::test_no_deadlock_on_small_cluster_resources[True-3-100] PASSED [ 58%]
[2025-11-24T21:42:12Z] python/ray/data/tests/test_backpressure_e2e.py::test_no_deadlock_on_resource_contention[False] PASSED [ 66%]
[2025-11-24T21:42:12Z] python/ray/data/tests/test_backpressure_e2e.py::test_no_deadlock_on_resource_contention[True] PASSED [ 75%]
[2025-11-24T21:42:12Z] python/ray/data/tests/test_backpressure_e2e.py::test_no_deadlock_with_preserve_order PASSED [ 83%]
[2025-11-24T21:42:12Z] python/ray/data/tests/test_backpressure_e2e.py::test_input_backpressure_e2e PASSED [ 91%]================================================================================

To make this easier to debug, this PR enables the -s flag to log more information.

Signed-off-by: Balaji Veeramani <bveeramani@berkeley.edu>
@bveeramani bveeramani requested a review from a team as a code owner November 25, 2025 19:01
@ray-gardener ray-gardener bot added the data Ray Data-related issues label Nov 25, 2025
@bveeramani bveeramani enabled auto-merge (squash) November 25, 2025 19:07
@github-actions github-actions bot added the go add ONLY when ready to merge, run all tests label Nov 25, 2025
Copy link
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request adds the -s flag to the pytest command in test_backpressure_e2e.py to aid in debugging flaky tests. The change is functionally correct. I've added one suggestion to improve code readability and maintainability.

import sys

sys.exit(pytest.main(["-v", __file__]))
sys.exit(pytest.main(["-sv", __file__]))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

For better readability, it's good practice to separate combined short options into their own arguments. Additionally, adding a comment to explain why the -s flag is necessary for this specific test file would be helpful for future maintainers, given the inconsistency with other tests in the repository.

For example:

# The -s flag is added to ease debugging of flaky tests.
Suggested change
sys.exit(pytest.main(["-sv", __file__]))
sys.exit(pytest.main(["-s", "-v", __file__]))

@bveeramani bveeramani merged commit 3a71a6f into master Nov 25, 2025
5 of 7 checks passed
@bveeramani bveeramani deleted the data-add-logs branch November 25, 2025 20:09
SheldonTsen pushed a commit to SheldonTsen/ray that referenced this pull request Dec 1, 2025
## Description

`test_backpressure_e2e` occasionally fails without any traceback or
warning message:
```
[2025-11-24T21:42:12Z] ==================== Test output for //python/ray/data:test_backpressure_e2e:
--
[2025-11-24T21:42:12Z] /opt/miniforge/lib/python3.12/site-packages/paramiko/pkey.py:82: CryptographyDeprecationWarning: TripleDES has been moved to cryptography.hazmat.decrepit.ciphers.algorithms.TripleDES and will be removed from cryptography.hazmat.primitives.ciphers.algorithms in 48.0.0.
[2025-11-24T21:42:12Z]   "cipher": algorithms.TripleDES,
[2025-11-24T21:42:12Z] /opt/miniforge/lib/python3.12/site-packages/paramiko/transport.py:253: CryptographyDeprecationWarning: TripleDES has been moved to cryptography.hazmat.decrepit.ciphers.algorithms.TripleDES and will be removed from cryptography.hazmat.primitives.ciphers.algorithms in 48.0.0.
[2025-11-24T21:42:12Z]   "class": algorithms.TripleDES,
[2025-11-24T21:42:12Z] ============================= test session starts ==============================
[2025-11-24T21:42:12Z] platform linux -- Python 3.12.9, pytest-7.4.4, pluggy-1.3.0 -- /opt/miniforge/bin/python3
[2025-11-24T21:42:12Z] cachedir: .pytest_cache
[2025-11-24T21:42:12Z] rootdir: /root/.cache/bazel/_bazel_root/1df605deb6d24fc8068f6e25793ec703/execroot/io_ray
[2025-11-24T21:42:12Z] configfile: pytest.ini
[2025-11-24T21:42:12Z] plugins: repeat-0.9.3, anyio-3.7.1, fugue-0.8.7, aiohttp-1.1.0, asyncio-0.17.2, docker-tools-3.1.3, forked-1.4.0, pytest_httpserver-1.1.3, lazy-fixtures-1.1.2, mock-3.14.0, remotedata-0.3.2, rerunfailures-11.1.2, sphinx-0.5.1.dev0, sugar-0.9.5, timeout-2.1.0, typeguard-2.13.3
[2025-11-24T21:42:12Z] asyncio: mode=Mode.AUTO
[2025-11-24T21:42:12Z] timeout: 180.0s
[2025-11-24T21:42:12Z] timeout method: signal
[2025-11-24T21:42:12Z] timeout func_only: False
[2025-11-24T21:42:12Z] collecting ... collected 12 items
[2025-11-24T21:42:12Z]
[2025-11-24T21:42:12Z] python/ray/data/tests/test_backpressure_e2e.py::test_large_e2e_backpressure_no_spilling PASSED [  8%]
[2025-11-24T21:42:12Z] python/ray/data/tests/test_backpressure_e2e.py::test_no_deadlock_on_small_cluster_resources[False-3-500] PASSED [ 16%]
[2025-11-24T21:42:12Z] python/ray/data/tests/test_backpressure_e2e.py::test_no_deadlock_on_small_cluster_resources[False-4-100] PASSED [ 25%]
[2025-11-24T21:42:12Z] python/ray/data/tests/test_backpressure_e2e.py::test_no_deadlock_on_small_cluster_resources[False-3-100] PASSED [ 33%]
[2025-11-24T21:42:12Z] python/ray/data/tests/test_backpressure_e2e.py::test_no_deadlock_on_small_cluster_resources[True-3-500] PASSED [ 41%]
[2025-11-24T21:42:12Z] python/ray/data/tests/test_backpressure_e2e.py::test_no_deadlock_on_small_cluster_resources[True-4-100] PASSED [ 50%]
[2025-11-24T21:42:12Z] python/ray/data/tests/test_backpressure_e2e.py::test_no_deadlock_on_small_cluster_resources[True-3-100] PASSED [ 58%]
[2025-11-24T21:42:12Z] python/ray/data/tests/test_backpressure_e2e.py::test_no_deadlock_on_resource_contention[False] PASSED [ 66%]
[2025-11-24T21:42:12Z] python/ray/data/tests/test_backpressure_e2e.py::test_no_deadlock_on_resource_contention[True] PASSED [ 75%]
[2025-11-24T21:42:12Z] python/ray/data/tests/test_backpressure_e2e.py::test_no_deadlock_with_preserve_order PASSED [ 83%]
[2025-11-24T21:42:12Z] python/ray/data/tests/test_backpressure_e2e.py::test_input_backpressure_e2e PASSED [ 91%]================================================================================
```

To make this easier to debug, this PR enables the `-s` flag to log more
information.

Signed-off-by: Balaji Veeramani <bveeramani@berkeley.edu>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

data Ray Data-related issues go add ONLY when ready to merge, run all tests

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants