-
Notifications
You must be signed in to change notification settings - Fork 7k
[Data] Add -s flag to test_backpressure_e2e.py
#58977
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
Signed-off-by: Balaji Veeramani <bveeramani@berkeley.edu>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Code Review
This pull request adds the -s flag to the pytest command in test_backpressure_e2e.py to aid in debugging flaky tests. The change is functionally correct. I've added one suggestion to improve code readability and maintainability.
| import sys | ||
|
|
||
| sys.exit(pytest.main(["-v", __file__])) | ||
| sys.exit(pytest.main(["-sv", __file__])) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
For better readability, it's good practice to separate combined short options into their own arguments. Additionally, adding a comment to explain why the -s flag is necessary for this specific test file would be helpful for future maintainers, given the inconsistency with other tests in the repository.
For example:
# The -s flag is added to ease debugging of flaky tests.| sys.exit(pytest.main(["-sv", __file__])) | |
| sys.exit(pytest.main(["-s", "-v", __file__])) |
## Description `test_backpressure_e2e` occasionally fails without any traceback or warning message: ``` [2025-11-24T21:42:12Z] ==================== Test output for //python/ray/data:test_backpressure_e2e: -- [2025-11-24T21:42:12Z] /opt/miniforge/lib/python3.12/site-packages/paramiko/pkey.py:82: CryptographyDeprecationWarning: TripleDES has been moved to cryptography.hazmat.decrepit.ciphers.algorithms.TripleDES and will be removed from cryptography.hazmat.primitives.ciphers.algorithms in 48.0.0. [2025-11-24T21:42:12Z] "cipher": algorithms.TripleDES, [2025-11-24T21:42:12Z] /opt/miniforge/lib/python3.12/site-packages/paramiko/transport.py:253: CryptographyDeprecationWarning: TripleDES has been moved to cryptography.hazmat.decrepit.ciphers.algorithms.TripleDES and will be removed from cryptography.hazmat.primitives.ciphers.algorithms in 48.0.0. [2025-11-24T21:42:12Z] "class": algorithms.TripleDES, [2025-11-24T21:42:12Z] ============================= test session starts ============================== [2025-11-24T21:42:12Z] platform linux -- Python 3.12.9, pytest-7.4.4, pluggy-1.3.0 -- /opt/miniforge/bin/python3 [2025-11-24T21:42:12Z] cachedir: .pytest_cache [2025-11-24T21:42:12Z] rootdir: /root/.cache/bazel/_bazel_root/1df605deb6d24fc8068f6e25793ec703/execroot/io_ray [2025-11-24T21:42:12Z] configfile: pytest.ini [2025-11-24T21:42:12Z] plugins: repeat-0.9.3, anyio-3.7.1, fugue-0.8.7, aiohttp-1.1.0, asyncio-0.17.2, docker-tools-3.1.3, forked-1.4.0, pytest_httpserver-1.1.3, lazy-fixtures-1.1.2, mock-3.14.0, remotedata-0.3.2, rerunfailures-11.1.2, sphinx-0.5.1.dev0, sugar-0.9.5, timeout-2.1.0, typeguard-2.13.3 [2025-11-24T21:42:12Z] asyncio: mode=Mode.AUTO [2025-11-24T21:42:12Z] timeout: 180.0s [2025-11-24T21:42:12Z] timeout method: signal [2025-11-24T21:42:12Z] timeout func_only: False [2025-11-24T21:42:12Z] collecting ... collected 12 items [2025-11-24T21:42:12Z] [2025-11-24T21:42:12Z] python/ray/data/tests/test_backpressure_e2e.py::test_large_e2e_backpressure_no_spilling PASSED [ 8%] [2025-11-24T21:42:12Z] python/ray/data/tests/test_backpressure_e2e.py::test_no_deadlock_on_small_cluster_resources[False-3-500] PASSED [ 16%] [2025-11-24T21:42:12Z] python/ray/data/tests/test_backpressure_e2e.py::test_no_deadlock_on_small_cluster_resources[False-4-100] PASSED [ 25%] [2025-11-24T21:42:12Z] python/ray/data/tests/test_backpressure_e2e.py::test_no_deadlock_on_small_cluster_resources[False-3-100] PASSED [ 33%] [2025-11-24T21:42:12Z] python/ray/data/tests/test_backpressure_e2e.py::test_no_deadlock_on_small_cluster_resources[True-3-500] PASSED [ 41%] [2025-11-24T21:42:12Z] python/ray/data/tests/test_backpressure_e2e.py::test_no_deadlock_on_small_cluster_resources[True-4-100] PASSED [ 50%] [2025-11-24T21:42:12Z] python/ray/data/tests/test_backpressure_e2e.py::test_no_deadlock_on_small_cluster_resources[True-3-100] PASSED [ 58%] [2025-11-24T21:42:12Z] python/ray/data/tests/test_backpressure_e2e.py::test_no_deadlock_on_resource_contention[False] PASSED [ 66%] [2025-11-24T21:42:12Z] python/ray/data/tests/test_backpressure_e2e.py::test_no_deadlock_on_resource_contention[True] PASSED [ 75%] [2025-11-24T21:42:12Z] python/ray/data/tests/test_backpressure_e2e.py::test_no_deadlock_with_preserve_order PASSED [ 83%] [2025-11-24T21:42:12Z] python/ray/data/tests/test_backpressure_e2e.py::test_input_backpressure_e2e PASSED [ 91%]================================================================================ ``` To make this easier to debug, this PR enables the `-s` flag to log more information. Signed-off-by: Balaji Veeramani <bveeramani@berkeley.edu>
Description
test_backpressure_e2eoccasionally fails without any traceback or warning message:To make this easier to debug, this PR enables the
-sflag to log more information.