IdeaPluginIntegrationTest is flaky with Python 3 #7150
Comments
|
Interesting, these tests have there for a while and were not flaky. Wonder what's changed. |
|
Hm, I saw this while working on #7141 and thought it was due to unicode issues that I solved in the PR. I'll dig more into this right now and try to stabilize. |
|
@stuhood I'm able to reproduce locally with I could not reproduce with #!/bin/bash
while $@; do :; doneSo, this started flaking as of #7141. My hypothesis is the command has always been flakey with Py3 for some reason and we simply weren't testing it as it was on the blacklist. I do not think the changes I made in #7141 caused it, as I distinctly remember the error before I made the fixes, so it's more likely they were always there. -- I'm trying to find the cause.. With how Python works, we know the output file exists. The issue is it's not being written to. Let me know if we should revert #7141 until we solve this. |
Eric-Arellano
changed the title
### Problem See #7150. The idea plugin gen integration test became flaky on py3 as of #7141. ### Solution Blacklist test on py3 as per [slack discussion](https://pantsbuild.slack.com/archives/CBNMV1LRH/p1548436157036300).
… ConsoleTask interface (#7460) ### Problem The Idea plugin integration tests would frequently flake with Python 3. This appears to be because the task is a `ConsoleTask`, but it overrides `execute()` when it is instead expected to override `console_output()`. With this override, we would no longer call `self._outstream.flush()`, so the tests would sometimes complain that there was no entry in the `--output-file`, as the change would not always be persisted. It makes sense that this did not start flaking until Python 3, because IO streams changed their behavior in Python 3. Refer to `ConsoleTask` for everything we need to do to properly output to the console and files: https://github.com/pantsbuild/pants/blob/e620a9e862a245088bc5b477aed2b87e2bc6a307/src/python/pants/task/console_task.py#L52-L62 Will close #7150. ### Solution No longer override `execute()` and instead move the logic into `console_output()`. ### Result The tests no longer flake. This was confirmed locally by creating the script `untilfail`: ```bash #!/bin/bash while $@; do :; done ``` Then running `./untilfail ./pants clean-all test.pytest tests/python/pants_test/backend/project_info/tasks:idea_plugin_integration -- -k test_idea_plugin_single_target` for 10 minutes on both macOS and Ubuntu without fail.
… ConsoleTask interface (#7460) ### Problem The Idea plugin integration tests would frequently flake with Python 3. This appears to be because the task is a `ConsoleTask`, but it overrides `execute()` when it is instead expected to override `console_output()`. With this override, we would no longer call `self._outstream.flush()`, so the tests would sometimes complain that there was no entry in the `--output-file`, as the change would not always be persisted. It makes sense that this did not start flaking until Python 3, because IO streams changed their behavior in Python 3. Refer to `ConsoleTask` for everything we need to do to properly output to the console and files: https://github.com/pantsbuild/pants/blob/e620a9e862a245088bc5b477aed2b87e2bc6a307/src/python/pants/task/console_task.py#L52-L62 Will close #7150. ### Solution No longer override `execute()` and instead move the logic into `console_output()`. ### Result The tests no longer flake. This was confirmed locally by creating the script `untilfail`: ```bash #!/bin/bash while $@; do :; done ``` Then running `./untilfail ./pants clean-all test.pytest tests/python/pants_test/backend/project_info/tasks:idea_plugin_integration -- -k test_idea_plugin_single_target` for 10 minutes on both macOS and Ubuntu without fail.
Various test methods (one of which is
test_idea_plugin_single_target) can fail with:This is likely caused by some other underlying flakiness (unfortunately). But nonetheless.
The text was updated successfully, but these errors were encountered: