Fix autest flakiness #9372
Merged
Fix autest flakiness #9372
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
The fatal.test.py test has been flaky because of a race condition between the process shutdown caused by TSFatal and the autest shutting down the process because the Default Process print statement finished. This patch adds a FileContaines ready condition for the TSFatal log message before ending the TestRun which makes it run correctly more reliably.
The remap_load_empty_failure.test.py had a race condition between the autest run finishing due to the Default echo process finishing and the traffic server instance writing the "remap.config failed to load" message to the diags.log. This updates the Ready condition for the traffic server process to wait until that log is printed before running the echo and ending the test.
The emergency.test.py test has been flaky because of a race condition between the process shutdown caused by TSEmergency and the autest shutting down the process because the Default Process print statement finished. This patch adds a FileContaines ready condition for the TSEmergency log message before ending the TestRun which makes it run correctly more reliably.
zwoop
approved these changes
Feb 2, 2023
randall
approved these changes
Feb 2, 2023
moleksy
pushed a commit
to moleksy/trafficserver
that referenced
this pull request
Feb 8, 2023
A number of tests were flakey because of race conditions between the AuTest framework finishing the Default TestRun process and ending the test, and therefore the traffic server process, before traffic server had time to exercise the expected functionality of the test. This improves the Ready conditions for these tests which should improve their reliability.
bneradt
added a commit
to bneradt/trafficserver
that referenced
this pull request
Feb 8, 2023
This is a second iteration to further improve some of the autests that exercise ATS shutting down during test execution. See apache#9372 for the previous patch. The previous update improved reliability, but still had a race condition between autest recognizing ATS process shutdown and ATS writing the expected error log. If it detected the ts process ending before the log was written, the autest would fail because the framework thought the process ended before the Ready condition was satisfied. This patch addresses this by using a separate process to wait upon the log entry instead of making the ts Ready condition the content of the log file.
bneradt
added a commit
that referenced
this pull request
Feb 8, 2023
This is a second iteration to further improve some of the autests that exercise ATS shutting down during test execution. See #9372 for the previous patch. The previous update improved reliability, but still had a race condition between autest recognizing ATS process shutdown and ATS writing the expected error log. If it detected the ts process ending before the log was written, the autest would fail because the framework thought the process ended before the Ready condition was satisfied. This patch addresses this by using a separate process to wait upon the log entry instead of making the ts Ready condition the content of the log file.
bneradt
added a commit
to bneradt/trafficserver
that referenced
this pull request
Apr 25, 2023
A number of tests were flakey because of race conditions between the AuTest framework finishing the Default TestRun process and ending the test, and therefore the traffic server process, before traffic server had time to exercise the expected functionality of the test. This improves the Ready conditions for these tests which should improve their reliability. (cherry picked from commit c175aa4)
bneradt
added a commit
to bneradt/trafficserver
that referenced
this pull request
Apr 25, 2023
This is a second iteration to further improve some of the autests that exercise ATS shutting down during test execution. See apache#9372 for the previous patch. The previous update improved reliability, but still had a race condition between autest recognizing ATS process shutdown and ATS writing the expected error log. If it detected the ts process ending before the log was written, the autest would fail because the framework thought the process ended before the Ready condition was satisfied. This patch addresses this by using a separate process to wait upon the log entry instead of making the ts Ready condition the content of the log file. (cherry picked from commit 086501c)
zwoop
pushed a commit
that referenced
this pull request
Apr 26, 2023
* Fix autest flakiness due better Ready Conditions (#9372) A number of tests were flakey because of race conditions between the AuTest framework finishing the Default TestRun process and ending the test, and therefore the traffic server process, before traffic server had time to exercise the expected functionality of the test. This improves the Ready conditions for these tests which should improve their reliability. (cherry picked from commit c175aa4) * Make autest ts shutdown tests more reliable (#9391) This is a second iteration to further improve some of the autests that exercise ATS shutting down during test execution. See #9372 for the previous patch. The previous update improved reliability, but still had a race condition between autest recognizing ATS process shutdown and ATS writing the expected error log. If it detected the ts process ending before the log was written, the autest would fail because the framework thought the process ended before the Ready condition was satisfied. This patch addresses this by using a separate process to wait upon the log entry instead of making the ts Ready condition the content of the log file. (cherry picked from commit 086501c)
JosiahWI
pushed a commit
to JosiahWI/trafficserver
that referenced
this pull request
Jul 19, 2023
…e#769) A number of tests were flakey because of race conditions between the AuTest framework finishing the Default TestRun process and ending the test, and therefore the traffic server process, before traffic server had time to exercise the expected functionality of the test. This improves the Ready conditions for these tests which should improve their reliability. (cherry picked from commit c175aa4)
masaori335
pushed a commit
to masaori335/trafficserver
that referenced
this pull request
Jul 24, 2023
* Fix autest flakiness due better Ready Conditions (apache#9372) A number of tests were flakey because of race conditions between the AuTest framework finishing the Default TestRun process and ending the test, and therefore the traffic server process, before traffic server had time to exercise the expected functionality of the test. This improves the Ready conditions for these tests which should improve their reliability. (cherry picked from commit c175aa4) * Make autest ts shutdown tests more reliable (apache#9391) This is a second iteration to further improve some of the autests that exercise ATS shutting down during test execution. See apache#9372 for the previous patch. The previous update improved reliability, but still had a race condition between autest recognizing ATS process shutdown and ATS writing the expected error log. If it detected the ts process ending before the log was written, the autest would fail because the framework thought the process ended before the Ready condition was satisfied. This patch addresses this by using a separate process to wait upon the log entry instead of making the ts Ready condition the content of the log file. (cherry picked from commit 086501c) (cherry picked from commit a1eb655)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
A number of tests were flakey because of race conditions between the AuTest framework finishing the Default TestRun process and ending the test, and therefore the traffic server process, before traffic server had time to exercise the expected functionality of the test. This improves the Ready conditions for these tests which should improve their reliability.