Change integration test harness to support scaling tests #1500
Conversation
objmagic
changed the title
| @@ -86,6 +86,9 @@ public void execute(Tuple tuple) { | |||
| collector.emit(Constants.INTEGRATION_TEST_CONTROL_STREAM_ID, | |||
| tuple, | |||
| new Values(Constants.INTEGRATION_TEST_TERMINAL)); | |||
| } else { | |||
| LOG.info(String.format( | |||
| "Received a terminals, waiting to receive %s more", terminalsToReceive)); | |||
There was a problem hiding this comment.
it should be Received a terminal or Received terminals
| @@ -14,18 +14,18 @@ | |||
| # The location of default configure file | |||
| DEFAULT_TEST_CONF_FILE = "integration-test/src/python/test_runner/resources/test.json" | |||
|
|
|||
| RETRY_ATTEMPTS = 15 | |||
| RETRY_ATTEMPTS = 25 | |||
There was a problem hiding this comment.
why the change from 15 to 25?
There was a problem hiding this comment.
When there are legit failures retrying 25 times is excessive and it takes over 4 minutes of results checking per test. Even 15 times seems high. If we have to retry anything this many times for simple, small test topologies, something is wrong.
That said, I can keep it if our tests are known to be too flaky when set below 25.
There was a problem hiding this comment.
Most of the times the integration tests not flaky, I believe.
There was a problem hiding this comment.
That's what I thought. Let's reduce it and see if it causes problems.
|
All comments addressed and CI is passing. Let me know if anyone has additional comments. |
|
LGTM 👍 |
* Change int test harness to support scaling tests * fix arg parsing for python tests * change retry back to 25 * fix bug during iteration * fix topo args bug * clean up log messages * reduce attempts back to 15
* Change int test harness to support scaling tests * fix arg parsing for python tests * change retry back to 25 * fix bug during iteration * fix topo args bug * clean up log messages * reduce attempts back to 15
Made a number of changes to support scaling tests:
updateArgs, the test is considered a scaling (i.e., update) testMultiPhaseTestSpoutwhich outputs tuples, polls fortopology_updatedstate changes on thehttp_serverand outputs more. This class relies onHttpGetConditionthat polls for expected state.topology_startedstate.Apologies that this PR got large, but it was a hard one for me to break up.
There is still more work to be done to handle tests the scale down. Currently the aggregator bolt waits for teminal tuples from all upstream instances so removing one leaves it hanging. Will tackle that in a separate review.