Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Batch Scenario Runs Into Single Test #10878

Merged
merged 6 commits into from
Jun 15, 2022
Merged

Conversation

nisdas
Copy link
Member

@nisdas nisdas commented Jun 14, 2022

What type of PR is this?

E2E Improvement

What does this PR do? Why is it needed?

We have many different scenarios that we want to run, however having a separate test for them increases flakiness significantly. Along with that we run these tests in multiple separate shards, if these are multiclient tests that are run they can significantly stress our test runner. The increased stress can lead to more flakes happening. Therefore this PR instead batches all these scenarios into a single test, that way we do not need to run so many tests and the test setup only needs to happen once.

Which issues(s) does this PR fix?

N.A

Other notes for review

@nisdas nisdas added Ready For Review E2E Tests End-To-End testing labels Jun 14, 2022
@nisdas nisdas requested a review from a team as a code owner June 14, 2022 09:54
Copy link
Contributor

@rkapka rkapka left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We skip epochs 13 and 18 - is this deliberate?

}

func (r *testRunner) singleNodeOfflineMulticlient(epoch uint64, _ []*grpc.ClientConn) bool {
// This interceptor will define the multi scenario run for our minimal tests.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

2) is missing from the comment

@nisdas nisdas dismissed a stale review via 10fbfd4 June 14, 2022 13:04
@prylabs-bulldozer prylabs-bulldozer bot merged commit 9e4ba75 into develop Jun 15, 2022
@delete-merged-branch delete-merged-branch bot deleted the batchScenarios branch June 15, 2022 08:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
E2E Tests End-To-End testing
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants