Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

chore: Enable random execution order of tests #554

Merged
merged 3 commits into from
Jun 27, 2022
Merged

chore: Enable random execution order of tests #554

merged 3 commits into from
Jun 27, 2022

Conversation

orpheuslummis
Copy link
Contributor

@orpheuslummis orpheuslummis commented Jun 22, 2022 •
edited
Loading

Relevant issue(s)

Resolves #553

Description

Enables the randomization the execution order of tests on make test. The option, go test -shuffle= is available since Go 1.17. The -shuffle=on switch sets the random seed at clock time.

Tasks

  • I made sure the code is well commented, particularly hard-to-understand areas.
  • I made sure the repository-held documentation is changed accordingly.
  • I made sure the pull request title adheres to the conventional commit style (the subset used in the project can be found in tools/configs/chglog/config.yml).
  • I made sure to discuss its limitations such as threats to validity, vulnerability to mistake and misuse, robustness to invalidation of assumptions, resource requirements, ...

How has this been tested?

Specify the platform(s) on which this was tested:

  • Debian Linux
  • MacOS

@orpheuslummis orpheuslummis added area/testing Related to any test or testing suite ci/build This is issue is about the build or CI system, and the administration of it. labels Jun 22, 2022
@orpheuslummis orpheuslummis added this to the DefraDB v0.3 milestone Jun 22, 2022
@orpheuslummis orpheuslummis self-assigned this Jun 22, 2022
@orpheuslummis
Copy link
Contributor Author

orpheuslummis commented Jun 22, 2022 •
edited
Loading

I expect this to fail until some bugs are fixed. #555 #556

@orpheuslummis orpheuslummis added the action/no-benchmark Skips the action that runs the benchmark. label Jun 22, 2022
@sourcenetwork sourcenetwork deleted a comment from source-devs Jun 22, 2022
@sourcenetwork sourcenetwork deleted a comment from source-devs Jun 22, 2022
@orpheuslummis orpheuslummis marked this pull request as ready for review June 24, 2022 13:59
fredcarle
fredcarle approved these changes Jun 24, 2022
View reviewed changes
Copy link
Collaborator

@fredcarle fredcarle left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

shahzadlone
shahzadlone reviewed Jun 27, 2022
View reviewed changes
.circleci/config.yml Outdated Show resolved Hide resolved
shahzadlone
shahzadlone reviewed Jun 27, 2022
View reviewed changes
Copy link
Member

@shahzadlone shahzadlone left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we should wait until this gets merged : #552

Also locally my test runs fail on this branch (using the -shuffle option).

@orpheuslummis
Copy link
Contributor Author

Maybe we should wait until this gets merged : #552

Also locally my test runs fail on this branch (using the -shuffle option).

yes exactly

@fredcarle
Copy link
Collaborator

#552 is now merged and tests are now passing. You can merge now @orpheuslummis 🙂

@orpheuslummis
Copy link
Contributor Author

#552 is now merged and tests are now passing. You can merge now @orpheuslummis 🙂

thanks. I'm doing more shuffle runs on my local machines before merging because I'm paranoid

@orpheuslummis orpheuslummis force-pushed the orpheus/chore/test-shuffle branch from fa828e2 to a62e55b Compare June 27, 2022 19:55
@orpheuslummis
Copy link
Contributor Author

#552 is now merged and tests are now passing. You can merge now @orpheuslummis 🙂

thanks. I'm doing more shuffle runs on my local machines before merging because I'm paranoid

I ran the suite 300 times on each linux & macos to make myself feel better.

@orpheuslummis orpheuslummis merged commit 2078fbe into develop Jun 27, 2022
@orpheuslummis orpheuslummis deleted the orpheus/chore/test-shuffle branch June 27, 2022 20:03
shahzadlone added a commit to shahzadlone/defradb that referenced this pull request Feb 23, 2024
@orpheuslummis @shahzadlone
chore: Enable random execution order of tests (sourcenetwork#554)
c3e6c70 
Co-authored-by: Shahzad Lone <shahzadlone@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@shahzadlone shahzadlone shahzadlone left review comments

@fredcarle fredcarle fredcarle approved these changes

Assignees

@orpheuslummis orpheuslummis

Labels
action/no-benchmark Skips the action that runs the benchmark. area/testing Related to any test or testing suite ci/build This is issue is about the build or CI system, and the administration of it.
Projects
None yet
Milestone
DefraDB v0.3
Development

Successfully merging this pull request may close these issues.

Randomize the execution order of tests
3 participants
@orpheuslummis @fredcarle @shahzadlone