Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test: run tests shuffled #1645

Merged
merged 13 commits into from
Feb 1, 2023
Merged

test: run tests shuffled #1645

merged 13 commits into from
Feb 1, 2023

Conversation

Shinigami92
Copy link
Member

@Shinigami92 Shinigami92 commented Dec 9, 2022

@Shinigami92 Shinigami92 added s: on hold Blocked by something or frozen to avoid conflicts p: 1-normal Nothing urgent c: test labels Dec 9, 2022
@Shinigami92 Shinigami92 added this to the v8.0 - Module Re-Shuffling milestone Dec 9, 2022
@Shinigami92 Shinigami92 self-assigned this Dec 9, 2022
@ST-DDT
Copy link
Member

ST-DDT commented Dec 11, 2022

While this seems to solve the issue of testing always with the same values, it also renders our random seeded tests useless, because the tests are no longer reproducible (by setting the previous/failing seed) in case they uncover a test failure.

@Shinigami92 Shinigami92 requested a review from ST-DDT December 11, 2022 12:50
@Shinigami92 Shinigami92 removed the s: on hold Blocked by something or frozen to avoid conflicts label Dec 25, 2022
@codecov
Copy link

codecov bot commented Dec 25, 2022

Codecov Report

Merging #1645 (6dcec83) into next (5b705d4) will decrease coverage by 0.01%.
The diff coverage is n/a.

Additional details and impacted files
@@            Coverage Diff             @@
##             next    #1645      +/-   ##
==========================================
- Coverage   99.63%   99.63%   -0.01%     
==========================================
  Files        2346     2346              
  Lines      235002   235002              
  Branches     1133     1132       -1     
==========================================
- Hits       234147   234138       -9     
- Misses        833      842       +9     
  Partials       22       22              
Impacted Files Coverage Δ
src/modules/internet/user-agent.ts 92.01% <0.00%> (-2.08%) ⬇️
src/modules/location/index.ts 97.54% <0.00%> (-0.38%) ⬇️

@ST-DDT
Copy link
Member

ST-DDT commented Dec 25, 2022

I assume this still is likely to use seeds values close to our seeded tests (fixed seed + 1 or 2 calls).

@Shinigami92 Shinigami92 marked this pull request as ready for review January 29, 2023 14:03
@Shinigami92 Shinigami92 requested a review from a team as a code owner January 29, 2023 14:03
import-brain
import-brain previously approved these changes Jan 29, 2023
@Shinigami92
Copy link
Member Author

Uh damn it, I accidentally pushed it directly to this PR 🤷
I wanted to extract it to its own PR, but nevermind anyway.
The new commit fixes the mersenne test and additionally removed a todo from a test so it actually gets executed.

@ST-DDT ST-DDT enabled auto-merge (squash) February 1, 2023 15:34
@ST-DDT ST-DDT merged commit de7ed2e into next Feb 1, 2023
@ST-DDT ST-DDT deleted the 1318-run-test-shuffled branch February 1, 2023 15:40
matthewmayer pushed a commit to matthewmayer/faker that referenced this pull request Feb 18, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
No open projects
Status: Done
Development

Successfully merging this pull request may close these issues.

Random seeded tests are not random
4 participants