[UnitTest] Fuzz based on seed rather than random value. #10515
Conversation
Some extensions to run tests in parallel (e.g. `pytest-xdist`) require that test collection be deterministic. Using the random seed as the test parameter instead of the random value makes the test collection be deterministic.
Lunderberg
requested review from
comaniac,
junrushao,
tqchen,
jroesch,
areusch,
yzhliu,
merrymercy and
icemelon
as code owners
|
That's correct. I initially added this test to validate copying across a few orders of magnitude, and a fixed input size would work just as well. If we want to have the test be randomized per run, we could remove the call to Most of the other unit tests don't run into the xdist incompatibility, because they didn't change the name of the test being run. It only became an issue here because the seed value was included in the parametrized test name. I've found it really useful to be able to use xdist to speed up local test times. Other than some false positives from running out of GPU memory, most of the other tests are xdist-friendly. |
|
I think there was some discussion of using |
|
we want to add it since we have a lot of CPU headroom we're not using for tests in most cases, but there are a bunch of pitfalls like this waiting around that someone has to wade through and we haven't gotten around to it yet |
Some extensions to run tests in parallel (e.g. `pytest-xdist`) require that test collection be deterministic. Using the random seed as the test parameter instead of the random value makes the test collection be deterministic.
Some extensions to run tests in parallel (e.g.
pytest-xdist) require that test collection be deterministic. Using the random seed as the test parameter instead of the random value makes the test collection be deterministic.