Flaky Test test_random.test_poisson_generator failed CI

[zachgk]

test_random.test_poisson_generator failed the CI at http://jenkins.mxnet-ci.amazon-ml.com/blue/organizations/jenkins/mxnet-validation%2Fcentos-cpu/detail/PR-13364/7/pipeline on unrelated PR #13364.

See output below:

======================================================================

FAIL: test_random.test_poisson_generator

----------------------------------------------------------------------

Traceback (most recent call last):

  File "/usr/lib/python3.6/site-packages/nose/case.py", line 198, in runTest

    self.test(*self.arg)

  File "/work/mxnet/tests/python/unittest/common.py", line 173, in test_new

    orig_test(*args, **kwargs)

  File "/work/mxnet/tests/python/unittest/test_random.py", line 583, in test_poisson_generator

    verify_generator(generator=generator_mx_same_seed, buckets=buckets, probs=probs)

  File "/work/mxnet/python/mxnet/test_utils.py", line 1970, in verify_generator

    str(buckets), str(probs)))

AssertionError: Generator test fails, Chi-square p=[0.034616615290469575, 0.033783181858245, 0.0003193443257745248, 0.26556350876099005, 0.002523698170023564], obs_freq=[array([457561, 540751,   1688]), array([456707, 541736,   1557]), array([459340, 538963,   1697]), array([458738, 539671,   1591]), array([458654, 539885,   1461])], expected_freq=[array([457929, 540482,   1588], dtype=int32), array([457929, 540482,   1588], dtype=int32), array([457929, 540482,   1588], dtype=int32), array([457929, 540482,   1588], dtype=int32), array([457929, 540482,   1588], dtype=int32)].

buckets=[(-1.0, 9.5), (9.5, 20.5), (20.5, inf)], probs=[0.4579297144718523, 0.5404820248662896, 0.0015882606618580208]

-------------------- >> begin captured logging << --------------------

common: INFO: Setting test np/mx/python random seeds, use MXNET_TEST_SEED=1588927696 to reproduce.

--------------------- >> end captured logging << ---------------------

[zachgk]

@mxnet-label-bot add [Test, Flaky, Python]

[shesung]

same problem.

http://jenkins.mxnet-ci.amazon-ml.com/blue/organizations/jenkins/mxnet-validation%2Fwindows-gpu/detail/PR-13088/3/pipeline

[pengzhao-intel]

same problem.

http://jenkins.mxnet-ci.amazon-ml.com/blue/organizations/jenkins/mxnet-validation%2Fwindows-gpu/detail/PR-13088/3/pipeline

@marcoabreu @larroy could you help take a look for CI issue?

[larroy]

Is this the same problem? http://jenkins.mxnet-ci.amazon-ml.com/blue/rest/organizations/jenkins/pipelines/mxnet-validation/pipelines/sanity/branches/PR-14122/runs/3/nodes/39/steps/79/log/?start=0