Skip to content
This repository has been archived by the owner on Nov 17, 2023. It is now read-only.

Flaky test: test_gru_bidirectional #11219

Closed
eric-haibin-lin opened this issue Jun 10, 2018 · 12 comments
Closed

Flaky test: test_gru_bidirectional #11219

eric-haibin-lin opened this issue Jun 10, 2018 · 12 comments
Labels
Flaky focus progress Test

Comments

@eric-haibin-lin
Copy link
Member

http://jenkins.mxnet-ci.amazon-ml.com/blue/organizations/jenkins/incubator-mxnet/detail/PR-11197/7/pipeline

test_operator.test_gru_sym ... ok

test_operator.test_gru_bidirectional ... 

(py2) C:\jenkins_slave\workspace\ut-python-gpu>IF -1073741819 NEQ 0 exit /b 1 

script returned exit code 1
@larroy
Copy link
Contributor

larroy commented Jun 11, 2018

can be reproduced with MXNET_MODULE_SEED=1447306965
python -mnose -v test_operator.py:test_gru_bidirectional

@ThomasDelteil
Copy link
Contributor

duplicate here of this one
#11202

@ThomasDelteil
Copy link
Contributor

ThomasDelteil commented Jun 11, 2018 •
edited
Loading

This one is happening way too often, had it 3 times in 6 builds. Should we upgrade to bug?

@ThomasDelteil ThomasDelteil mentioned this issue Jun 12, 2018
Merged
3 tasks
@ThomasDelteil
Copy link
Contributor

@lihaofd can you take a look please? thanks!

@ThomasDelteil
Copy link
Contributor

http://jenkins.mxnet-ci.amazon-ml.com/blue/organizations/jenkins/incubator-mxnet/detail/PR-11236/1/pipeline

@ThomasDelteil
Copy link
Contributor

http://jenkins.mxnet-ci.amazon-ml.com/blue/organizations/jenkins/incubator-mxnet/detail/PR-11225/4/pipeline

@szha szha mentioned this issue Jun 12, 2018
Closed
@aaronmarkham
Copy link
Contributor

aaronmarkham commented Jun 14, 2018 •
edited
Loading

Still a problem...
http://jenkins.mxnet-ci.amazon-ml.com/blue/organizations/jenkins/incubator-mxnet/detail/PR-11261/4/pipeline
edit
...For a bunch of PRs... almost like nothing is getting through because of this!

@ThomasDelteil
Copy link
Contributor

@lihaofd could you disable this test while you investigate why it is failing on windows?

@lihaofd lihaofd mentioned this issue Jun 14, 2018
Merged
@aaronmarkham
Copy link
Contributor

Doesn't seem disabled to me:
http://jenkins.mxnet-ci.amazon-ml.com/blue/organizations/jenkins/incubator-mxnet/detail/PR-11287/1/pipeline

@ThomasDelteil
Copy link
Contributor

might have been just merged and you got unlucky, got merged 1h ago as well

@aaronmarkham
Copy link
Contributor

Ya, you're right. I saw the 16 hours ago and "merged" nearby so I thought that's when it happened. I definitely missed the window.

@lihaofd lihaofd mentioned this issue Jun 19, 2018
Merged
3 tasks
szha pushed a commit that referenced this issue Jun 22, 2018
@szha
Fix issue of test_gru_bidirectional #11219 and add robust code (#11333)
5550c0a 
* fix param init bug and remove memcpy/memset

* fix bug for bidirection size

* add 100 times loop for test_gru_bidirectional robust checking

* add test_loop_gru_bidirectional

* record number of passed

* remove 1000 times case
@pengzhao-intel
Copy link
Contributor

the fix is merged. Feel free to ping me if you see the errors again.

XinYao1994 pushed a commit to XinYao1994/incubator-mxnet that referenced this issue Aug 29, 2018
@szha
Fix issue of test_gru_bidirectional apache#11219 and add robust code (a…
6e0e4d4 
…pache#11333)

* fix param init bug and remove memcpy/memset

* fix bug for bidirection size

* add 100 times loop for test_gru_bidirectional robust checking

* add test_loop_gru_bidirectional

* record number of passed

* remove 1000 times case
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
Flaky focus progress Test
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
6 participants
@larroy @szha @ThomasDelteil @eric-haibin-lin @aaronmarkham @pengzhao-intel