Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Flaky] TFLite quantized conv test #6084

Merged
merged 6 commits into from
Jul 24, 2020
Merged

[Flaky] TFLite quantized conv test #6084

merged 6 commits into from
Jul 24, 2020

Conversation

anijain2305
Copy link
Contributor

@tqchen
Issue - #6064

The error earlier was for single element out of 3456 elements. I think this is because of rounding. For now, set the seed to 0 for deterministic randomness.

E       Mismatched elements: 1 / 3456 (0.0289%)
E       Max absolute difference: 0.06574249
E       Max relative difference: 0.0147059
E        x: array([[[ 5.193651,  4.864939,  4.470484, ...,  4.930681,  7.165924,
E                 5.390878],
E               [ 8.217802,  8.41503 ,  6.5085  , ...,  7.757606,  9.664136,...
E        y: array([[[ 5.193651,  4.864939,  4.470485, ...,  4.930681,  7.165924,
E                 5.390878],
E               [ 8.217802,  8.41503 ,  6.508499, ...,  7.757606,  9.664135,...

@tqchen
Copy link
Member

tqchen commented Jul 20, 2020

Ideally we should instead construct the test case to be reliable to the randomness and avoid reliance on the random seeds. Let us simply disable the flaky test for now knowing that they are not supported well.

@tqchen tqchen added the status: need update need update based on feedbacks label Jul 20, 2020
This was referenced Jul 22, 2020
Closed
Merged
@anijain2305
Copy link
Contributor Author

@tqchen This is ready to go in!

@lhutton1 lhutton1 mentioned this pull request Jul 23, 2020
Merged
@tqchen tqchen merged commit 4a1089d into apache:master Jul 24, 2020
trevor-m pushed a commit to trevor-m/tvm that referenced this pull request Aug 26, 2020
@anijain2305
[Flaky] TFLite quantized conv test (apache#6084)
8076724
trevor-m pushed a commit to trevor-m/tvm that referenced this pull request Aug 26, 2020
@anijain2305
[Flaky] TFLite quantized conv test (apache#6084)
6d68dd6
trevor-m pushed a commit to trevor-m/tvm that referenced this pull request Aug 26, 2020
@anijain2305
[Flaky] TFLite quantized conv test (apache#6084)
2afe40f
trevor-m pushed a commit to trevor-m/tvm that referenced this pull request Sep 2, 2020
@anijain2305
[Flaky] TFLite quantized conv test (apache#6084)
0f8e7b5
trevor-m pushed a commit to neo-ai/tvm that referenced this pull request Sep 3, 2020
@anijain2305 @trevor-m
[Flaky] TFLite quantized conv test (apache#6084)
be8efb3
@ZihengJiang ZihengJiang mentioned this pull request Sep 25, 2020
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@tqchen tqchen tqchen approved these changes

Assignees
No one assigned
Labels
status: need update need update based on feedbacks
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@anijain2305 @tqchen