Skip to content
This repository has been archived by the owner on Nov 17, 2023. It is now read-only.

CI error in unix gpu test_quantization_gpu.test_quantized_conv #16830

Closed
leezu opened this issue Nov 15, 2019 · 9 comments
Closed

CI error in unix gpu test_quantization_gpu.test_quantized_conv #16830

leezu opened this issue Nov 15, 2019 · 9 comments
Labels
Flaky MKLDNN Quantization Issues/Feature Requests related to Quantization Test

Comments

@leezu
Copy link
Contributor

leezu commented Nov 15, 2019

more details

http://jenkins.mxnet-ci.amazon-ml.com/blue/rest/organizations/jenkins/pipelines/mxnet-validation/pipelines/unix-cpu/branches/PR-16810/runs/6/nodes/300/steps/700/log/?start=0

FAIL: test_quantization_mkldnn.test_quantized_conv
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/local/lib/python3.5/dist-packages/nose/case.py", line 198, in runTest
    self.test(*self.arg)
  File "/usr/local/lib/python3.5/dist-packages/nose/util.py", line 620, in newfunc
    return func(*arg, **kw)
  File "/work/mxnet/tests/python/mkl/../unittest/common.py", line 177, in test_new
    orig_test(*args, **kwargs)
  File "/work/mxnet/tests/python/mkl/../quantization/test_quantization.py", line 277, in test_quantized_conv
    check_quantized_conv((3, 4, 28, 28), (3, 3), 128, (1, 1), (1, 1), False, qdtype)
  File "/work/mxnet/tests/python/mkl/../quantization/test_quantization.py", line 273, in check_quantized_conv
    assert cond == 0
AssertionError: 
-------------------- >> begin captured stdout << ---------------------
skipped testing quantized_conv for mkldnn cpu int8 since it is not supported yet
skipped testing quantized_conv for mkldnn cpu int8 since it is not supported yet

--------------------- >> end captured stdout << ----------------------
-------------------- >> begin captured logging << --------------------
common: INFO: Setting test np/mx/python random seeds, use MXNET_TEST_SEED=752964439 to reproduce.
--------------------- >> end captured logging << ---------------------
@pengzhao-intel
Copy link
Contributor

@xinyu-intel could you help take a look ?

@pengzhao-intel pengzhao-intel added Quantization Issues/Feature Requests related to Quantization MKLDNN labels Nov 19, 2019
@xinyu-intel xinyu-intel mentioned this issue Nov 20, 2019
Merged
7 tasks
@haojin2
Copy link
Contributor

haojin2 commented Nov 21, 2019

Happening again: http://jenkins.mxnet-ci.amazon-ml.com/blue/organizations/jenkins/mxnet-validation%2Funix-cpu/detail/PR-16862/7/pipeline/299

======================================================================

FAIL: test_quantization_mkldnn.test_quantized_conv

----------------------------------------------------------------------

Traceback (most recent call last):

  File "/usr/local/lib/python3.5/dist-packages/nose/case.py", line 198, in runTest

    self.test(*self.arg)

  File "/usr/local/lib/python3.5/dist-packages/nose/util.py", line 620, in newfunc

    return func(*arg, **kw)

  File "/work/mxnet/tests/python/mkl/../unittest/common.py", line 177, in test_new

    orig_test(*args, **kwargs)

  File "/work/mxnet/tests/python/mkl/../quantization/test_quantization.py", line 277, in test_quantized_conv

    check_quantized_conv((3, 4, 28, 28), (3, 3), 128, (1, 1), (1, 1), False, qdtype)

  File "/work/mxnet/tests/python/mkl/../quantization/test_quantization.py", line 273, in check_quantized_conv

    assert cond == 0

AssertionError: 

-------------------- >> begin captured stdout << ---------------------

skipped testing quantized_conv for mkldnn cpu int8 since it is not supported yet

skipped testing quantized_conv for mkldnn cpu int8 since it is not supported yet



--------------------- >> end captured stdout << ----------------------

-------------------- >> begin captured logging << --------------------

common: INFO: Setting test np/mx/python random seeds, use MXNET_TEST_SEED=139132436 to reproduce.

--------------------- >> end captured logging << ---------------------

@ChaiBapchya
Copy link
Contributor

Happening again
http://jenkins.mxnet-ci.amazon-ml.com/blue/organizations/jenkins/mxnet-validation%2Funix-cpu/detail/PR-16894/9/pipeline
#16894

@haojin2
Copy link
Contributor

haojin2 commented Nov 26, 2019

Happening again: http://jenkins.mxnet-ci.amazon-ml.com/blue/organizations/jenkins/mxnet-validation%2Funix-cpu/detail/master/1325/pipeline


======================================================================

FAIL: test_quantization_mkldnn.test_quantized_conv

----------------------------------------------------------------------

Traceback (most recent call last):

  File "/usr/local/lib/python3.5/dist-packages/nose/case.py", line 198, in runTest

    self.test(*self.arg)

  File "/usr/local/lib/python3.5/dist-packages/nose/util.py", line 620, in newfunc

    return func(*arg, **kw)

  File "/work/mxnet/tests/python/mkl/../unittest/common.py", line 177, in test_new

    orig_test(*args, **kwargs)

  File "/work/mxnet/tests/python/mkl/../quantization/test_quantization.py", line 277, in test_quantized_conv

    check_quantized_conv((3, 4, 28, 28), (3, 3), 128, (1, 1), (1, 1), False, qdtype)

  File "/work/mxnet/tests/python/mkl/../quantization/test_quantization.py", line 273, in check_quantized_conv

    assert cond == 0

AssertionError: 

-------------------- >> begin captured stdout << ---------------------

skipped testing quantized_conv for mkldnn cpu int8 since it is not supported yet

skipped testing quantized_conv for mkldnn cpu int8 since it is not supported yet

@xinyu-intel @PatricZhao Anyone could take a look?

@pengzhao-intel
Copy link
Contributor

pengzhao-intel commented Nov 26, 2019 •
edited
Loading

@xinyu-intel @PatricZhao Anyone could take a look?

Yes, @xinyu-intel is looking for this and the PR is under the testing.

@pengzhao-intel
Copy link
Contributor

#16866

@haojin2
Copy link
Contributor

haojin2 commented Dec 9, 2019

Happening again: http://jenkins.mxnet-ci.amazon-ml.com/blue/organizations/jenkins/mxnet-validation%2Funix-cpu/detail/PR-17016/1/pipeline


======================================================================

FAIL: test_quantization_mkldnn.test_quantized_conv

----------------------------------------------------------------------

Traceback (most recent call last):

  File "/usr/local/lib/python3.5/dist-packages/nose/case.py", line 198, in runTest

    self.test(*self.arg)

  File "/usr/local/lib/python3.5/dist-packages/nose/util.py", line 620, in newfunc

    return func(*arg, **kw)

  File "/work/mxnet/tests/python/mkl/../unittest/common.py", line 177, in test_new

    orig_test(*args, **kwargs)

  File "/work/mxnet/tests/python/mkl/../quantization/test_quantization.py", line 277, in test_quantized_conv

    check_quantized_conv((3, 4, 28, 28), (3, 3), 128, (1, 1), (1, 1), False, qdtype)

  File "/work/mxnet/tests/python/mkl/../quantization/test_quantization.py", line 273, in check_quantized_conv

    assert cond == 0

AssertionError: 

-------------------- >> begin captured stdout << ---------------------

skipped testing quantized_conv for mkldnn cpu int8 since it is not supported yet

skipped testing quantized_conv for mkldnn cpu int8 since it is not supported yet



--------------------- >> end captured stdout << ----------------------

-------------------- >> begin captured logging << --------------------

common: INFO: Setting test np/mx/python random seeds, use MXNET_TEST_SEED=1860264925 to reproduce.

--------------------- >> end captured logging << ---------------------

@pengzhao-intel
Copy link
Contributor

skip the test now and please merge the code.

@TaoLv
Copy link
Member

TaoLv commented Jan 2, 2020

Closed via #16866

@TaoLv TaoLv closed this as completed Jan 2, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
Flaky MKLDNN Quantization Issues/Feature Requests related to Quantization Test
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@leezu @ChaiBapchya @haojin2 @TaoLv @pengzhao-intel