Mark test_masked_softmax as flaky

[leezu]

[2021-02-17T17:49:27.019Z] >       raise AssertionError(msg)
[2021-02-17T17:49:27.019Z] E       AssertionError: 
[2021-02-17T17:49:27.019Z] E       Items are not equal:
[2021-02-17T17:49:27.019Z] E       Error 1.014057 exceeds tolerance rtol=1.000000e-02, atol=2.000000e-03 (mismatch 0.002058%).
[2021-02-17T17:49:27.019Z] E       Location of maximum error: (2, 2, 1, 3, 7), EXPECTED_data=0.00046028, BACKWARD_data=0.00251389
[2021-02-17T17:49:27.019Z] E        ACTUAL: array([[[[[-0.        , -0.        , -0.        , ..., -0.        ,
[2021-02-17T17:49:27.019Z] E                  -0.        ,  0.        ],
[2021-02-17T17:49:27.019Z] E                 [ 0.        , -0.        , -0.        , ..., -0.        ,...
[2021-02-17T17:49:27.019Z] E        DESIRED: array([[[[[ 0.        ,  0.        ,  0.        , ...,  0.        ,
[2021-02-17T17:49:27.019Z] E                   0.        ,  0.        ],
[2021-02-17T17:49:27.019Z] E                 [ 0.        ,  0.        ,  0.        , ...,  0.        ,...

https://jenkins.mxnet-ci.amazon-ml.com/blue/rest/organizations/jenkins/pipelines/mxnet-validation/pipelines/unix-cpu/branches/PR-19906/runs/2/nodes/283/steps/394/log/?start=0

[mxnet-bot]

Hey @leezu , Thanks for submitting the PR
All tests are already queued to run once. If tests fail, you can trigger one or more tests again with the following commands:

• To trigger all jobs: @mxnet-bot run ci [all]
• To trigger specific jobs: @mxnet-bot run ci [job1, job2]

---

CI supported jobs: [unix-gpu, clang, website, edge, windows-cpu, sanity, unix-cpu, miscellaneous, centos-gpu, centos-cpu, windows-gpu]

---

Note:
Only following 3 categories can trigger CI :PR Author, MXNet Committer, Jenkins Admin.
All CI tests must pass before the PR can be merged.

[leezu]

Also
[2021-02-17T19:49:09.680Z] ________________ test_subgraph_exe1[sym14-op_names14-default] _________________
[2021-02-17T19:49:09.680Z] [gw3] win32 -- Python 3.7.0 C:\Python37\python.exe
[2021-02-17T19:49:09.680Z]
[2021-02-17T19:49:09.680Z] sym = , subgraph_backend = 'default'
[2021-02-17T19:49:09.680Z] op_names = ['sin', 'Convolution']
[2021-02-17T19:49:09.680Z]
[2021-02-17T19:49:09.680Z] @pytest.mark.parametrize('subgraph_backend', ['default', 'default_v2'])
[2021-02-17T19:49:09.680Z] @pytest.mark.parametrize('sym,op_names', get_graphs())
[2021-02-17T19:49:09.680Z] def test_subgraph_exe1(sym, subgraph_backend, op_names):
[2021-02-17T19:49:09.680Z] """Use the partitioned sym to _simple_bind an executor and compare the outputs
[2021-02-17T19:49:09.680Z] with those of the original executor"""
[2021-02-17T19:49:09.680Z] sym, _, _ = sym
[2021-02-17T19:49:09.680Z] out = SymbolHandle()
[2021-02-17T19:49:09.680Z] check_call(_LIB.MXBuildSubgraphByOpNames(sym.handle, c_str(subgraph_backend), mx_uint(len(op_names)),
[2021-02-17T19:49:09.680Z] c_str_array(op_names), ctypes.byref(out)))
[2021-02-17T19:49:09.680Z]
[2021-02-17T19:49:09.680Z] partitioned_sym = Symbol(out)
[2021-02-17T19:49:09.680Z] assert partitioned_sym.list_inputs() == sym.list_inputs()
[2021-02-17T19:49:09.680Z] assert partitioned_sym.list_arguments() == sym.list_arguments()
[2021-02-17T19:49:09.680Z] assert partitioned_sym.list_auxiliary_states() == sym.list_auxiliary_states()
[2021-02-17T19:49:09.680Z] > exe = sym._simple_bind(ctx=mx.current_context(), grad_req='null')
[2021-02-17T19:49:09.680Z]
[2021-02-17T19:49:09.680Z] tests\python\unittest\test_subgraph_op.py:137:
[2021-02-17T19:49:09.680Z] _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
[2021-02-17T19:49:09.680Z] windows_package\python\mxnet\symbol\symbol.py:1746: in _simple_bind
[2021-02-17T19:49:09.680Z] arg_shapes, _, aux_shapes = self.infer_shape(**kwargs)
[2021-02-17T19:49:09.680Z] windows_package\python\mxnet\symbol\symbol.py:1132: in infer_shape
[2021-02-17T19:49:09.680Z] res = self._infer_shape_impl(False, *args, **kwargs)
[2021-02-17T19:49:09.680Z] windows_package\python\mxnet\symbol\symbol.py:1267: in _infer_shape_impl
[2021-02-17T19:49:09.680Z] ctypes.byref(complete)))
[2021-02-17T19:49:09.680Z] _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
[2021-02-17T19:49:09.680Z]
[2021-02-17T19:49:09.680Z] ret = -1
[2021-02-17T19:49:09.680Z]
[2021-02-17T19:49:09.680Z] def check_call(ret):
[2021-02-17T19:49:09.680Z] """Check the return value of C API call.
[2021-02-17T19:49:09.680Z]
[2021-02-17T19:49:09.680Z] This function will raise an exception when an error occurs.
[2021-02-17T19:49:09.680Z] Wrap every API call with this function.
[2021-02-17T19:49:09.680Z]
[2021-02-17T19:49:09.680Z] Parameters
[2021-02-17T19:49:09.680Z] ----------
[2021-02-17T19:49:09.680Z] ret : int
[2021-02-17T19:49:09.680Z] return value from API calls.
[2021-02-17T19:49:09.680Z] """
[2021-02-17T19:49:09.680Z] if ret != 0:
[2021-02-17T19:49:09.680Z] > raise get_last_ffi_error()
[2021-02-17T19:49:09.680Z] E mxnet.base.MXNetError: MXNetError: Error in operator convolution20: Shape inconsistent, Provided = [1,0,2,2], inferred shape=(1,3,2,2)
[2021-02-17T19:49:09.680Z]
[2021-02-17T19:49:09.680Z] windows_package\python\mxnet\base.py:246: MXNetError

[leezu]

@mxnet-bot run ci [windows-cpu, windows-gpu]

[mxnet-bot]

Jenkins CI successfully triggered : [windows-cpu, windows-gpu]

[samskalicky]

Im ok disabling these tests, but we should have a plan to address this. otherwise we're just building the tech debt and forgetting about it. will just come back to bite us later.

[leezu]

@mxnet-bot run ci [centos-gpu]

[mxnet-bot]

Jenkins CI successfully triggered : [centos-gpu]