Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MXNet official tutorial model to tensorflow #351

Closed
ShoufaChen opened this issue Aug 8, 2018 · 8 comments
Closed

MXNet official tutorial model to tensorflow #351

ShoufaChen opened this issue Aug 8, 2018 · 8 comments

Comments

@ShoufaChen
Copy link

ShoufaChen commented Aug 8, 2018

Platform (like ubuntu 16.04/win10): ubuntu16.04 Anaconda

Python version:3.6

Source framework with version (like Tensorflow 1.4.1 with GPU):1.8 with GPU

Destination framework with version (like CNTK 2.3 with GPU): MXNet 1.2.1 with GPU

Pre-trained model path (webpath or webdisk path): https://drive.google.com/drive/folders/1MQUYEPYnvWRmGBPh338Le5DFYnq-yr7Z?usp=sharing

(this model is got from the MXNet official tutorial Saving and Loading Gluon Models
Running scripts:

 python -m mmdnn.conversion._script.convertToIR -f mxnet -n lenet-symbol.json -w lenet-0001.params -d resnet --inputShape 1 28 28

Error:

el_names=['softmax_label']) but input with name 'softmax_label' is not found in symbol.list_arguments(). Did you mean one of:
        data
  warnings.warn(msg)
Warning: MXNet Parser has not supported operator null with name data.
Warning: convert the null operator with name [data] into input layer.
infer_shape error. Arguments:
  data: (1, 1)
Traceback (most recent call last):
  File "/home/csf/anaconda3/envs/tf18/lib/python3.6/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/home/csf/anaconda3/envs/tf18/lib/python3.6/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/home/csf/anaconda3/envs/tf18/lib/python3.6/site-packages/mmdnn/conversion/_script/convertToIR.py", line 199, in <module>
    _main()
  File "/home/csf/anaconda3/envs/tf18/lib/python3.6/site-packages/mmdnn/conversion/_script/convertToIR.py", line 194, in _main
    ret = _convert(args)
  File "/home/csf/anaconda3/envs/tf18/lib/python3.6/site-packages/mmdnn/conversion/_script/convertToIR.py", line 117, in _convert
    parser.run(args.dstPath)
  File "/home/csf/anaconda3/envs/tf18/lib/python3.6/site-packages/mmdnn/conversion/common/DataStructure/parser.py", line 22, in run
    self.gen_IR()
  File "/home/csf/anaconda3/envs/tf18/lib/python3.6/site-packages/mmdnn/conversion/mxnet/mxnet_parser.py", line 262, in gen_IR
    func(current_node)
  File "/home/csf/anaconda3/envs/tf18/lib/python3.6/site-packages/mmdnn/conversion/mxnet/mxnet_parser.py", line 424, in rename_Convolution
    self.set_output_shape(source_node, IR_node)
  File "/home/csf/anaconda3/envs/tf18/lib/python3.6/site-packages/mmdnn/conversion/mxnet/mxnet_parser.py", line 281, in set_output_shape
    arg_shape, output_shape, aux_shape = sym.infer_shape(data = self.data_shape)
  File "/home/csf/anaconda3/envs/tf18/lib/python3.6/site-packages/mxnet/symbol/symbol.py", line 990, in infer_shape
    res = self._infer_shape_impl(False, *args, **kwargs)
  File "/home/csf/anaconda3/envs/tf18/lib/python3.6/site-packages/mxnet/symbol/symbol.py", line 1120, in _infer_shape_impl
    ctypes.byref(complete)))
  File "/home/csf/anaconda3/envs/tf18/lib/python3.6/site-packages/mxnet/base.py", line 149, in check_call
    raise MXNetError(py_str(_LIB.MXGetLastError()))
mxnet.base.MXNetError: Error in operator hybridsequential0_conv0_fwd: [09:51:51] src/operator/nn/convolution.cc:143: Check failed: dshp.ndim() == 4U (2 vs. 4) Input data should be 4D in batch-num_filter-y-x

Stack trace returned 10 entries:
[bt] (0) /home/csf/anaconda3/envs/tf18/lib/python3.6/site-packages/mxnet/libmxnet.so(+0x31f81a) [0x7f03434c581a]
[bt] (1) /home/csf/anaconda3/envs/tf18/lib/python3.6/site-packages/mxnet/libmxnet.so(+0x31fe41) [0x7f03434c5e41]
[bt] (2) /home/csf/anaconda3/envs/tf18/lib/python3.6/site-packages/mxnet/libmxnet.so(+0x54a1e1) [0x7f03436f01e1]
[bt] (3) /home/csf/anaconda3/envs/tf18/lib/python3.6/site-packages/mxnet/libmxnet.so(+0x24de24a) [0x7f034568424a]
[bt] (4) /home/csf/anaconda3/envs/tf18/lib/python3.6/site-packages/mxnet/libmxnet.so(+0x24e0b40) [0x7f0345686b40]
[bt] (5) /home/csf/anaconda3/envs/tf18/lib/python3.6/site-packages/mxnet/libmxnet.so(MXSymbolInferShape+0x15aa) [0x7f03456008ba]
[bt] (6) /home/csf/anaconda3/envs/tf18/lib/python3.6/lib-dynload/../../libffi.so.6(ffi_call_unix64+0x4c) [0x7f0371dc1ec0]
[bt] (7) /home/csf/anaconda3/envs/tf18/lib/python3.6/lib-dynload/../../libffi.so.6(ffi_call+0x22d) [0x7f0371dc187d]
[bt] (8) /home/csf/anaconda3/envs/tf18/lib/python3.6/lib-dynload/_ctypes.cpython-36m-x86_64-linux-gnu.so(_ctypes_callproc+0x2ce) [0x7f0371fd6e2e]
[bt] (9) /home/csf/anaconda3/envs/tf18/lib/python3.6/lib-dynload/_ctypes.cpython-36m-x86_64-linux-gnu.so(+0x12865) [0x7f0371fd7865]
@namizzz
Copy link
Contributor

namizzz commented Aug 8, 2018

Hi @ShoufaChen ,The error shows "Input data should be 4D in batch-num_filter-y-x". you may need to use:

--inputShape 1,28,28
not
--inputShape 1 28 28

@ShoufaChen
Copy link
Author

ShoufaChen commented Aug 8, 2018

After change the `--inputShaep`` I met another queation:

Running scripts:

python -m mmdnn.conversion._script.convertToIR -f mxnet -n lenet-symbol.json -w lenet-0001.params -d tflenet --inputShape 1,28,28

Error:

/home/csf/anaconda3/envs/tf18/lib/python3.6/site-packages/mxnet/module/base_module.py:54: UserWarning: You created Module with Module(..., label_names=['softmax_label']) but input with name 'softmax_label' is not found in symbol.list_arguments(). Did you mean one of:
        data
  warnings.warn(msg)
Warning: MXNet Parser has not supported operator null with name data.
Warning: convert the null operator with name [data] into input layer.
Traceback (most recent call last):
  File "/home/csf/anaconda3/envs/tf18/lib/python3.6/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/home/csf/anaconda3/envs/tf18/lib/python3.6/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/home/csf/anaconda3/envs/tf18/lib/python3.6/site-packages/mmdnn/conversion/_script/convertToIR.py", line 199, in <module>
    _main()
  File "/home/csf/anaconda3/envs/tf18/lib/python3.6/site-packages/mmdnn/conversion/_script/convertToIR.py", line 194, in _main
    ret = _convert(args)
  File "/home/csf/anaconda3/envs/tf18/lib/python3.6/site-packages/mmdnn/conversion/_script/convertToIR.py", line 117, in _convert
    parser.run(args.dstPath)
  File "/home/csf/anaconda3/envs/tf18/lib/python3.6/site-packages/mmdnn/conversion/common/DataStructure/parser.py", line 22, in run
    self.gen_IR()
  File "/home/csf/anaconda3/envs/tf18/lib/python3.6/site-packages/mmdnn/conversion/mxnet/mxnet_parser.py", line 262, in gen_IR
    func(current_node)
  File "/home/csf/anaconda3/envs/tf18/lib/python3.6/site-packages/mmdnn/conversion/mxnet/mxnet_parser.py", line 495, in rename_Convolution
    weight = self.weight_data.get(source_node.name + "_weight").asnumpy()
AttributeError: 'NoneType' object has no attribute 'asnumpy

@namizzz
Copy link
Contributor

namizzz commented Aug 8, 2018

Hi @ShoufaChen , if you are using Gluon not the original MXNet. The name of weights and bias is different . You can refer to this #208.

@hellolingling
Copy link

hellolingling commented Aug 20, 2018

@ShoufaChen I also have the same problem as you. When changing the -- input shape as 3,112,112, I also got the same problem as you. Have you fixed your problem?

@namizzz
Copy link
Contributor

namizzz commented Aug 21, 2018

Hello @hellolingling , if you are using Gluon not the original MXNet. The name of weights and bias is different . You can refer to this #208

@hellolingling
Copy link

Hi, @namizzz thank you for your reply!
But, I referred to #208, and changed the code as suggested. And I am not using Gluon. I met the same problem as shown below:
Input:
python -m mmdnn.conversion._script.convertToIR -f mxnet -n model-symbol.json -w model-0000.params -d model-descr512 --inputShape 3,112,112

<mmdnn.conversion.mxnet.mxnet_parser.MXNetParser object at 0x7fdd6b129f10> Warning: MXNet Parser has not supported operator null with name data. Warning: convert the null operator with name [data] into input layer. Traceback (most recent call last): File "/home/zhulingling/anaconda2/lib/python2.7/runpy.py", line 174, in _run_module_as_main "__main__", fname, loader, pkg_name) File "/home/zhulingling/anaconda2/lib/python2.7/runpy.py", line 72, in _run_code exec code in run_globals File "/home/zhulingling/workspace/model_conversion/MMdnn/mmdnn/conversion/_script/convertToIR.py", line 207, in <module> _main() File "/home/zhulingling/workspace/model_conversion/MMdnn/mmdnn/conversion/_script/convertToIR.py", line 201, in _main ret = _convert(args) File "/home/zhulingling/workspace/model_conversion/MMdnn/mmdnn/conversion/_script/convertToIR.py", line 121, in _convert parser.run(args.dstPath) File "mmdnn/conversion/common/DataStructure/parser.py", line 22, in run self.gen_IR() File "mmdnn/conversion/mxnet/mxnet_parser.py", line 264, in gen_IR func(current_node) File "mmdnn/conversion/mxnet/mxnet_parser.py", line 400, in rename_FullyConnected weight = self.weight_data.get(source_node.name.replace("fwd","weight")).asnumpy().transpose((1, 0)) AttributeError: 'NoneType' object has no attribute 'asnumpy'
Can you help solve the problem?

@hellolingling
Copy link

Hi, @namizzz ,according the issue https://github.com/Microsoft/MMdnn/issues/231.
I changed the
File "mmdnn/conversion/mxnet/mxnet_parser.py", line 401, in rename_FullyConnected weight = self.weight_data.get(source_node.name + "_weight").asnumpy().transpose((1, 0))
into
weight = self.weight_data.get("fc1_weight").asnumpy().transpose((1, 0)).
This worked, thank you!

@hellolingling
Copy link

hellolingling commented Aug 21, 2018

refer to #231

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants