[tvmc] command line driver 'compile' (part 2/4) #6302

leandron · 2020-08-19T08:30:16Z

This is a follow-up PR on top of #6112, introducing compile subcommand on tvmc or python -m tvm.driver.tvmc

Add 'compile' subcommand into tvmc (tvm.driver.tvmc)
Add frontends: Keras, ONNX, TensorFlow, tflite, PyTorch
Add tests for the 'compile' subcommand
Add frontend dependencies on setup.py to make TVM python package ready-to-use on install

Known limitations:

It covers a subset of TVM supported frontends: Keras, ONNX, tflite, TensorFlow and PyTorch
It assumes graph runtime only

There are still 2 patches to be submitted: tvmc tune and tvmc run, both depending on this.

In case you want to have a look and test compile, you can use the --dump-codegen ll or --dump-codegen asm or --dump-codegen relay to check the output module as source.

A sample usage would look like (assumes TVM is built and working):

wget https://storage.googleapis.com/download.tensorflow.org/models/mobilenet_v1_2018_08_02/mobilenet_v1_1.0_224_quant.tgz

tar xvvzf mobilenet_v1_1.0_224_quant.tgz

python -m tvm.driver.tvmc compile --target=llvm --output model.tar --dump-code relay  mobilenet_v1_1.0_224_quant.tflite

cat model.relay

# should output mobilenet as relay

@tqchen @comaniac @jroesch, can you have a look?

leandron · 2020-08-19T08:34:05Z

Also cc @tom-gall who sent a patch on top of my previous PR adding supporting AArch32. Can you send that one again, once this one is merged?

tom-gall · 2020-08-19T13:44:13Z

Also cc @tom-gall who sent a patch on top of my previous PR adding supporting AArch32. Can you send that one again, once this one is merged?

Sure happy to.

comaniac

Reviewed the argument interface only and have some comments. I'd prefer to have consents on the interface before diving into the implementations.

python/tvm/driver/tvmc/__init__.py

python/tvm/driver/tvmc/compiler.py

tqchen · 2020-08-20T18:22:37Z

One thing to note is that given we are updating the target convention to use the json as an input, it might be helpful to think about what is the target that can be accepted from the command line. cc @comaniac @junrushao1994

junrushao · 2020-08-20T18:24:46Z

@tqchen I am thinking about unifying target creation from legacy string, JSON file and tags.

comaniac · 2020-08-20T19:03:08Z

One thing to note is that given we are updating the target convention to use the json as an input, it might be helpful to think about what is the target that can be accepted from the command line. cc @comaniac @junrushao1994

Zhi and I had a discussion with ARM folks today and here is a rough proposal:

--target could accept either a target or a JSON file path.
In the case of target string, it should match either a predefined tag (e.g., rasp4b), or it has to be a valid target string that can be parsed by target.create().
In the case of JSON file, it must be valid to be parsed by target JSON parser.

As a result, the users that know nothing about the target can simply provide a tag. On ther other hand, advance users can still specify targets in any supported formats.

python/tvm/driver/tvmc/common.py

python/tvm/driver/tvmc/compiler.py

python/tvm/driver/tvmc/frontends.py

tests/python/driver/tvmc/test_frontends.py

python/tvm/driver/tvmc/common.py

leandron · 2020-09-09T16:03:13Z

I updated it removing a skipped test due to #6410, which is now fixed.

leandron · 2020-09-10T15:45:03Z

There was an issue we discovered with ConvertLayout when running the new tests introduced here, once #6442 is merged, all the tests should pass.

@comaniac would you mind having another look into this PR?

comaniac

Thanks for the changes. It looks much better now.

python/setup.py

python/tvm/driver/tvmc/compiler.py

python/tvm/driver/tvmc/frontends.py

tests/python/driver/tvmc/test_frontends.py

comaniac · 2020-09-10T18:04:29Z

cc @masahi @junrushao1994 @tqchen for a final review.

leandron · 2020-09-11T10:13:23Z

Thanks @comaniac - I submitted a new version incorporating pretty much all the feedback (apart from the Target one, details in the comment).

One test is still expected to fail due to #6442, which is blocked by the Python linting massive refactoring #6448 to come.

leandron · 2020-09-16T15:17:58Z

I recently rebased with the latest changes and un-skipped the test fixed by #6442

@tqchen @comaniac @junrushao1994 would you mind having another look into this PR?

comaniac

LGTM. We can merge it after passing the CI.
Thanks for the efforts :)

python/setup.py

* Add 'compile' subcommand into tvmc (tvm.driver.tvmc) * Add frontends: Keras, ONNX, TensorFlow, tflite, PyTorch * Add tests for the 'compile' subcommand * Enable command line driver tests as part of integration tests * Skip tests if the cross-compilation toolchain is not installed Co-authored-by: Marcus Shawcroft <marcus.shawcroft@arm.com> Co-authored-by: Matthew Barrett <Matthew.Barrett@arm.com> Co-authored-by: Dmitriy Smirnov <dmitriy.smirnov@arm.com> Co-authored-by: Luke Hutton <luke.hutton@arm.com> Co-authored-by: Giuseppe Rossini <giuseppe.rossini@arm.com> Co-authored-by: Matthew Barrett <matthew.barrett@arm.com> Co-authored-by: Elen Kalda <elen.kalda@arm.com> Co-authored-by: Ramana Radhakrishnan <ramana.radhakrishnan@arm.com> Co-authored-by: Jeremy Johnson <jeremy.johnson@arm.com> Co-authored-by: Ina Dobreva <Ina.Dobreva@arm.com>

…ages

Co-authored-by: Cody Yu <comaniac0422@gmail.com>

python/setup.py

tqchen · 2020-09-18T18:37:41Z

Thanks @leandron @comaniac @masahi !

* [tvmc] command line driver 'compile' (part 2/4) * Add 'compile' subcommand into tvmc (tvm.driver.tvmc) * Add frontends: Keras, ONNX, TensorFlow, tflite, PyTorch * Add tests for the 'compile' subcommand * Enable command line driver tests as part of integration tests * Skip tests if the cross-compilation toolchain is not installed Co-authored-by: Marcus Shawcroft <marcus.shawcroft@arm.com> Co-authored-by: Matthew Barrett <Matthew.Barrett@arm.com> Co-authored-by: Dmitriy Smirnov <dmitriy.smirnov@arm.com> Co-authored-by: Luke Hutton <luke.hutton@arm.com> Co-authored-by: Giuseppe Rossini <giuseppe.rossini@arm.com> Co-authored-by: Matthew Barrett <matthew.barrett@arm.com> Co-authored-by: Elen Kalda <elen.kalda@arm.com> Co-authored-by: Ramana Radhakrishnan <ramana.radhakrishnan@arm.com> Co-authored-by: Jeremy Johnson <jeremy.johnson@arm.com> Co-authored-by: Ina Dobreva <Ina.Dobreva@arm.com> * tvmc: adjust TODOs * tvmc: fix linting errors * Address code-review comments * Adjust pytest fixture to not break when there is no tensorflow * Fix frontend tests, to cope with different frameworks in different images * Apply suggestions from code review Co-authored-by: Cody Yu <comaniac0422@gmail.com> * Fix lint and code-review issues * Re-format with black. * tvmc: Move dependencies to extras_requires Co-authored-by: Marcus Shawcroft <marcus.shawcroft@arm.com> Co-authored-by: Matthew Barrett <Matthew.Barrett@arm.com> Co-authored-by: Dmitriy Smirnov <dmitriy.smirnov@arm.com> Co-authored-by: Luke Hutton <luke.hutton@arm.com> Co-authored-by: Giuseppe Rossini <giuseppe.rossini@arm.com> Co-authored-by: Elen Kalda <elen.kalda@arm.com> Co-authored-by: Ramana Radhakrishnan <ramana.radhakrishnan@arm.com> Co-authored-by: Jeremy Johnson <jeremy.johnson@arm.com> Co-authored-by: Ina Dobreva <Ina.Dobreva@arm.com> Co-authored-by: Cody Yu <comaniac0422@gmail.com>

OneIsAll-w · 2021-03-15T08:34:09Z

tvmc first-run with 'compile' problem
I’m attempting to compile the resnet-50-v2.7.onnx model using the command :
python -m tvm.driver.tvmc compile --target "llvm" --output compiled_module.tar resnet50-v2-7.onnx
and I’m running into an error:
WARNING:root:Attribute momentum is ignored in relay.sym.batch_norm
WARNING:root:Attribute spatial is ignored in relay.sym.batch_norm
Traceback (most recent call last):
File "/home/tw/anaconda3/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"main", mod_spec)
File "/home/tw/anaconda3/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/home/tw/tvm/python/tvm/driver/tvmc/main.py", line 24, in
tvmc.main.main()
File "/home/tw/tvm/python/tvm/driver/tvmc/main.py", line 92, in main
sys.exit(_main(sys.argv[1:]))
File "/home/tw/tvm/python/tvm/driver/tvmc/main.py", line 85, in _main
return args.func(args)
File "/home/tw/tvm/python/tvm/driver/tvmc/compiler.py", line 114, in drive_compile
args.desired_layout,
File "/home/tw/tvm/python/tvm/driver/tvmc/compiler.py", line 182, in compile_model
with open(target) as target_file:
IsADirectoryError: [Errno 21] Is a directory: 'llvm'
Therefore,I want to ask you this problem,Where might I be going wrong?

leandron · 2021-03-15T10:04:13Z

tvmc first-run with 'compile' problem
I’m attempting to compile the resnet-50-v2.7.onnx model using the command :
python -m tvm.driver.tvmc compile --target "llvm" --output compiled_module.tar resnet50-v2-7.onnx
and I’m running into an error:
WARNING:root:Attribute momentum is ignored in relay.sym.batch_norm
WARNING:root:Attribute spatial is ignored in relay.sym.batch_norm
Traceback (most recent call last):
File "/home/tw/anaconda3/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"main", mod_spec)
File "/home/tw/anaconda3/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/home/tw/tvm/python/tvm/driver/tvmc/main.py", line 24, in
tvmc.main.main()
File "/home/tw/tvm/python/tvm/driver/tvmc/main.py", line 92, in main
sys.exit(_main(sys.argv[1:]))
File "/home/tw/tvm/python/tvm/driver/tvmc/main.py", line 85, in _main
return args.func(args)
File "/home/tw/tvm/python/tvm/driver/tvmc/compiler.py", line 114, in drive_compile
args.desired_layout,
File "/home/tw/tvm/python/tvm/driver/tvmc/compiler.py", line 182, in compile_model
with open(target) as target_file:
IsADirectoryError: [Errno 21] Is a directory: 'llvm'
Therefore,I want to ask you this problem,Where might I be going wrong?

This is a small issue, that only reproduce because you probably have an llvm directory in your current directory. I just submitted a fix to it in #7663. Thanks for reporting.

OneIsAll-w · 2021-03-15T11:41:01Z

Thanks a lot @leandron
but I still have some problems in compiling the resnet-50-v2.7.onnx model using the command :
python -m tvm.driver.tvmc compile --target "llvm" --output compiled_module.tar resnet50-v2-7.onnx
and I’m running into an error:
WARNING:root:Attribute momentum is ignored in relay.sym.batch_norm
WARNING:root:Attribute spatial is ignored in relay.sym.batch_norm
Traceback (most recent call last):
File "/home/tw/anaconda3/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"main", mod_spec)
File "/home/tw/anaconda3/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/home/tw/tvm/python/tvm/driver/tvmc/main.py", line 24, in
tvmc.main.main()
File "/home/tw/tvm/python/tvm/driver/tvmc/main.py", line 92, in main
sys.exit(_main(sys.argv[1:]))
File "/home/tw/tvm/python/tvm/driver/tvmc/main.py", line 85, in _main
return args.func(args)
File "/home/tw/tvm/python/tvm/driver/tvmc/compiler.py", line 114, in drive_compile
args.desired_layout,
File "/home/tw/tvm/python/tvm/driver/tvmc/compiler.py", line 189, in compile_model
tvm_target = tvm.target.Target(target)
File "/home/tw/tvm/python/tvm/target/target.py", line 92, in init
self.init_handle_by_constructor(_ffi_api.Target, tag_or_str_or_dict)
File "/home/tw/tvm/python/tvm/_ffi/_ctypes/object.py", line 131, in init_handle_by_constructor
handle = init_by_constructor(fconstructor, args)
File "/home/tw/tvm/python/tvm/_ffi/_ctypes/packed_func.py", line 246, in init_handle_by_constructor
values, tcodes, num_args = _make_tvm_args(args, temp_args)
File "/home/tw/tvm/python/tvm/_ffi/_ctypes/packed_func.py", line 163, in _make_tvm_args
values[i].v_str = c_str(arg)
File "/home/tw/tvm/python/tvm/_ffi/base.py", line 88, in c_str
return ctypes.c_char_p(string.encode("utf-8"))
UnicodeEncodeError: 'utf-8' codec can't encode characters in position 0-1: surrogates not allowed
Sorry，I still cannot solve this problem

leandron · 2021-03-15T12:13:42Z

Thanks a lot @leandron
but I still have some problems in compiling the resnet-50-v2.7.onnx model using the command :
python -m tvm.driver.tvmc compile --target "llvm" --output compiled_module.tar resnet50-v2-7.onnx
and I’m running into an error:
WARNING:root:Attribute momentum is ignored in relay.sym.batch_norm
WARNING:root:Attribute spatial is ignored in relay.sym.batch_norm
Traceback (most recent call last):
File "/home/tw/anaconda3/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"main", mod_spec)
File "/home/tw/anaconda3/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/home/tw/tvm/python/tvm/driver/tvmc/main.py", line 24, in
tvmc.main.main()
File "/home/tw/tvm/python/tvm/driver/tvmc/main.py", line 92, in main
sys.exit(_main(sys.argv[1:]))
File "/home/tw/tvm/python/tvm/driver/tvmc/main.py", line 85, in _main
return args.func(args)
File "/home/tw/tvm/python/tvm/driver/tvmc/compiler.py", line 114, in drive_compile
args.desired_layout,
File "/home/tw/tvm/python/tvm/driver/tvmc/compiler.py", line 189, in compile_model
tvm_target = tvm.target.Target(target)
File "/home/tw/tvm/python/tvm/target/target.py", line 92, in init
self.init_handle_by_constructor(_ffi_api.Target, tag_or_str_or_dict)
File "/home/tw/tvm/python/tvm/_ffi/_ctypes/object.py", line 131, in init_handle_by_constructor
handle = init_by_constructor(fconstructor, args)
File "/home/tw/tvm/python/tvm/_ffi/_ctypes/packed_func.py", line 246, in init_handle_by_constructor
values, tcodes, num_args = _make_tvm_args(args, temp_args)
File "/home/tw/tvm/python/tvm/_ffi/_ctypes/packed_func.py", line 163, in _make_tvm_args
values[i].v_str = c_str(arg)
File "/home/tw/tvm/python/tvm/_ffi/base.py", line 88, in c_str
return ctypes.c_char_p(string.encode("utf-8"))
UnicodeEncodeError: 'utf-8' codec can't encode characters in position 0-1: surrogates not allowed
Sorry，I still cannot solve this problem

It seems it has something to do with the actual string you're sending on the command line. It is not strictly a bug in tvmc at this point. Can you please file an issue so that we can investigate that?

OneIsAll-w · 2021-03-15T12:20:49Z

Thanks a lot @leandron
but I still have some problems in compiling the resnet-50-v2.7.onnx model using the command :
python -m tvm.driver.tvmc compile --target "llvm" --output compiled_module.tar resnet50-v2-7.onnx
and I’m running into an error:
WARNING:root:Attribute momentum is ignored in relay.sym.batch_norm
WARNING:root:Attribute spatial is ignored in relay.sym.batch_norm
Traceback (most recent call last):
File "/home/tw/anaconda3/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"main", mod_spec)
File "/home/tw/anaconda3/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/home/tw/tvm/python/tvm/driver/tvmc/main.py", line 24, in
tvmc.main.main()
File "/home/tw/tvm/python/tvm/driver/tvmc/main.py", line 92, in main
sys.exit(_main(sys.argv[1:]))
File "/home/tw/tvm/python/tvm/driver/tvmc/main.py", line 85, in _main
return args.func(args)
File "/home/tw/tvm/python/tvm/driver/tvmc/compiler.py", line 114, in drive_compile
args.desired_layout,
File "/home/tw/tvm/python/tvm/driver/tvmc/compiler.py", line 189, in compile_model
tvm_target = tvm.target.Target(target)
File "/home/tw/tvm/python/tvm/target/target.py", line 92, in init
self.init_handle_by_constructor(_ffi_api.Target, tag_or_str_or_dict)
File "/home/tw/tvm/python/tvm/_ffi/_ctypes/object.py", line 131, in init_handle_by_constructor
handle = init_by_constructor(fconstructor, args)
File "/home/tw/tvm/python/tvm/_ffi/_ctypes/packed_func.py", line 246, in init_handle_by_constructor
values, tcodes, num_args = _make_tvm_args(args, temp_args)
File "/home/tw/tvm/python/tvm/_ffi/_ctypes/packed_func.py", line 163, in _make_tvm_args
values[i].v_str = c_str(arg)
File "/home/tw/tvm/python/tvm/_ffi/base.py", line 88, in c_str
return ctypes.c_char_p(string.encode("utf-8"))
UnicodeEncodeError: 'utf-8' codec can't encode characters in position 0-1: surrogates not allowed
Sorry，I still cannot solve this problem

It seems it has something to do with the actual string you're sending on the command line. It is not strictly a bug in tvmc at this point. Can you please file an issue so that we can investigate that?

I didn't change anything, I still used the command:
python -m tvm.driver.tvmc compile --target "llvm" --output compiled_module.tar resnet50-v2-7.onnx

comaniac · 2021-03-15T17:08:04Z

@OneIsAll-w please do not use the closed PR for bug reporting and discussion. It would be better to create a post in https://discuss.tvm.apache.org/ and point to this PR.

leandron mentioned this pull request Aug 19, 2020

[TEST][FLAKY] test_auto_scheduler_cost_model.py::test_xgb_model #6305

Closed

comaniac requested changes Aug 19, 2020

View reviewed changes

ZihengJiang assigned comaniac Aug 21, 2020

ZihengJiang added status: need review status: review in progress and removed status: need review labels Aug 21, 2020

leandron mentioned this pull request Aug 21, 2020

[Target] Add python binding to new JSON target construction. #6315

Merged

comaniac requested changes Aug 28, 2020

View reviewed changes

This was referenced Sep 7, 2020

AttributeError during ConvertLayout to NHWC #6410

Closed

[FLAKY] relay/test_op_level1.py::test_binary_op failing intermittently #6417

Closed

comaniac requested changes Sep 10, 2020

View reviewed changes

comaniac approved these changes Sep 16, 2020

View reviewed changes

tqchen approved these changes Sep 16, 2020

View reviewed changes

tqchen requested changes Sep 16, 2020

View reviewed changes

python/setup.py Outdated Show resolved Hide resolved

leandron and others added 6 commits September 17, 2020 11:25

tvmc: adjust TODOs

143f7e0

tvmc: fix linting errors

5ec6551

Address code-review comments

1e6ef0c

Adjust pytest fixture to not break when there is no tensorflow

9db5680

Fix frontend tests, to cope with different frameworks in different im…

c217622

…ages

leandron and others added 3 commits September 17, 2020 11:25

Apply suggestions from code review

9887550

Co-authored-by: Cody Yu <comaniac0422@gmail.com>

Fix lint and code-review issues

7fd5afa

Re-format with black.

dd2af42

tqchen requested changes Sep 17, 2020

View reviewed changes

python/setup.py Outdated Show resolved Hide resolved

tvmc: Move dependencies to extras_requires

c55f8f2

leandron requested a review from tqchen September 18, 2020 08:48

tqchen approved these changes Sep 18, 2020

View reviewed changes

tqchen merged commit 292b640 into apache:master Sep 18, 2020

tqchen added status: accepted and removed status: review in progress labels Sep 18, 2020

ZihengJiang mentioned this pull request Sep 25, 2020

TVM v0.7 Release Note Candidate #6486

Closed

leandron deleted the tvmc_compile_feature branch October 1, 2020 13:55

leandron mentioned this pull request Mar 15, 2021

[TVMC] Fix to check whether a path passed to --target is strictly a file #7663

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[tvmc] command line driver 'compile' (part 2/4) #6302

[tvmc] command line driver 'compile' (part 2/4) #6302

leandron commented Aug 19, 2020 •

edited

Loading

leandron commented Aug 19, 2020

tom-gall commented Aug 19, 2020

comaniac left a comment

tqchen commented Aug 20, 2020

junrushao commented Aug 20, 2020

comaniac commented Aug 20, 2020

leandron commented Sep 9, 2020

leandron commented Sep 10, 2020 •

edited

Loading

comaniac left a comment

comaniac commented Sep 10, 2020

leandron commented Sep 11, 2020 •

edited

Loading

leandron commented Sep 16, 2020 •

edited

Loading

comaniac left a comment

tqchen commented Sep 18, 2020

OneIsAll-w commented Mar 15, 2021

leandron commented Mar 15, 2021

OneIsAll-w commented Mar 15, 2021

leandron commented Mar 15, 2021

OneIsAll-w commented Mar 15, 2021

comaniac commented Mar 15, 2021

[tvmc] command line driver 'compile' (part 2/4) #6302

[tvmc] command line driver 'compile' (part 2/4) #6302

Conversation

leandron commented Aug 19, 2020 • edited Loading

leandron commented Aug 19, 2020

tom-gall commented Aug 19, 2020

comaniac left a comment

Choose a reason for hiding this comment

tqchen commented Aug 20, 2020

junrushao commented Aug 20, 2020

comaniac commented Aug 20, 2020

leandron commented Sep 9, 2020

leandron commented Sep 10, 2020 • edited Loading

comaniac left a comment

Choose a reason for hiding this comment

comaniac commented Sep 10, 2020

leandron commented Sep 11, 2020 • edited Loading

leandron commented Sep 16, 2020 • edited Loading

comaniac left a comment

Choose a reason for hiding this comment

tqchen commented Sep 18, 2020

OneIsAll-w commented Mar 15, 2021

leandron commented Mar 15, 2021

OneIsAll-w commented Mar 15, 2021

leandron commented Mar 15, 2021

OneIsAll-w commented Mar 15, 2021

comaniac commented Mar 15, 2021

leandron commented Aug 19, 2020 •

edited

Loading

leandron commented Sep 10, 2020 •

edited

Loading

leandron commented Sep 11, 2020 •

edited

Loading

leandron commented Sep 16, 2020 •

edited

Loading