New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[microTVM][PyTorch][Tutorial]Adding a PyTorch tutorial for microTVM with CRT #13324

Merged

gromero merged 9 commits into apache:main from mehrdadh:microtvm/pytorch_tutorial

Nov 9, 2022

Member

mehrdadh commented Nov 8, 2022 •

edited

Loading

This PR adds a tutorial to compile/run a PyTorch model using microTVM CRT.

mehrdadh requested a review from gromero

November 8, 2022 17:24

Collaborator

tvm-bot commented Nov 8, 2022 •

edited

Loading

Thanks for contributing to TVM! Please refer to the contributing guidelines https://tvm.apache.org/docs/contribute/ for useful information and tips. Please request code reviews from Reviewers by @-ing them in a comment.

cc @alanmacd, @gromero, @yelite _{See #10317 for details}
Built docs for commit 19d87dd can be found here.

_{Generated by tvm-bot}

Collaborator

tvm-bot commented Nov 8, 2022

Thanks for contributing to TVM! Please refer to the contributing guidelines https://tvm.apache.org/docs/contribute/ for useful information and tips. Please request code reviews from Reviewers by @-ing them in a comment.

cc @alanmacd, @gromero, @yelite _{See #10317 for details}

_{Generated by tvm-bot}

alanmacd approved these changes

View reviewed changes

Contributor

alanmacd left a comment

this is great, thanks for writing it up!

src/runtime/crt/host/microtvm_api_server.py

		@@ -35,6 +37,18 @@

		IS_TEMPLATE = not os.path.exists(os.path.join(PROJECT_DIR, MODEL_LIBRARY_FORMAT_RELPATH))

		MEMORY_SIZE_BYTES = 2 * 1024 * 1024

Contributor

alanmacd Nov 8, 2022

nit: maybe add comment as to why this is default memory size

Member Author

mehrdadh Nov 8, 2022

added, it's not an interesting reason. It's basically chosen to pass CRT tests in TVM

Contributor

gromero Nov 8, 2022 •

edited

Loading

@mehrdadh I think that's actually interesting / important info: the constant is determined experimentally, it's really good to have comment about it as @alanmacd requested, I don't even consider it a nit :-)

mehrdadh added 3 commits

November 8, 2022 13:32


          [microTVM] Add Pytorch tutorial

58e4497


          update tutorial

7acff25


          add comment

091d332

mehrdadh force-pushed the microtvm/pytorch_tutorial branch from b7e242a to 091d332 Compare

November 8, 2022 21:32


          fix hook

mehrdadh force-pushed the microtvm/pytorch_tutorial branch from b1de307 to 5284076 Compare

November 8, 2022 22:14

gromero requested changes

View reviewed changes

Contributor

gromero left a comment

@mehrdadh Thanks for the tutorial! Nice.

I just have nits (see inline) and one question:

Could we add Wno-unused-variable flag to MODEL_CFLAGS? like:

diff --git a/src/runtime/crt/host/Makefile.template b/src/runtime/crt/host/Makefile.template
index a8e725ade..2caf7ba0b 100644
--- a/src/runtime/crt/host/Makefile.template
+++ b/src/runtime/crt/host/Makefile.template
@@ -22,7 +22,7 @@ CXXFLAGS ?= -Werror -Wall -std=c++11 -DTVM_HOST_USE_GRAPH_EXECUTOR_MODULE -DMEMO
 LDFLAGS ?= -Werror -Wall
 
 # Codegen produces spurious lines like: int32_t arg2_code = ((int32_t*)arg_type_ids)[(2)];
-MODEL_CFLAGS ?= -Wno-error=unused-variable -Wno-error=missing-braces -Wno-error=unused-const-variable
+MODEL_CFLAGS ?= -Wno-error=unused-variable -Wno-error=missing-braces -Wno-error=unused-const-variable -Wno-unused-variable
 
 AR ?= ${PREFIX}ar
 CC ?= ${PREFIX}gcc

Otherwise I get bazillions of unused variables warnings when I run the script which is not encouraging for people running / exploring the tutorial for the first time. -Wno-error=unused-variable is not enough to silence it. wdyt?

gallery/how_to/work_with_microtvm/micro_pytorch.py Outdated

+              # Define Target, Runtime and Executor
+              # -------------------------------
+              #
+              # In this tutorial we use AOT host driven executor. To compile the model

Contributor

gromero Nov 8, 2022

nit: host-driven?

Member Author

mehrdadh Nov 8, 2022

done

gallery/how_to/work_with_microtvm/micro_pytorch.py Outdated

+              from tvm.contrib.download import download_testdata
+              from tvm.relay.backend import Executor
+              #################################

Contributor

gromero Nov 8, 2022

nit: add one more # to "cover" the line below?

Member Author

mehrdadh Nov 8, 2022

done

gallery/how_to/work_with_microtvm/micro_pytorch.py Outdated

+              #################################
+              # Load a pre-trained PyTorch model
+              # -------------------------------

Contributor

gromero Nov 8, 2022

nit: add one more - to match end of line above?

Member Author

mehrdadh Nov 8, 2022

done

gallery/how_to/work_with_microtvm/micro_pytorch.py Outdated

+              shape_list = [(input_name, input_shape)]
+              relay_mod, params = relay.frontend.from_pytorch(scripted_model, shape_list)
+              #################################

Contributor

gromero Nov 8, 2022

nit: add more # chars to "cover" end of line below?

Member Author

mehrdadh Nov 8, 2022

done

gallery/how_to/work_with_microtvm/micro_pytorch.py Outdated

+              #################################
+              # Define Target, Runtime and Executor
+              # -------------------------------

Contributor

gromero Nov 8, 2022

nit: add enough - chars to align this to the end of line above?

Member Author

mehrdadh Nov 8, 2022

done

gallery/how_to/work_with_microtvm/micro_pytorch.py Outdated

+              # -------------------------------
+              #
+              # In this tutorial we use AOT host driven executor. To compile the model
+              # for an emulated embedded environment on an X86 machine we use C runtime (CRT)

Contributor

gromero Nov 8, 2022

s/X86/x86/

Member Author

mehrdadh Nov 8, 2022

done

gallery/how_to/work_with_microtvm/micro_pytorch.py Outdated

+              # In this tutorial we use AOT host driven executor. To compile the model
+              # for an emulated embedded environment on an X86 machine we use C runtime (CRT)
+              # and we use `host` micro target. Using this setup, TVM compiles the model
+              # for C runtime which can run on a X86 CPU machine with the same flow that

Contributor

gromero Nov 8, 2022

same here: x86 instead of X86

Member Author

mehrdadh Nov 8, 2022

done

gallery/how_to/work_with_microtvm/micro_pytorch.py Outdated



		# Simulate a microcontroller on the host machine. Uses the main() from `src/runtime/crt/host/main.cc`
		# To use physical hardware, replace "host" with something matching your hardware.

Contributor

gromero Nov 8, 2022

How about instead of using "something'" say "replace 'hosts' with another physical micro target, e.g. 'nrf52840' or 'mps2_an521' -- see more more target examples in micro_train.py and micro_tflite.py tutorials "?

Member Author

mehrdadh Nov 8, 2022

done

mehrdadh added 2 commits

November 8, 2022 15:42


          remove warning on unsed variable

39aa4df


          address comments

f34d3b8

Member Author

mehrdadh commented Nov 8, 2022

@gromero that's a good point. I added that compiler flag.

mehrdadh added 3 commits

November 8, 2022 16:01


          remove torch warninigs

31a8aea


          fix format

0e12e41


          cleanup micro_aot.py

19d87dd

gromero approved these changes

View reviewed changes

Contributor

gromero left a comment

@mehrdadh Thanks, LGTM.

gromero merged commit fbe174b into apache:main

mehrdadh deleted the microtvm/pytorch_tutorial branch

November 9, 2022 16:44

xinetzone pushed a commit to daobook/tvm that referenced this pull request


          [microTVM][CRT][DOCS] Add a PyTorch tutorial for microTVM with CRT (a…

c42d2eb

…pache#13324)

This commit adds a tutorial to compile and run a PyTorch model using   
microTVM, the AOT host-driven executor, and C runtime (CRT).

xinetzone pushed a commit to daobook/tvm that referenced this pull request


          [microTVM][CRT][DOCS] Add a PyTorch tutorial for microTVM with CRT (a…

588537b

…pache#13324)

This commit adds a tutorial to compile and run a PyTorch model using   
microTVM, the AOT host-driven executor, and C runtime (CRT).

leandron mentioned this pull request

TVM v0.11.0 Release Candidate Notes #13899

Closed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet