Contributing the STM32 port #7742

stoa · 2021-03-25T15:04:11Z

We propose to contribute the STM32 port of the TVM.
The STM32 are mainly bare-metal boards and often use-cases do not suppose using an OS: Linux, embed,RTOS,etc.
The STM32 development tools (ex, CubeMX) build standalone applications (no OS) that can be flashed to the board and executed.
In this TVM port, the TVM is used to compile AI networks into a C code compliant with the STM32 AI API.
In particular, the TVM is used to build a C module (kernels library + JSON graph + params); then the stm32.emitter.py generates the C implementation from the TVM module's JSON graph and writes out a C implementation of the kernels.
The generated C code can then be included as part of a larger STM32 application, and managed within a standard STM32 application development flow.
This contribution includes:

a python/tvm/contrib/stm32 directory with scripts necessary for compiling and communication with the STM32 board
a src/runtime/stm32 directory that includes runtime support used by the STM32 applications
an apps/stm32 README and examples running in the STM32 DISCOVERY H747 board.
Tested on development board with several models including mnist, inceprion, squeezenet, mobilenet, yolo, and a number of proprietary models.
This contribution does not modify any of the original TVM files.
@areusch
@tqchen

tqchen · 2021-03-25T16:27:57Z

Thanks @stoa , given this is a new feature. It would be great if you can post an RFC thread

areusch · 2021-03-25T17:29:05Z

hi @stoa, thanks for your contribution! I agree with @tqchen an RFC would be great.

looking over the code I see a lot of concepts implemented that are fairly similar to what we have implemented in microTVM now:

a remote execution driver
a project generation script
a minimal C runtime, including graph runtime and c_backend_api.h implementation

For microTVM, we're taking the approach of reusing C runtime components between platforms and/or SoCs, and allowing platform-specific e.g. memory allocators by implementing include/tvm/runtime/crt/platform.h. We are also adopting a standard "interface layer" between TVM and generated firmware code (composed of "Model Library Format," "Project API," and the "TVM RPC server"), which provides a great way to reduce boilerplate code when targeting µC.

It would be great if we could consider a way to unify your approach with our microTVM approach. It seems like the STM32 firmware code could become an implementation of our Project API (similar to the host demo project). This would allow us to bring further improvements to TVM currently under development (e.g. AOT runtime, graph-level optimization, tvmc integration), as well as enable AutoTVM on your platform.

Here are some pointers to RFCs that describe our approach in more detail:

Please also feel free to suggest improvements or changes to our current implementation. It is very much in development and we would like to encourage contributions!

stoa · 2021-03-25T19:54:30Z

Hello @areusch and @tqchen .
Thanks for the prompt consideration. We are very interested in collaborating in the mTVM project and to contribute to the TVM model targeting micro/bare-metal devices.
In practical terms, I will post an RFC like you suggested, and then we can discuss on the basis of the mTVM RFC and the new one. Could you point me at how one starts a TVM RFC thread ?
Thanks and cheers,
Arthur

areusch · 2021-03-25T20:36:27Z

@stoa

Hi Arthur! That would be great. Feel free to open a new topic categorized under Development > RFC on our discuss forum. If you look in that category you can see some examples of how people have titled and structured their RFCs. However, the process is fairly informal at this point. This PR would be a great PoC to backlink from the RFC as well, so people can read more specifics if they like.

Looking forward to it!

Thanks,
Andrew

stoa · 2021-03-29T16:07:04Z

Hello, Andrew
I just have posted this: https://discuss.tvm.apache.org/t/rfc-standalone-code-generation-and-c-runtime-for-stm32-bare-metal-devices/9562
Please, feel free to correct if I have missed something from on-going projects. The RFC is a basis to see if/how we can contribute on the micro TVM topics. I have tried to describe our prospective.
Looking forward to the discussion.
Arthur

areusch · 2021-04-15T20:46:53Z

@stoa please ping me when you want me to take another look here

stoa · 2021-04-16T10:40:07Z

@areusch
How are you, Andrew
Can you help me out with this: we have some .h files for inclusion with the C code, they are checked via cpplint and the build fails with things like 'replace C-style casts ...'. How do you make these pass the lint checks ?
Thanks and cheers
Arthur

areusch · 2021-04-16T17:43:00Z

hi @stoa, try adding // LINT__C_FILE as a comment near the top of those files and that should fix that. sorry this isn't clear!

areusch · 2021-09-20T06:19:38Z

@stoa let me know when you're ready for another review

stoa · 2021-09-20T08:59:51Z

@areusch
How are you, Andrew
Please review the most recent commit.
cheers
Arthur

areusch

@stoa thanks for fixing the test. could you address my may 24 comments on emitter.py? there are a bunch of style things we should fix and simplifications to the code to make it read like the rest of the TVM repo and therefore easier to maintain. inside apps we can have some flexibility, but code in python should all follow similar convention. i've left a couple more comments as well.

areusch · 2021-09-21T20:59:54Z

include/tvm/runtime/c_runtime_api.h

@@ -190,6 +190,7 @@ TVM_DLL void TVMAPISetLastError(const char* msg);
 *  \return error info
 */
 TVM_DLL const char* TVMGetLastError(void);
+


ping on this one

areusch · 2021-09-21T21:01:07Z

python/tvm/contrib/stm32/emitter.py

+import tvm
+from tvm.contrib import utils
+
+AI_TOOLS_VERSION_MAJOR = 1


ping on this one

areusch · 2021-09-21T22:36:34Z

python/tvm/contrib/stm32/emitter.py

+    return node_name.replace(":", "_")
+
+
+# ==========================================================


are you guys ok with deleting the # ======= comments?

python/tvm/contrib/stm32/emitter.py

python/tvm/micro/contrib/stm32/emitter.py

src/runtime/crt/contrib/stm32/ai_runtime_api.c

src/runtime/crt/contrib/stm32/runtime.c

tests/crt/contrib/stm32/src/main.c

stoa · 2021-09-23T08:33:55Z

@areusch
Hello, Andrew
Applied changes that you've requested.
Sorry for the comments from May 25 - some went with the revert and some I have not seen because they were hidden with the github page.
The last CI fails strangely, probably has nothing to do with my changes, I will re-run but this should not block the review.
cheers
Arthur

areusch

@stoa thanks for addressing those last comments. i just have a few more for this one and i think we are good to merge after we've addressed these. thanks for your patience here!

tests/micro/stm32/test_code_emitter.py

python/tvm/micro/contrib/stm32/emitter.py

areusch · 2021-09-24T17:14:15Z

python/tvm/micro/contrib/stm32/emitter.py

+           The quantization information for model inputs/outputs.
+        """
+
+        for key in self.graph_:


from other file: i would suggest not extracting these from self.graph_ and instead just accessing them using self.graph_["nodes"]. it may be very slightly slower, but it makes the code easier to maintain as you don't need to keep track of where each of these properties comes from. if you want to assert there aren't unknown keys in the graph, you could assert(list(sorted(self.graph_)) == ["arg_nodes", "attrs", "heads", "metadata", "node_row_ptr", "nodes"])

I prefer to keep a separate data structure for the emitter. Generally, this data structure is not necessarily equal to the TVM graph_['nodes']. For example, I may decide to not include OPs that are a 'noop' in the code, etc. This is done on purpose to simplify development. No problem maintaining - the structure is built in a single place from the graph.

ok, i don't think we need to block on this but i don't see you doing any modification to self._nodes, self._arg_nodes, self._node_row_ptr, self._outputs, or self._attrs right now.

python/tvm/micro/contrib/stm32/emitter.py

stoa · 2021-09-27T07:35:02Z

@areusch
How are you, Andrew.
Done. Please check.
Arthur

areusch

@stoa sorry for the long delay. found a couple more unaddressed style things and suggested some small changes to make the tests more robust. otherwise, looks good now. thanks for working through these!

areusch · 2021-10-03T16:27:12Z

python/tvm/micro/contrib/stm32/emitter.py

+    return size * _get_type_size(dltype)
+
+
+def _preprocess_code(src):


just curious: in general i think our code emitter should properly include all the stdlibs required to compile the code. i think maybe originally this included "ai_platform_api.h" if my memory serves? just curious if you need this function still?

Yes. Because math.h is missing (perhaps no longer missing in the newwest TVM), and because we have a CMSIS option that I will contribute later that needs more.

areusch · 2021-10-03T16:30:23Z

python/tvm/micro/contrib/stm32/emitter.py

+           The quantization information for model inputs/outputs.
+        """
+
+        for key in self.graph_:


ok, i don't think we need to block on this but i don't see you doing any modification to self._nodes, self._arg_nodes, self._node_row_ptr, self._outputs, or self._attrs right now.

areusch · 2021-10-03T16:33:35Z

python/tvm/micro/contrib/stm32/emitter.py

+                src = fin.read()
+                src_files.append(src)
+
+        self.graph_ = graph_dict


think there are a couple more style fixes (e.g. self._graph and self._lib)

areusch · 2021-10-03T16:38:42Z

python/tvm/micro/contrib/stm32/emitter.py

+        model_h_name = os.path.join(dest_dir, model_name + ".h")
+        model_c_name = os.path.join(dest_dir, model_name + ".c")
+
+        data_h = open(data_h_name, "w")


could you do this like so:

import contextlib with contextlib.ExitStack() as exit_stack: data_h = exit_stack.enter_context(open(data_h_name, "w")) data_c = exit_stack.enter_context(open(data_c_name, "w")) out_h = exit_stack.enter_context(open(out_h_name, "w")) out_c = exit_stack.enter_context(open(out_h_name, "w")) self._emit_params_data(...) self.emit_open(...) # .. all other self._emit # no need to close() the ExitStack does this for you

this way, if an exception occurs in any self._emit_*, Python will auto-close the files for you.

areusch · 2021-10-03T16:50:03Z

tests/micro/stm32/test_code_emitter.py

+def test_mnist():
+    curr_path = os.path.dirname(os.path.abspath(os.path.expanduser(__file__)))
+    model_path = os.path.join(curr_path, "models/mnist.tflite")
+    build_dir = os.path.join(curr_path, BUILD_DIR)


as we add more tests, we'll need to config filesystem modifications in a test-specific directory. can you do this:

DEBUG = False tempdir_root = None if DEBUG: tempdir_root = os.path.join(curr_path, f"workspace", "test_mnist", datetime.datetime.now().strftime("%Y-%m-%dT%H-%M-%S")) build_dir = tvm.contrib.utils.tempdir(tempdir_root)

areusch

thanks @stoa for working through these issues! i think we can merge now!

* Contribute apps/stm32 application. * Removed a useless file. * STM32: Use Model Library Format in demo. Added test. * STM32: Added quantized mnist test. * STM32: Fixed apps/stm32/Makefile. * STM32: Added quantized models test. * STM32: Added tests to tests/scripts/task_python_microtvm.sh * STM32: Listed specific files with lint. * STM32: removed external copyright notices. * STM32: Added missing ASF copyright notice. * STM32: style fixes. * STM32: more style fixes. * STM32: fixed liny for C files. * STM32: Does extern C help with cpplint. * STM32: Fixed wrong LINT_C_FILE spelling. * STM32: Still some lint fixes. * STM32: more style. * STM32: More fixes lint+formatting. * STM32: cleanup. * STM32: style cleanup. * STM32: Moved ai_runner to the apps/stm32. * Alignment with PR 7742 * lint cleanup. * STM32: Use crt_backend_api.c with standalone build. * STM32: Fixed the CI test. * STM32: style fixes. * STM32: Removed unused files. * STM32: Moved to crt_backend_api * STM32: style fix. * STM32: style fix. * Revert "STM32: Removed unused files." This reverts commit d72f8e5. Undo changes to c_backend_api/c_runtime_api. * Revert "STM32: Moved to crt_backend_api" This reverts commit 6c0e666. Undo changes to the c_backend-api/c_runtime_api. * stm32: aligned to micro TVM structure. * stm32: improved the python style. * stm32: cpplint fixes. * stm32: Fixed the test * stm32: style fixes. * stm32: style fixes. * stm32: style fixes. * stm32: style fixes.

stoa added 2 commits March 24, 2021 14:32

Contribute apps/stm32 application.

b78ea16

Removed a useless file.

b6b67c3

tqchen added the status: need RFC need RFC discussion label Mar 25, 2021

tqchen assigned areusch Mar 25, 2021

areusch added status: need update need update based on feedbacks and removed status: need RFC need RFC discussion labels Apr 2, 2021

stoa added 6 commits April 12, 2021 18:09

STM32: Use Model Library Format in demo. Added test.

a5e11eb

STM32: Added quantized mnist test.

72e0774

STM32: Fixed apps/stm32/Makefile.

d13cf67

STM32: Added quantized models test.

5069d46

STM32: Added tests to tests/scripts/task_python_microtvm.sh

0762080

Merge branch 'main' into stm32-apps-h747

efd8203

stoa added 6 commits April 16, 2021 00:15

STM32: Listed specific files with lint.

a1e57fd

pull request merge.

4ac1f6c

STM32: removed external copyright notices.

5839c32

STM32: Added missing ASF copyright notice.

25068c3

STM32: style fixes.

9862d52

STM32: more style fixes.

9dca8e4

stoa added 4 commits April 16, 2021 20:45

STM32: fixed liny for C files.

247461f

STM32: Does extern C help with cpplint.

6b197d0

STM32: Fixed wrong LINT_C_FILE spelling.

f5fbfcb

STM32: Still some lint fixes.

c0728fd

stoa requested review from liangfu, manupak, masahi, merrymercy, tmoreau89, tqchen, vinx13, yzhliu, zhiics and ZihengJiang as code owners September 14, 2021 10:46

stoa added 4 commits September 14, 2021 15:04

stm32: improved the python style.

e113bbe

Merge branch 'main' into stm32-apps-h747

3d607ad

stm32: cpplint fixes.

0f77005

stm32: Fixed the test

b4e4fde

stoa requested a review from areusch September 20, 2021 08:40

areusch reviewed Sep 21, 2021

View reviewed changes

stm32: style fixes.

b8354a5

areusch reviewed Sep 24, 2021

View reviewed changes

stm32: style fixes.

bc4ab04

areusch reviewed Oct 3, 2021

View reviewed changes

stoa added 2 commits October 4, 2021 14:12

stm32: style fixes.

5b9ec61

stm32: style fixes.

457dd98

areusch approved these changes Oct 4, 2021

View reviewed changes

areusch merged commit b48c9ef into apache:main Oct 6, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Contributing the STM32 port #7742

Contributing the STM32 port #7742

stoa commented Mar 25, 2021

tqchen commented Mar 25, 2021

areusch commented Mar 25, 2021

stoa commented Mar 25, 2021

areusch commented Mar 25, 2021

stoa commented Mar 29, 2021

areusch commented Apr 15, 2021

stoa commented Apr 16, 2021

areusch commented Apr 16, 2021

areusch commented Sep 20, 2021

stoa commented Sep 20, 2021

areusch left a comment

areusch Sep 21, 2021

areusch Sep 21, 2021

areusch Sep 21, 2021

stoa commented Sep 23, 2021

areusch left a comment

areusch Sep 24, 2021

stoa Sep 26, 2021

areusch Oct 3, 2021

stoa commented Sep 27, 2021

areusch left a comment

areusch Oct 3, 2021

stoa Oct 3, 2021

areusch Oct 3, 2021

areusch Oct 3, 2021

areusch Oct 3, 2021

areusch Oct 3, 2021

areusch left a comment

		return node_name.replace(":", "_")


		# ==========================================================

		return size * _get_type_size(dltype)


		def _preprocess_code(src):

Contributing the STM32 port #7742

Contributing the STM32 port #7742

Conversation

stoa commented Mar 25, 2021

tqchen commented Mar 25, 2021

areusch commented Mar 25, 2021

stoa commented Mar 25, 2021

areusch commented Mar 25, 2021

stoa commented Mar 29, 2021

areusch commented Apr 15, 2021

stoa commented Apr 16, 2021

areusch commented Apr 16, 2021

areusch commented Sep 20, 2021

stoa commented Sep 20, 2021

areusch left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

stoa commented Sep 23, 2021

areusch left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

stoa commented Sep 27, 2021

areusch left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

areusch left a comment

Choose a reason for hiding this comment