[µTVM] Add --runtime=c, remove micro_dev target, enable LLVM backend #6145

areusch · 2020-07-27T16:25:48Z

This PR adds a --runtime attr to the c and llvm targets. --runtime is expected to be c or c++ currently, and this option specifies the way in which generated modules handle function lookup. in the c++ case, global initializer functions may be called (system lib) or the symbol table may be consulted by library_module.cc. in the c case, a FuncRegistry is generated and an explicit initialization hook TVMSystemLibEntryPoint is defined.

currently, this PR only supports generating system libs for the C runtime, but the functionality is split into two pieces so that it's clear what would need to be addressed to support a form of dynamically-linked C libs. much of this PR is written assuming a bare metal environment (explicit initializer, function lookup table in flash, no presumed use of dlopen, etc), so a "dynamically-linked" solution would ideally support that environment as well.

this PR is in service of MISRA-C changes for RPC support RFC; if you think it deserves its own RFC, I'm happy to write one up.

tqchen · 2020-07-28T21:59:30Z

cc @liangfu @tom-gall @junrushao1994

liangfu · 2020-07-29T02:41:59Z

apps/bundle_deploy/bundle_static.c

+  if (nptrs < 0) {
+    perror("backtracing");
+  } else {
+    backtrace_symbols_fd(trace, nptrs, STDOUT_FILENO);


This is a great feature to have backtrace enabled, however, since these functions are not part of the standard c programming language, and it would have potential portability issue, can we add a macro to disable this feature for the platforms where backtracing is not supported?

i'm actually just trying to use this to debug the regression, I couldn't reproduce the gpu failure on my side. but, now i've found a node where i should be able to run tvmai/ci-gpu, so i'm going to take that approach.

for merging, I don't know--I know we had issues with the CRT being flaky in the regression before. it might be worth it to leave this code in, since flaky test failures are almost always solved with more data. so, i'd be happy to put it behind a #define and enable that for the regression, if you're also happy with that?

Yes, this is exactly what I wanted from the previous comment, and I think leaving backtracing flag on should be fine.

okay, added a Makefile flag to control this.

liangfu · 2020-07-29T02:51:25Z

tests/python/unittest/test_target_codegen_llvm.py

-    test_dwarf_debug_information()
-    test_llvm_shuffle()
-    test_llvm_bf16()
+    # test_multiple_func()


did you disable these unintentionally?

whoops, after the flaky CI CUDA issue is resolved i'll push a patch with this reverted

uncommented

areusch · 2020-07-29T22:11:23Z

I think this is now ready for review (CI failure is due to an unrelated flake, I believe).

@u99127 @weberlo @tmoreau89

liangfu · 2020-08-02T09:26:35Z

The CI error seems unrelated, please rebase to trigger it again.

liangfu · 2020-08-02T09:29:22Z

apps/bundle_deploy/Makefile

 	@mkdir -p $(@D)
-	gcc $(PKG_CFLAGS) -o $@ $^
+	gcc $(PKG_CFLAGS) -o $@ $^ -ldl


I think linking to dl library within a static target isn't the desired intention. @mehrdadh what's your take on this?

it's needed for the backtrace logic, so wherever that was used.

liangfu

Left a minor comment, LGTM overall.

liangfu · 2020-08-06T16:07:11Z

apps/bundle_deploy/bundle_static.c

@@ -22,7 +22,11 @@
 #include <tvm/runtime/crt/crt.h>
 #include <tvm/runtime/crt/graph_runtime.h>
 #include <tvm/runtime/crt/packed_func.h>
+#include <unistd.h>


why do we need this header file here?

Ah I think I missed this one, it’s probably not needed. But, I won’t be able to address this for a week, so perhaps we can merge as is? Happy to take an issue to fix in a followup PR if that’s ok with you.

I suggest this gets trimmed in a follow up PR

tom-gall · 2020-08-06T20:11:50Z

As part of this PR there is a small change to tvm/tests/micro/test_runtime_micro_on_arm.py, attempting to run the tests fail.

tom-gall · 2020-08-06T21:11:32Z

Since there isn't an RFC, I'll drop this comment here, what does a somewhat working example look like?

Updating the micro tvm tutorial as inspiration :

target = "c --system-lib --runtime=c"
input_tensor = "dense_4_input"
input_shape = (1,)
input_dtype = "float32"

dev_config = micro.device.arm.stm32f746xx.generate_config("127.0.0.1", 6666)

mod, params = relay.frontend.from_tflite(tflite_model,
shape_dict={input_tensor: input_shape},
dtype_dict={input_tensor: input_dtype})

with micro.Session(dev_config) as sess:
ctx = tvm.micro_dev(0)

with tvm.transform.PassContext(disabled_pass={'FuseOps'}, config={"tir.disable_vectorize": True}):
    graph, c_mod, params = relay.build(mod, target=target, params=params)

micro_mod = micro.create_micro_mod(c_mod, dev_config)
mod = graph_runtime.create(graph, micro_mod, ctx)

mod.set_input(**params)
mod.set_input(input_tensor, tvm.nd.array(np.array([0.5], dtype="float32")))

mod.run()

# Get output
tvm_output = mod.get_output(0).asnumpy()

print("result is: "+str(tvm_output))

This feels like it should be close however it fails at micro.create_micro_mod

/tmp/tmpok80dx2r/temp.c:232:5: warning: initialization of 'int (*)(TVMValue *, int *, int, TVMValue *, int *, void )' {aka 'int ()(union *, int *, int, union *, int *, void )'} from incompatible pointer type 'int32_t ()(void *, void *, int32_t, void *, void *, void )' {aka 'long int ()(void *, void *, long int, void *, void *, void )'} [-Wincompatible-pointer-types]
232 | fused_nn_dense_nn_bias_add_nn_relu,
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/tmp/tmpok80dx2r/temp.c:232:5: note: (near initialization for '_tvm_func_array[0]')
/tmp/tmpok80dx2r/temp.c:233:5: warning: initialization of 'int ()(TVMValue *, int *, int, TVMValue *, int *, void )' {aka 'int ()(union *, int *, int, union *, int *, void )'} from incompatible pointer type 'int32_t ()(void *, void *, int32_t, void *, void *, void )' {aka 'long int ()(void *, void *, long int, void *, void *, void )'} [-Wincompatible-pointer-types]
233 | fused_reshape_1,
| ^~~~~~~~~~~~~~~
/tmp/tmpok80dx2r/temp.c:233:5: note: (near initialization for '_tvm_func_array[1]')
/tmp/tmpok80dx2r/temp.c:234:5: warning: initialization of 'int ()(TVMValue *, int *, int, TVMValue *, int *, void )' {aka 'int ()(union *, int *, int, union *, int *, void )'} from incompatible pointer type 'int32_t ()(void *, void *, int32_t, void *, void *, void )' {aka 'long int ()(void *, void *, long int, void *, void *, void )'} [-Wincompatible-pointer-types]
234 | fused_reshape,
| ^~~~~~~~~~~~~
/tmp/tmpok80dx2r/temp.c:234:5: note: (near initialization for '_tvm_func_array[2]')
/tmp/tmpok80dx2r/temp.c:235:5: warning: initialization of 'int ()(TVMValue *, int *, int, TVMValue *, int *, void )' {aka 'int ()(union *, int *, int, union *, int *, void )'} from incompatible pointer type 'int32_t ()(void *, void *, int32_t, void *, void *, void )' {aka 'long int ()(void *, void *, long int, void *, void *, void )'} [-Wincompatible-pointer-types]
235 | fused_nn_dense_nn_bias_add,
| ^~~~~~~~~~~~~~~~~~~~~~~~~~
/tmp/tmpok80dx2r/temp.c:235:5: note: (near initialization for '_tvm_func_array[3]')
/tmp/tmpok80dx2r/temp.c:236:5: warning: initialization of 'int ()(TVMValue *, int *, int, TVMValue *, int *, void )' {aka 'int ()(union *, int *, int, union *, int *, void )'} from incompatible pointer type 'int32_t ()(void *, void *, int32_t, void *, void *, void )' {aka 'long int ()(void *, void *, long int, void *, void *, void *)'} [-Wincompatible-pointer-types]
236 | fused_nn_dense_nn_bias_add_nn_relu_1,
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/tmp/tmpok80dx2r/temp.c:236:5: note: (near initialization for '_tvm_func_array[4]')
/tmp/tmpok80dx2r/temp.c:237:1: error: expected expression before 'static'
237 | static const TVMFuncRegistry _tvm_func_registry = {

Should that be expected for the time being?

tmoreau89 · 2020-08-06T23:10:03Z

Thanks @areusch , @liangfu , @tom-gall for the comments. I made the decision to merge the PR in order to avoid bit-rot.

tmoreau89 · 2020-08-06T23:10:46Z

@weberlo can help address your question @tom-gall

weberlo · 2020-08-07T00:29:42Z

@tom-gall I inspected the generated C from the example code you posted, and I'm seeing this at the end of the file:

                                                                                                    
static TVMBackendPackedCFunc _tvm_func_array[] = {                                                  
    fused_reshape,                                                                                  
    fused_reshape_1,                                                                                
    fused_nn_dense_nn_bias_add,                                                                     
    fused_nn_dense_nn_bias_add_nn_relu,                                                             
    fused_nn_dense_nn_bias_add_nn_relu_1,                                                           
static const TVMFuncRegistry _tvm_func_registry = {                                                 
    "\484849fused_reshape\484848fused_reshape_1\484848fused_nn_dense_nn_bias_add\484848fused_nn_dense_nn_bias_..."
};                                                                                                  
static const TVMModule _tvm_system_lib = {                                                          
    &system_lib_registry,                                                                           
};                                                                                                  
const TVMModule* TVMSystemLibEntryPoint(void) {                                                     
    return &system_lib;                                                                             
}

So my guess is there's a closing brace not being generated here.

@areusch Maybe we should add a test that compiles the module source?

…pache#6145) * need to fill address of globals in tvmfuncregistry * llvm func registry generator works! * lint fixes * rm hexdump include * bring bundle_deploy back to life and add to CI * revert gcda additions * git-clang-format * fix check for --system-lib and test_runtime_micro target * fixup compile flags for bundle_deploy CRT and improve robustness * git-clang-format * add debugging info * git-clang-format * initialize ret_values in PackedFunc_Call. * retrigger CI * fix log messages * git-clang-format * remove default for --runtime target opt * put backtrace behind a flag and enable it * simpify ReadString(), fixing bad instruction exception on os x. * git-clang-format * uncomment tests * reorder backtrace ldflags for linux gcc

areusch added 5 commits July 27, 2020 09:06

need to fill address of globals in tvmfuncregistry

2502874

llvm func registry generator works!

88e0fe3

lint fixes

bc8c3a3

rm hexdump include

73e0fdc

bring bundle_deploy back to life and add to CI

15c2a2f

areusch force-pushed the utvm-target-id branch from d51a5bf to 15c2a2f Compare July 27, 2020 23:01

areusch added 7 commits July 27, 2020 16:29

revert gcda additions

6971d41

git-clang-format

c678370

fix check for --system-lib and test_runtime_micro target

8d9c6c7

fixup compile flags for bundle_deploy CRT and improve robustness

7f52395

git-clang-format

9b9d1d8

add debugging info

ef96809

git-clang-format

c3c51a6

tqchen added the status: need review label Jul 28, 2020

liangfu mentioned this pull request Jul 29, 2020

[CRT][FLAKY] Flaky failures on 32bit platforms #5298

Closed

liangfu reviewed Jul 29, 2020

View reviewed changes

initialize ret_values in PackedFunc_Call.

d91b2c2

areusch mentioned this pull request Jul 30, 2020

Bundle_deploy app doesn't build demo_dynamic #6175

Closed

areusch added 4 commits July 30, 2020 11:56

retrigger CI

3727f58

fix log messages

e429bd7

git-clang-format

6c2cc72

Merge remote-tracking branch 'tvm/master' into utvm-target-id

c200ede

liangfu reviewed Aug 2, 2020

View reviewed changes

liangfu self-assigned this Aug 2, 2020

areusch added 3 commits August 3, 2020 09:41

remove default for --runtime target opt

1551001

put backtrace behind a flag and enable it

9550913

simpify ReadString(), fixing bad instruction exception on os x.

b431915

areusch added 4 commits August 5, 2020 15:49

git-clang-format

fcfdbbb

uncomment tests

f9ca334

Merge remote-tracking branch 'tvm/master' into HEAD

9150bce

reorder backtrace ldflags for linux gcc

0e5a8e8

liangfu approved these changes Aug 6, 2020

View reviewed changes

tmoreau89 approved these changes Aug 6, 2020

View reviewed changes

tmoreau89 merged commit b485d47 into apache:master Aug 6, 2020

windclarion mentioned this pull request Aug 7, 2020

[uTVM] fix crt building and running error #6231

Merged

ZihengJiang mentioned this pull request Sep 25, 2020

TVM v0.7 Release Note Candidate #6486

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[µTVM] Add --runtime=c, remove micro_dev target, enable LLVM backend #6145

[µTVM] Add --runtime=c, remove micro_dev target, enable LLVM backend #6145

areusch commented Jul 27, 2020

tqchen commented Jul 28, 2020 •

edited

Loading

liangfu Jul 29, 2020

areusch Jul 29, 2020

liangfu Aug 2, 2020

areusch Aug 5, 2020

liangfu Jul 29, 2020

areusch Jul 29, 2020 •

edited

Loading

areusch Aug 5, 2020

areusch commented Jul 29, 2020

liangfu commented Aug 2, 2020

liangfu Aug 2, 2020

areusch Aug 5, 2020

liangfu left a comment

liangfu Aug 6, 2020

areusch Aug 6, 2020

tmoreau89 Aug 6, 2020

tom-gall commented Aug 6, 2020 •

edited

Loading

tom-gall commented Aug 6, 2020 •

edited

Loading

tmoreau89 commented Aug 6, 2020

tmoreau89 commented Aug 6, 2020

weberlo commented Aug 7, 2020

[µTVM] Add --runtime=c, remove micro_dev target, enable LLVM backend #6145

[µTVM] Add --runtime=c, remove micro_dev target, enable LLVM backend #6145

Conversation

areusch commented Jul 27, 2020

tqchen commented Jul 28, 2020 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

areusch Jul 29, 2020 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

areusch commented Jul 29, 2020

liangfu commented Aug 2, 2020

Choose a reason for hiding this comment

Choose a reason for hiding this comment

liangfu left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

tom-gall commented Aug 6, 2020 • edited Loading

tom-gall commented Aug 6, 2020 • edited Loading

tmoreau89 commented Aug 6, 2020

tmoreau89 commented Aug 6, 2020

weberlo commented Aug 7, 2020

tqchen commented Jul 28, 2020 •

edited

Loading

areusch Jul 29, 2020 •

edited

Loading

tom-gall commented Aug 6, 2020 •

edited

Loading

tom-gall commented Aug 6, 2020 •

edited

Loading