Rename runtime-config to executor-config and add documentation for Model Library Format #8270

areusch · 2021-06-17T00:32:16Z

This PR:

renames runtime-config to executor-config to match Rename GraphRuntime to GraphExecutor #7653.
adds documentation for Model Library Format to the tree. It is preparation to introduce a Project API RFC (for which it's required reading).

@mehrdadh @leandron @gromero @manupa-arm @jwfromm @guberti @Mousius @giuseros

leandron

It looks very clear documentation. I think users would benefit also from a short code snippet giving an example of how to generate a MLF package.

Also a few nits and suggestions.

leandron · 2021-06-17T10:23:05Z

docs/dev/model_library_format.rst

+--------------------------
+
+TVM traditionally exports generated libraries as Dynamic Shared Objects
+(e.g. DLLs (Windows) or .so (linux)). Inference can be performed on those libraries by loading them


Suggested change

(e.g. DLLs (Windows) or .so (linux)). Inference can be performed on those libraries by loading them

(e.g. DLLs (Windows) or .so (linux)). Inferences can be performed on those libraries by loading them

leandron · 2021-06-17T10:23:59Z

docs/dev/model_library_format.rst

+into an executable using ``libtvm_runtime.so``. This process is very dependent on services provided
+by traditional OS.
+
+For deployment to unconventional platforms (e.g. those lacking traditional OS), the microTVM project


Maybe use embedded instead?

Suggested change

For deployment to unconventional platforms (e.g. those lacking traditional OS), the microTVM project

For deployment to embedded platforms (e.g. those lacking traditional OS), the microTVM project

i feel like it's not strictly limited to embedded though

I agree.

to that end, I think we should word this as a general output format that produces what strictly TVM (code)generates. The creation of .so/.dll requires external toolchains called after tvm compilation -- such as LLVM and C compilers.
WDYT ?

leandron · 2021-06-17T10:26:36Z

docs/dev/model_library_format.rst

+Model Library Format
+====================


I think it would be beneficial to introduce MLF as an acronym for Model Lbrary Format early in the document, so that we can get people used to it?

Suggested change

Model Library Format

====================

Model Library Format (MLF)

==========================

i haven't used the acronym yet in this doc though. but i agree it's an easy shorthand for the format. maybe it would make sense more in tvmc docs, where it's a command-line param? wdyt?

Mousius · 2021-06-17T10:37:27Z

Would it be worth adding a JSON Schema for metadata.json at this point to serve as both documentation and a validation mechanism?

mehrdadh

@areusch thanks for this PR!
Mostly good, I have just a few suggestions.

mehrdadh · 2021-06-17T15:41:00Z

docs/dev/model_library_format.rst

+ * ``<target>`` - Identifies the TVM target on which the code should run. Currently, only ``host``
+   is supported.
+ * ``<unique_name>`` - A unique slug identifying this file. Currently ``lib<n>``, with ``<n>>` an
+   autoincrementing integer.


maybe change to auto-incrementing?

mehrdadh · 2021-06-17T15:42:22Z

docs/dev/model_library_format.rst

+``parameters``
+^^^^^^^^^^^^^^
+
+Contains machine-parseable parameters. A variety of formats may be provided, but at present, only


manupak

Looks good, just one more nit -- (bar other's comments)

manupak · 2021-06-17T16:04:38Z

docs/dev/model_library_format.rst

+  `Memory Usage Summary`_.
+- ``model_name``: The name of this model (e.g. the ``name`` parameter supplied to
+  ``tvm.relay.build``).
+- ``runtimes``: A list of runtimes supported by this model. Currently, this list is always


maybe executors ?

good catch :)

areusch · 2021-07-09T00:29:22Z

@leandron @manupa-arm please take another look!

manupak

Did some (final) nit picking! hope you dont mind.
Mostly text changes.

manupak · 2021-07-12T17:04:48Z

docs/dev/model_library_format.rst

+into an executable using ``libtvm_runtime.so``. This process is very dependent on services provided
+by traditional OS.
+
+For deployment to unconventional platforms (e.g. those lacking traditional OS), the microTVM project


I agree.

to that end, I think we should word this as a general output format that produces what strictly TVM (code)generates. The creation of .so/.dll requires external toolchains called after tvm compilation -- such as LLVM and C compilers.
WDYT ?

manupak · 2021-07-12T17:06:00Z

docs/dev/model_library_format.rst

+
+For deployment to unconventional platforms (e.g. those lacking traditional OS), the microTVM project
+can be used to export a generated library in pieces. In this case, microTVM provides another output
+format, Model Library Format. Model Library Format is a tarball containing a file for each part of


I think we should not use "microTVM provides another output format", though we could use microTVM as an example this output format becomes useful.

manupak · 2021-07-12T17:07:38Z

docs/dev/model_library_format.rst

+format, Model Library Format. Model Library Format is a tarball containing a file for each part of
+the TVM compiler output.
+
+What can be Exported


Suggested change

What can be Exported

What can be exported ?

just nit picking :)

manupak · 2021-07-12T17:08:57Z

docs/dev/model_library_format.rst

+Directory Layout
+----------------
+
+Model Library Format is traditionally contained within a tarball. All paths are relative to the root


Suggested change

Model Library Format is traditionally contained within a tarball. All paths are relative to the root

Model Library Format is contained within a tarball. All paths are relative to the root

nit: we dont have any other way. Do we?

good point :)

manupak · 2021-07-12T17:12:08Z

docs/dev/model_library_format.rst

+  sub-target which describes that relay backend used for that ``device_type``.
+- ``version``: A numeric version number that identifies the format used in this Model Library
+  Format. This number is incremented when the metadata structure or on-disk structure changes.
+  This document reflects version ``3``.


Suggested change

This document reflects version ``3``.

This document reflects version ``5``.

areusch · 2021-07-15T19:34:48Z

@manupa-arm it won't let me continue your earlier thread, but basically i'd agree with that idea in the medium term however we cannot yet export BYOC Module in MLF. once that is possible, i think we should promote it to a general TVM output format.

areusch · 2021-07-15T20:05:02Z

@manupa-arm please take another look

manupak

LGTM

leandron

LGTM

leandron · 2021-07-20T19:14:47Z

Thanks @areusch @mehrdadh @Mousius @manupa-arm, this is merged now!

…del Library Format (apache#8270) * Rename runtime-config to executor-config. * Add documentation. * address comments, make tests pass * fix unit test * fix sphinx doc errors * address manupa comments

areusch added 2 commits June 10, 2021 13:42

Rename runtime-config to executor-config.

37b67de

Add documentation.

fe65a82

areusch changed the title ~~Add documentation for Model Library Format~~ Rename runtime-config to executor-config and add documentation for Model Library Format Jun 17, 2021

leandron reviewed Jun 17, 2021

View reviewed changes

mehrdadh reviewed Jun 17, 2021

View reviewed changes

manupak reviewed Jun 17, 2021

View reviewed changes

address comments, make tests pass

861e510

mehrdadh approved these changes Jun 30, 2021

View reviewed changes

areusch mentioned this pull request Jun 30, 2021

Add Project API RFC apache/tvm-rfcs#8

Merged

Merge remote-tracking branch 'origin/main' into model-library-format-doc

c96b9e9

areusch added 2 commits July 9, 2021 09:01

fix unit test

d529ab9

fix sphinx doc errors

a0211d9

manupak reviewed Jul 12, 2021

View reviewed changes

address manupa comments

5269289

manupak approved these changes Jul 16, 2021

View reviewed changes

leandron approved these changes Jul 20, 2021

View reviewed changes

leandron merged commit ca28dff into apache:main Jul 20, 2021

junrushao mentioned this pull request Nov 1, 2021

Apache TVM v0.8 Release Note Candidate #9416

Closed

	(e.g. DLLs (Windows) or .so (linux)). Inference can be performed on those libraries by loading them
	(e.g. DLLs (Windows) or .so (linux)). Inferences can be performed on those libraries by loading them

	For deployment to unconventional platforms (e.g. those lacking traditional OS), the microTVM project
	For deployment to embedded platforms (e.g. those lacking traditional OS), the microTVM project

	Model Library Format is traditionally contained within a tarball. All paths are relative to the root
	Model Library Format is contained within a tarball. All paths are relative to the root

	This document reflects version ``3``.
	This document reflects version ``5``.

Rename runtime-config to executor-config and add documentation for Model Library Format #8270

Rename runtime-config to executor-config and add documentation for Model Library Format #8270

Conversation

areusch commented Jun 17, 2021 • edited Loading

leandron left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Mousius commented Jun 17, 2021

mehrdadh left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

manupak left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

areusch commented Jul 9, 2021

manupak left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

areusch commented Jul 15, 2021

areusch commented Jul 15, 2021

manupak left a comment

Choose a reason for hiding this comment

leandron left a comment

Choose a reason for hiding this comment

leandron commented Jul 20, 2021

areusch commented Jun 17, 2021 •

edited

Loading