Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[microTVM] Enable micro tvmc tutorial testing in CI #10555

Merged
merged 5 commits into from
Mar 16, 2022
Merged

[microTVM] Enable micro tvmc tutorial testing in CI #10555

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
0090877
Add script convertor
mehrdadh Mar 9, 2022
85dd77a
Address comments: added test
mehrdadh Mar 15, 2022
c83fe69
address comments
mehrdadh Mar 15, 2022
b986cd3
Merge branch 'main' into add_tvmc_to_ci
mehrdadh Mar 15, 2022
8325c59
Merge branch 'main' into add_tvmc_to_ci
mehrdadh Mar 16, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
75 changes: 75 additions & 0 deletions docs/script_convert.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,75 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.

import argparse
import pathlib

BASH = "# bash"
BASH_IGNORE = "# bash-ignore"
BASH_MULTILINE_COMMENT_START = ": '"
BASH_MULTILINE_COMMENT_END = "'"


def convert(args: argparse.Namespace):
src_path = pathlib.Path(args.script)
des_path = src_path.parent / f"{src_path.stem}.py"
with open(src_path, "r") as src_f:
with open(des_path, "w") as des_f:
line = src_f.readline()
bash_block = []
bash_detected = False
bash_ignore_detected = False
while line:
line = line.strip("\n").strip("\r")
if bash_detected:
if line == BASH:
# write the bash block to destination
python_code = "# .. code-block:: bash\n#\n"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should we run the bash script from this file and include the output? we could probably do that by injecting output markers into the script e.g.

echo -e "\nSTART OF COMMAND 1 OUTPUT --->"
tvmc ...
echo -e "\n<--- START OF COMMAND 1 OUTPUT"

then we could take the bash-ignore commented areas, extract them into a common prelude, and also include stuff like set -eux so that people write proper bash scripts. we could maybe do this by putting that into a docs/bash_gallery_prelude.sh and erroring in this script if we don't see a line like:

source docs/bash_gallery_prelude.sh || exit 2

might need to think about the path a little bit

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is the idea to add the output of each command to generated python file and finally have it as part of generated HTML file?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah exactly

for bash_line in bash_block:
python_code += f"#\t {bash_line}\n"
python_code += "#\n"
des_f.write(python_code)

bash_detected = False
bash_block = []
else:
# add new bash command line
bash_block.append(line)
elif bash_ignore_detected:
if line == BASH_IGNORE:
bash_ignore_detected = False
else:
pass
else:
if line == BASH:
bash_detected = True
elif line == BASH_IGNORE:
bash_ignore_detected = True
elif line in [BASH_MULTILINE_COMMENT_START, BASH_MULTILINE_COMMENT_END]:
des_f.write('"""\n')
else:
des_f.write(f"{line}\n")

line = src_f.readline()


if __name__ == "__main__":
parser = argparse.ArgumentParser(description="Convert tutorial script to Python.")
parser.add_argument("script", type=str, help="Path to script file.")

args = parser.parse_args()
convert(args)
110 changes: 54 additions & 56 deletions ...y/how_to/work_with_microtvm/micro_tvmc.py → ...y/how_to/work_with_microtvm/micro_tvmc.sh
100644 → 100755
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@
# specific language governing permissions and limitations
# under the License.

"""
: '
.. _tutorial-micro-tvmc:

Executing a Tiny Model with TVMC Micro
Expand All @@ -25,7 +25,7 @@
This tutorial explains how to compile a tiny model for a micro device,
build a program on Zephyr platform to execute this model, flash the program
and run the model all using `tvmc micro` command.
"""
'

######################################################################
# .. note::
Expand All @@ -42,6 +42,10 @@
# ./docker/bash.sh tlcpack/ci-qemu
#

# bash-ignore
shopt -s expand_aliases
alias tvmc="python3 -m tvm.driver.tvmc"
# bash-ignore

############################################################
# Using TVMC Micro
Expand All @@ -55,16 +59,15 @@
#
# To check if you have TVMC command installed on your machine, you can run:
#
# .. code-block:: bash
#
# tvmc --help
#
# bash
tvmc --help
# bash
# To compile a model for microtvm we use ``tvmc compile`` subcommand. The output of this command
# is used in next steps with ``tvmc micro`` subcommands. You can check the availability of TVMC Micro using:
#
# .. code-block:: bash
#
# tvmc micro --help
# bash
tvmc micro --help
# bash
#
# The main tasks that you can perform using ``tvmc micro`` are ``create``, ``build`` and ``flash``.
# To read about specific options under a givern subcommand, use
Expand All @@ -80,10 +83,9 @@
#
# For this tutorial we will be using the model in tflite format.
#
# .. code-block:: bash
#
# wget https://github.com/tensorflow/tflite-micro/raw/main/tensorflow/lite/micro/examples/magic_wand/magic_wand.tflite
#
# bash
wget https://github.com/tensorflow/tflite-micro/raw/main/tensorflow/lite/micro/examples/magic_wand/magic_wand.tflite
# bash

############################################################
# Compiling a TFLite model to a Model Library Format
Expand All @@ -95,19 +97,18 @@
#
# Here, we generate a MLF file for ``qemu_x86`` Zephyr board. To generate MLF output for the ``magic_wand`` tflite model:
#
# .. code-block:: bash
#
# tvmc compile magic_wand.tflite \
# --target='c -keys=cpu -link-params=0 -model=host' \
# --runtime=crt \
# --runtime-crt-system-lib 1 \
# --executor='graph' \
# --executor-graph-link-params 0 \
# --output model.tar \
# --output-format mlf \
# --pass-config tir.disable_vectorize=1 \
# --disabled-pass=AlterOpLayout
#
# bash
tvmc compile magic_wand.tflite \
--target='c -keys=cpu -link-params=0 -model=host' \
--runtime=crt \
--runtime-crt-system-lib 1 \
--executor='graph' \
--executor-graph-link-params 0 \
--output model.tar \
--output-format mlf \
--pass-config tir.disable_vectorize=1 \
--disabled-pass=AlterOpLayout
# bash
# This will generate a ``model.tar`` file which contains TVM compiler output files. To run this command for
# a different Zephyr device, you need to update ``target``. For instance, for ``nrf5340dk_nrf5340_cpuapp`` board
# the target is ``--target='c -keys=cpu -link-params=0 -model=nrf5340dk'``.
Expand All @@ -122,14 +123,13 @@
# for the project to ``create`` subcommand along with project options. Project options for each
# platform (Zephyr/Arduino) are defined in their Project API server file. To generate Zephyr project, run:
#
# .. code-block:: bash
#
# tvmc micro create \
# project \
# model.tar \
# zephyr \
# --project-option project_type=host_driven zephyr_board=qemu_x86
#
# bash
tvmc micro create \
project \
model.tar \
zephyr \
--project-option project_type=host_driven zephyr_board=qemu_x86
# bash
# This will generate a ``Host-Driven`` Zephyr project for ``qemu_x86`` Zephyr board. In Host-Driven template project,
# the Graph Executor will run on host and perform the model execution on Zephyr device by issuing commands to the
# device using an RPC mechanism. Read more about `Host-Driven Execution <https://tvm.apache.org/docs/arch/microtvm_design.html#host-driven-execution>`_.
Expand All @@ -148,26 +148,24 @@
# Next step is to build the Zephyr project which includes TVM generated code for running the tiny model, Zephyr
# template code to run a model in Host-Driven mode and TVM runtime source/header files. To build the project:
#
# .. code-block:: bash
#
# tvmc micro build \
# project \
# zephyr \
# --project-option zephyr_board=qemu_x86
#
# bash
tvmc micro build \
project \
zephyr \
--project-option zephyr_board=qemu_x86
# bash
# This will build the project in ``project`` directory and generates binary files under ``project/build``. To build
# Zephyr project for a different Zephyr board, change ``zephyr_board`` project option.
#
# Next, we flash the Zephyr binary file to Zephyr device. For ``qemu_x86`` Zephyr board this step does not
# actually perform any action since QEMU will be used, however you need this step for physical hardware.
#
# .. code-block:: bash
#
# tvmc micro flash \
# project \
# zephyr \
# --project-option zephyr_board=qemu_x86
#
# bash
tvmc micro flash \
project \
zephyr \
--project-option zephyr_board=qemu_x86
# bash

############################################################
# Run Tiny Model on Micro Target
Expand All @@ -179,14 +177,14 @@
# and pass ``--device micro`` to specify the device type. This command will open a communication channel, set input
# values using ``Graph Executor`` on host and run full model on the device. Then it gets output from the device.
#
# .. code-block:: bash
#
# tvmc run \
# --device micro \
# project \
# --project-option zephyr_board=qemu_x86 \
# --fill-mode ones
# --print-top 4
# bash
tvmc run \
--device micro \
project \
--project-option zephyr_board=qemu_x86 \
--fill-mode ones \
--print-top 4
# bash
# # Output:
# #
# # INFO:__main__:b'[100%] [QEMU] CPU: qemu32,+nx,+pae\n'
Expand Down
21 changes: 21 additions & 0 deletions tests/scripts/task_convert_scripts_to_python.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
#!/usr/bin/env bash
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.

SCRIPTS_DIR=$(dirname "${BASH_SOURCE[0]}")
TVM_DIR=$(cd "${SCRIPTS_DIR}" && git rev-parse --show-toplevel)
python3 "${TVM_DIR}/docs/script_convert.py" "${TVM_DIR}/gallery/how_to/work_with_microtvm/micro_tvmc.sh"
3 changes: 3 additions & 0 deletions tests/scripts/task_lint.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,9 @@ cleanup()
trap cleanup 0


echo "Convert scripts to Python..."
tests/scripts/task_convert_scripts_to_python.sh

echo "Checking file types..."
python3 tests/lint/check_file_type.py

Expand Down
3 changes: 3 additions & 0 deletions tests/scripts/task_python_docs.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -58,6 +58,9 @@ sphinx_precheck() {

function join_by { local IFS="$1"; shift; echo "$*"; }

# Convert bash tutorials to Python format
tests/scripts/task_convert_scripts_to_python.sh

# These warnings are produced during the docs build for various reasons and are
# known to not signficantly affect the output. Don't add anything new to this
# list without special consideration of its effects, and don't add anything with
Expand Down
3 changes: 2 additions & 1 deletion tests/scripts/task_python_microtvm.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -45,9 +45,10 @@ run_pytest ctypes python-microtvm-stm32 tests/micro/stm32
run_pytest ctypes python-microtvm-common-qemu_x86 tests/micro/common --board=qemu_x86
run_pytest ctypes python-microtvm-common-due tests/micro/common --test-build-only --board=due

# # Tutorials running with host CRT
# Tutorials
python3 gallery/how_to/work_with_microtvm/micro_tflite.py
python3 gallery/how_to/work_with_microtvm/micro_autotune.py
./gallery/how_to/work_with_microtvm/micro_tvmc.sh
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we add this to make docs instead?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This script requires zephyr.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ah hmm. that is a bit challenging then....maybe my other questions are tricky to implement given this limitation of doc-builder.

Copy link
Member Author

@mehrdadh mehrdadh Mar 14, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that would be a bigger change. GPU docker image which is used for doc generation should be able to run Zephyr and other platforms in future. Other possibility is to have each docker step to run and convert the script which is related to their target and store the output in a shared directory which is passed to GPU doc stage. Not sure how much of this idea is feasible in our Jenkins configuration. 

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is almost what we do now (the GPU docker image runs the tutorials and sends a tar.gz to another step to deploy it), we could do the same here and add some way of unifying the two (like unpack, make sure no file names collide, send the whole thing to the docs deploy)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@driazati Does this work between executors? Can we run something on ci-qemu and pass it to ci-gpu for doc generation?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

k, so for this PR the delta is at least that now we test the tvmc commands to make sure they don't crash. next it would be great to do what @driazati mentioned, where we run script_convert.py in ci-qemu and then forward the output to the doc-builder step. let's merge this now though to make incremental progress.


# Tutorials running with Zephyr
export TVM_MICRO_USE_HW=1
Expand Down