Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

366 labs raspi support #416

Merged
merged 18 commits into from
Sep 1, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions _quarto.yml
Original file line number Diff line number Diff line change
Expand Up @@ -153,6 +153,12 @@ book:
- contents/labs/seeed/xiao_esp32s3/object_detection/object_detection.qmd
- contents/labs/seeed/xiao_esp32s3/kws/kws.qmd
- contents/labs/seeed/xiao_esp32s3/motion_classification/motion_classification.qmd
- part: contents/labs/raspi/raspi.qmd
chapters:
- contents/labs/raspi/setup/setup.qmd
- contents/labs/raspi/image_classification/image_classification.qmd
- contents/labs/raspi/object_detection/object_detection.qmd
- contents/labs/raspi/llm/llm.qmd
- part: contents/labs/shared/shared.qmd
chapters:
- contents/labs/shared/kws_feature_eng/kws_feature_eng.qmd
Expand Down
40 changes: 27 additions & 13 deletions contents/labs/getting_started.qmd
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Getting Started {.unnumbered}

Welcome to the exciting world of embedded machine learning and TinyML! In this hands-on lab series, you'll explore various projects that demonstrate the power of running machine learning models on resource-constrained devices. Before diving into the projects, let's ensure you have the necessary hardware and software set up.
Welcome to the exciting world of embedded machine learning and TinyML! In this hands-on lab series, you'll explore various projects demonstrating the power of running machine learning models on resource-constrained devices. Before diving into the projects, ensure you have the necessary hardware and software.

## Hardware Requirements

Expand All @@ -12,15 +12,25 @@ To follow along with the hands-on labs, you'll need the following hardware:
- [Arduino Nicla Vision pinout diagram](https://docs.arduino.cc/resources/pinouts/ABX00051-full-pinout.pdf)

2. **XIAO ESP32S3 Sense board**

- The Seeed Studio XIAO ESP32S3 Sense is a tiny, feature-packed board designed for makers, hobbyists, and students interested in exploring edge AI applications. It comes with a camera, microphone, and IMU, making it easy to get started with projects like image classification, keyword spotting, and motion detection.
- [XIAO ESP32S3 Sense specifications](https://wiki.seeedstudio.com/xiao_esp32s3_getting_started/#specification)
- [XIAO ESP32S3 Sense pinout diagram](https://wiki.seeedstudio.com/xiao_esp32s3_getting_started/#hardware-overview)

3. **Additional accessories**
- USB-C cable for programming and powering the boards

3. **Raspberry Pi - Single Computer board**
- The Raspberry Pi is a powerful and versatile single-board computer that has become an essential tool for engineers across various disciplines. Developed by the [Raspberry Pi Foundation](https://www.raspberrypi.org/), these compact devices offer a unique combination of affordability, computational power, and extensive GPIO (General Purpose Input/Output) capabilities, making them ideal for prototyping, embedded systems development, and advanced engineering projects.
- [Raspberry Pi Hardware Documentation](https://www.raspberrypi.com/documentation/computers/raspberry-pi.html)
- [Camera Documentation](https://www.raspberrypi.com/documentation/accessories/camera.html)

4. **Additional accessories**
- USB-C cable for programming and powering the XIAO
- Micro-USB cable for programming and powering the Nicla
- Power Supply for the Raspberries
- Breadboard and jumper wires (optional, for connecting additional sensors)

The Arduino Nicla Vision is tailored for professional-grade applications, offering advanced features and performance suitable for demanding industrial projects. On the other hand, the Seeed Studio XIAO ESP32S3 Sense is geared towards makers, hobbyists, and students who want to explore edge AI applications in a more accessible and beginner-friendly format. Both boards have their strengths and target audiences, allowing users to choose the one that best fits their needs and skill level.
The Arduino Nicla Vision is tailored for professional-grade applications, offering advanced features and performance suitable for demanding industrial projects. On the other hand, the Seeed Studio XIAO ESP32S3 Sense is geared toward makers, hobbyists, and students who want to explore edge AI applications in a more accessible and beginner-friendly format. Both boards have their strengths and target audiences, allowing users to choose the best fit for their needs and skill level. The Raspberry Pi is aimed at more advanced engineering and machine learning projects.



## Software Requirements

Expand All @@ -29,33 +39,37 @@ To program the boards and develop embedded machine learning projects, you'll nee
1. **Arduino IDE**
- Download and install
- Install [Arduino IDE](https://www.arduino.cc/en/software)
- [Arduino CLI](https://arduino.github.io/arduino-cli/1.0/)

- **Edge Impulse Studio**
- Follow the [installation guide](https://docs.arduino.cc/software/ide-v1/tutorials/Windows) for your specific OS.
- [Arduino CLI](https://arduino.github.io/arduino-cli/1.0/)

- Configure the Arduino IDE for the [Arduino Nicla Vision](https://docs.arduino.cc/software/ide-v1/tutorials/getting-started/cores/arduino-mbed_nicla) and [XIAO ESP32S3 Sense](https://wiki.seeedstudio.com/xiao_esp32s3_getting_started/#software-setup) boards.

2. **OpenMV IDE (optional)**

- Download and install the [OpenMV IDE](https://openmv.io/pages/download) for your operating system.
- Configure the OpenMV IDE for the [Arduino Nicla Vision](https://docs.arduino.cc/tutorials/nicla-vision/getting-started/).

3. **Edge Impulse Studio**
- Sign up for a free account on the [Edge Impulse Studio](https://studio.edgeimpulse.com/login).

- Install [Edge Impulse CLI](https://docs.edgeimpulse.com/docs/tools/edge-impulse-cli/cli-installation)

- Follow the guides to connect your [Arduino Nicla Vision](https://docs.edgeimpulse.com/docs/edge-ai-hardware/mcu/arduino-nicla-vision) and [XIAO ESP32S3 Sense](https://docs.edgeimpulse.com/docs/edge-ai-hardware/mcu/seeed-xiao-esp32s3-sense) boards to Edge Impulse Studio.
4. **Raspberry Pi OS**
- Download and install the [Raspberry Pi Imager](https://www.raspberrypi.com/software/)

## Network Connectivity

Some projects may require internet connectivity for data collection or model deployment. Ensure that your development environment has a stable internet connection, either through Wi-Fi or Ethernet.
Some projects may require internet connectivity for data collection or model deployment. Ensure your development environment connection is stable through Wi-Fi or Ethernet. For the Raspberry Pi, having a Wi-Fi or Ethernet connection is necessary for remote operation without the necessity to plug in a monitor, keyboard, and mouse.

- For the Arduino Nicla Vision, you can use the onboard Wi-Fi module to connect to a wireless network.

- For the XIAO ESP32S3 Sense, you can use the onboard Wi-Fi module or connect an external Wi-Fi or Ethernet module using the available pins.

- For the Raspberry Pi, you can use the onboard Wi-Fi module to connect an external Wi-Fi or Ethernet module using the available connector.

## Conclusion

With your hardware and software set up, you're now ready to embark on your embedded machine learning journey. The hands-on labs will guide you through various projects, covering topics such as image classification, object detection, keyword spotting, and motion classification.
With your hardware and software set up, you're ready to embark on your embedded machine learning journey. The hands-on labs will guide you through various projects, covering topics like image classification, object detection, keyword spotting, and motion classification.

If you encounter any issues or have questions, don't hesitate to consult the troubleshooting guides, forums, or reach out to the community for support.
If you encounter any issues or have questions, don't hesitate to consult the troubleshooting guides or forums or seek support from the community.

Let's dive in and unlock the potential of ML on real (tiny) systems!
38 changes: 19 additions & 19 deletions contents/labs/labs.qmd
Original file line number Diff line number Diff line change
Expand Up @@ -8,9 +8,9 @@ By completing these labs, we hope learners will:

:::{.callout-tip}

* Gain proficiency in setting up and deploying ML models on supported devices enabling you to tackle real-world ML deployment scenarios with confidence.
* Gain proficiency in setting up and deploying ML models on supported devices, enabling you to tackle real-world ML deployment scenarios with confidence.

* Understand the steps involved in adapting and experimenting with ML models for different applications allowing you to optimize performance and efficiency.
* Understand the steps involved in adapting and experimenting with ML models for different applications, allowing you to optimize performance and efficiency.

* Learn troubleshooting techniques specific to embedded ML deployments equipping you with the skills to overcome common pitfalls and challenges.

Expand All @@ -34,22 +34,21 @@ These labs are designed for:

## Supported Devices

+----------------------------+--------------------------+--------------------------+-------------------------------+
| Exercise | Nicla Vision | XIAO ESP32S3 | Raspberry Pi |
| | (Arduino Nicla Vision) | (Seeed XIAO ESP32S3) | (Raspberry Pi Foundation) |
+:===========================+:=========================+:=========================+:==============================+
| Installation & Setup | ✅ | ✅ | ✅ |
+----------------------------+--------------------------+--------------------------+-------------------------------+
| Keyword Spotting (KWS) | ✅ | ✅ | |
+----------------------------+--------------------------+--------------------------+-------------------------------+
| Image Classification | ✅ | ✅ | Coming soon. |
+----------------------------+--------------------------+--------------------------+-------------------------------+
| Object Detection | ✅ | ✅ | Coming soon. |
+----------------------------+--------------------------+--------------------------+-------------------------------+
| Motion Detection | ✅ | ✅ | |
+----------------------------+--------------------------+--------------------------+-------------------------------+
| Small Language Models (SLM)| | | Coming soon. |
+----------------------------+--------------------------+--------------------------+-------------------------------+
+----------------------------+----------------------------------------------------------------+----------------------------------------------------------------------------+----------------------------------------------+
| Exercise | [Nicla Vision](https://store.arduino.cc/products/nicla-vision) | [XIAO ESP32S3](https://wiki.seeedstudio.com/xiao_esp32s3_getting_started/) | [Raspberry Pi](https://www.raspberrypi.com/) |
+:===========================+:===============================================================+:===========================================================================+:=============================================+
| Installation & Setup | <i class="bi bi-check2-square"></i> | <i class="bi bi-check2-square"></i> | <i class="bi bi-check2-square"></i> |
+----------------------------+----------------------------------------------------------------+----------------------------------------------------------------------------+----------------------------------------------+
| Keyword Spotting (KWS) | <i class="bi bi-check2-square"></i> | <i class="bi bi-check2-square"></i> | |
+----------------------------+----------------------------------------------------------------+----------------------------------------------------------------------------+----------------------------------------------+
| Image Classification | <i class="bi bi-check2-square"></i> | <i class="bi bi-check2-square"></i> | Coming soon. |
+----------------------------+----------------------------------------------------------------+----------------------------------------------------------------------------+----------------------------------------------+
| Object Detection | <i class="bi bi-check2-square"></i> | <i class="bi bi-check2-square"></i> | Coming soon. |
+----------------------------+----------------------------------------------------------------+----------------------------------------------------------------------------+----------------------------------------------+
| Motion Detection | <i class="bi bi-check2-square"></i> | <i class="bi bi-check2-square"></i> | |
+----------------------------+----------------------------------------------------------------+----------------------------------------------------------------------------+----------------------------------------------+
| Small Language Models (SLM)| | | Coming soon. |
+----------------------------+----------------------------------------------------------------+----------------------------------------------------------------------------+----------------------------------------------+

## Lab Structure

Expand All @@ -71,4 +70,5 @@ If you encounter any issues during the labs, consult the troubleshooting comment

## Credits

Special credit and thanks to [Prof. Marcelo Rovai Mjrovai](https://github.com/Mjrovai) for his valuable contributions to the development and continuous refinement of these labs.
Special credit and thanks to [Prof. Marcelo Rovai](https://github.com/Mjrovai) for his valuable contributions to the development and continuous refinement of these labs.

Loading
Loading