We are excited to introduce ESP-DL v3.0.0 , a lightweight and efficient neural network inference framework tailored for ESP series chips. This release brings significant enhancements and new features, making it easier than ever to develop AI applications on Espressif's System on Chips (SoCs). However, please be aware that v3.0.0 is not compatible with v2.0. Below are the key updates and improvements in this release:
Key Features and Improvements
ESP-DL Standard Model Format
Introduced a new model format similar to ONNX but optimized for ESP-DL, using FlatBuffers instead of Protobuf. This format supports zero-copy deserialization and has a file extension of .espdl.
Efficient Operator Implementation
Enhanced support by ESP32-S3 and ESP32-P4 PIE (Processor Instruction Extensions) for common AI operators such as Conv, Gemm, Add, and Mul.
Static Memory Planner
A new memory planner automatically allocates layers to the optimal memory location based on the user-specified internal RAM size, ensuring both efficient memory usage and faster overall performance.
Dual-Core Scheduling
Introduced automatic dual-core scheduling to fully utilize the dual-core computing power of ESP chips. Currently, Conv2D and DepthwiseConv2D operators support dual-core scheduling.
8-bit LUT Activation
All activation functions except ReLU and PReLU are implemented using an 8-bit Look-Up Table (LUT) method to accelerate inference. This allows for flexible activation function usage while maintaining consistent computational complexity.
Supported Models
Supported Operators
The ESP-DL operator interface is aligned with ONNX. The opset 13 is recommended to export ONNX.
Currently, the following 31 operators have been implemented and tested. Some operators do not implement all functionalities and attributes. Please refer to the description of each operator or test cases for details.
| Operator | int8 | int16 | Description |
|---|---|---|---|
| Add(ESP-DL)(ONNX) | ✔ | ✔ | Support up to 4D |
| AveragePool(ESP-DL)(ONNX) | ✔ | ✔ | |
| Clip(ESP-DL)(ONNX) | ✔ | ✔ | |
| Concat(ESP-DL)(ONNX) | ✔ | ✔ | |
| Conv(ESP-DL)(ONNX) | ✔ | ✔ | Groups only support 1 or input_channels |
| Div(ESP-DL)(ONNX) | ✔ | ✔ | Support up to 4D |
| Exp(ESP-DL)(ONNX) | ✔ | ✔ | |
| Flatten(ESP-DL)(ONNX) | ✔ | ✔ | |
| Gemm(ESP-DL)(ONNX) | ✔ | ✔ | |
| GlobalAveragePool(ESP-DL)(ONNX) | ✔ | ✔ | |
| HardSigmoid(ESP-DL)(ONNX) | ✔ | ✔ | |
| HardSwish(ESP-DL)(ONNX) | ✔ | ✔ | |
| LeakyRelu(ESP-DL)(ONNX) | ✔ | ✔ | |
| Log(ESP-DL)(ONNX) | ✔ | ✔ | |
| MatMul(ESP-DL)(ONNX) | ✔ | ✔ | Support up to 4D |
| MaxPool(ESP-DL)(ONNX) | ✔ | ✔ | |
| Mul(ESP-DL)(ONNX) | ✔ | ✔ | Support up to 4D |
| Pad(ESP-DL)(ONNX) | ✔ | ✔ | Do not support wrap mode |
| PRelu(ESP-DL)(ONNX) | ✔ | ✔ | |
| Reshape(ESP-DL)(ONNX) | ✔ | ✔ | |
| Resize(ESP-DL)(ONNX) | ✔ | ✖ | Only support nearest and do not support roi |
| Sigmoid(ESP-DL)(ONNX) | ✔ | ✔ | |
| Slice(ESP-DL)(ONNX) | ✔ | ✔ | |
| Softmax(ESP-DL)(ONNX) | ✔ | ✔ | Dtype of output is float32 |
| Split(ESP-DL)(ONNX) | ✔ | ✔ | |
| Sqrt(ESP-DL)(ONNX) | ✔ | ✔ | |
| Squeeze(ESP-DL)(ONNX) | ✔ | ✔ | |
| Sub(ESP-DL)(ONNX) | ✔ | ✔ | Support up to 4D |
| Tanh(ESP-DL)(ONNX) | ✔ | ✔ | |
| Transpose(ESP-DL)(ONNX) | ✔ | ✔ | |
| Unsqueeze(ESP-DL)(ONNX) | ✔ | ✔ |
Important Note
- v3.0.0 is not backward compatible with v2.0.0. Please ensure you update your workflows and models accordingly.
We hope you enjoy the new features and improvements in ESP-DL v3.0.0. For any questions or feedback, feel free to reach out via the issues page. Happy coding!