refactor impl

README.md

@@ -4,7 +4,8 @@
 
 ## 简介
 
-**PyAXEngine** 基于 cffi 模块实现了 Axera NPU Runtime 的 Python API，并同时支持开发板和M.2算力卡形态，方便开源社区开发者使用 Python 脚本快速构建 NPU 推理脚本
+**PyAXEngine** 基于 cffi 模块实现了 Axera NPU Runtime 的 Python API，其 Python API 与 ONNXRuntime 高度兼(相)容(似)，并同时支持开发板和M.2算力卡形态，方便开源社区开发者使用
+Python 脚本快速构建 NPU 推理脚本
 
 支持芯片
 
@@ -16,61 +17,94 @@
 - python >= 3.8
 - cffi >= 1.0.0
 - ml-dtypes >= 0.1.0
+- numpy >= 1.22.0
 
 ## 快速上手
 
 基于社区开发板 **爱芯派Pro(AX650N)** 进行展示
 
 ### 获取 wheel 包并安装
 
-- [下载链接](https://github.com/AXERA-TECH/pyaxengine/releases/download/0.0.1rc3/axengine-0.0.1-py3-none-any.whl)
+- [下载链接](https://github.com/AXERA-TECH/pyaxengine/releases/latest)
 - 将 `axengine-x.x.x-py3-none-any.whl` 拷贝到开发板上，执行 `pip install axengine-x.x.x-py3-none-any.whl` 安装
 
 ### 简单示例
 
 将 [classification.py](https://github.com/AXERA-TECH/pyaxengine/blob/main/examples/classification.py) 拷贝到开发板上并执行。
 
-```
-root@ax650:~/samples# python3 classification.py
-[INFO] Chip type: ChipType.AX650
-[INFO] Engine version: 2.7.2a
+```bash
+root@ax650:~/samples# python3 classification.py -m /opt/data/npu/models/mobilenetv2.axmodel -i /opt/data/npu/images/cat.jpg
+[INFO] Available providers:  ['AXCLRTExecutionProvider', 'AxEngineExecutionProvider']
+[INFO] Using provider: AxEngineExecutionProvider
+[INFO] Chip type: ChipType.MC50
 [INFO] VNPU type: VNPUType.DISABLED
+[INFO] Engine version: 2.10.1s
 [INFO] Model type: 0 (single core)
 [INFO] Compiler version: 1.2-patch2 7e6b2b5f
-Top 5 Predictions:
-Class Index: 282, Score: 9.77352523803711
-Class Index: 278, Score: 8.981077194213867
-Class Index: 277, Score: 8.452778816223145
-Class Index: 281, Score: 8.320704460144043
-Class Index: 287, Score: 7.924479961395264
-
-# 默认将自动检测计算设备，但也可以强制要求跑在AX650 M.2算力卡上，假设设备号是1，（设备号必须大于等于0，具体查看axcl-smi）
-root@ax650:~/samples# python3 classification.py -b axcl -d 1
+  ------------------------------------------------------
+  Top 5 Predictions:
+    Class Index: 282, Score: 9.774
+    Class Index: 278, Score: 8.981
+    Class Index: 277, Score: 8.453
+    Class Index: 281, Score: 8.321
+    Class Index: 287, Score: 7.924
+  ------------------------------------------------------
+  min =   1.004 ms   max =   22.512 ms   avg =   1.252 ms
+  ------------------------------------------------------
+```
+
+示例也演示了如何选择计算设备：这意味着既可以在 **AX650/AX630C** 等开发板上运行，也可以在 AX650 M.2 算力卡上运行。
+
+切换计算设备的方式是通过 `-p` 参数指定，如 `-p AxEngineExecutionProvider` 表示使用开发板上的 NPU 进行推理，而 `-p AXCLRTExecutionProvider` 表示使用 M.2 算力卡进行推理。
+注意：在使用 M.2 算力卡进行推理时，需要将算力卡插入宿主机上，并且已经安装驱动，详见： [axcl](https://axcl-docs.readthedocs.io/zh-cn/latest/)。
+
+```bash
+root@ax650:~/samples# python3 classification.py -m /opt/data/npu/models/mobilenetv2.axmodel -i /opt/data/npu/images/cat.jpg -p AXCLRTExecutionProvider
+[INFO] Available providers:  ['AXCLRTExecutionProvider', 'AxEngineExecutionProvider']
+[INFO] Using provider: AXCLRTExecutionProvider
 [INFO] SOC Name: AX650N
-[INFO] Runtime version: 1.0.0
 [INFO] VNPU type: VNPUType.DISABLED
 [INFO] Compiler version: 1.2-patch2 7e6b2b5f
-grp_id: 0
-input size: 1
-	name: input
-		shape: 1 x 224 x 224 x 3
-output size: 1
-	name: output
-		shape: 1 x 1000
-[INFO] cost time in host to device: 0.617ms, inference: 1.087ms, device to host: 0.266ms
-Top 5 Predictions:
-Class Index: 282, Score: 9.77352523803711
-Class Index: 278, Score: 8.981077194213867
-Class Index: 277, Score: 8.452778816223145
-Class Index: 281, Score: 8.320704460144043
-Class Index: 287, Score: 7.924479961395264
+  ------------------------------------------------------
+  Top 5 Predictions:
+    Class Index: 282, Score: 9.774
+    Class Index: 278, Score: 8.981
+    Class Index: 277, Score: 8.453
+    Class Index: 281, Score: 8.321
+    Class Index: 287, Score: 7.924
+  ------------------------------------------------------
+  min =   1.673 ms   max =   12.400 ms   avg =   1.805 ms
+  ------------------------------------------------------
+root@ax650:~/samples# python3 classification.py -m /opt/data/npu/models/mobilenetv2.axmodel -i /opt/data/npu/images/cat.jpg -p AxEngineExecutionProvider
+[INFO] Available providers:  ['AXCLRTExecutionProvider', 'AxEngineExecutionProvider']
+[INFO] Using provider: AxEngineExecutionProvider
+[INFO] Chip type: ChipType.MC50
+[INFO] VNPU type: VNPUType.DISABLED
+[INFO] Engine version: 2.10.1s
+[INFO] Model type: 0 (single core)
+[INFO] Compiler version: 1.2-patch2 7e6b2b5f
+  ------------------------------------------------------
+  Top 5 Predictions:
+    Class Index: 282, Score: 9.774
+    Class Index: 278, Score: 8.981
+    Class Index: 277, Score: 8.453
+    Class Index: 281, Score: 8.321
+    Class Index: 287, Score: 7.924
+  ------------------------------------------------------
+  min =   1.004 ms   max =   22.512 ms   avg =   1.252 ms
+  ------------------------------------------------------
 ```
 
+## 社区贡献者
+
+[zylo117](https://github.com/zylo117): 提供了基于 cffi 的 AXCL Runtime Python API 实现
+
 ## 关联项目
 
 - [ax-samples](https://github.com/AXERA-TECH/ax-samples)
 - [ax-llm](https://github.com/AXERA-TECH/ax-llm)
 - [pulsar2](https://pulsar2-docs.readthedocs.io/zh-cn/latest/)
+- [axcl](https://axcl-docs.readthedocs.io/zh-cn/latest/)
 
 ## 技术讨论
 

axengine/__init__.py

@@ -5,6 +5,18 @@
 # written consent of Axera Semiconductor Co., Ltd.
 #
 
-from .session import InferenceSession
-from .ax_session import InferenceSession as AXInferenceSession
-from .axcl_session import InferenceSession as AXCLInferenceSession
+# thanks to community contributors list below:
+#   zylo117: https://github.com/zylo117, first implementation of the axclrt backend
+
+from ._providers import axengine_provider_name, axclrt_provider_name
+from ._providers import get_all_providers, get_available_providers
+
+# check if axclrt is installed, or is a supported chip(e.g. AX650, AX620E etc.)
+_available_providers = get_available_providers()
+if not _available_providers:
+    raise ImportError(
+        f"No providers found. Please make sure you have installed one of the following: {get_all_providers()}")
+print("[INFO] Available providers: ", _available_providers)
+
+from ._node import NodeArg
+from ._session import SessionOptions, InferenceSession