Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Doc] Add PicoDet & PaddleClas Android demo docs #412

Merged
merged 20 commits into from
Oct 23, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
20 commits
Select commit Hold shift + click to select a range
672479d
[Backend] Add override flag to lite backend
DefTruth Oct 20, 2022
e0198f6
Merge branch 'PaddlePaddle:develop' into android_build_docs
DefTruth Oct 20, 2022
95bf273
Merge branch 'PaddlePaddle:develop' into android_build_docs
DefTruth Oct 21, 2022
cda3e95
[Docs] Add Android C++ SDK build docs
DefTruth Oct 21, 2022
f5fdd76
Merge branch 'develop' into android_build_docs
DefTruth Oct 21, 2022
b97e2e2
[Doc] fix android_build_docs typos
DefTruth Oct 21, 2022
265f8ac
Update CMakeLists.txt
DefTruth Oct 21, 2022
3aca8c7
Update android.md
DefTruth Oct 21, 2022
75d9eba
Merge branch 'develop' into android_build_docs
DefTruth Oct 21, 2022
942c56e
[Doc] Add PicoDet Android demo docs
DefTruth Oct 21, 2022
699561e
Merge branch 'android_build_docs' of https://github.com/DefTruth/Fast…
DefTruth Oct 21, 2022
5f58e3f
Merge branch 'develop' into android_build_docs
DefTruth Oct 21, 2022
e10666a
Merge branch 'develop' into android_build_docs
DefTruth Oct 22, 2022
9217301
[Doc] Update PicoDet Andorid demo docs
DefTruth Oct 22, 2022
39f84c9
Merge branch 'PaddlePaddle:develop' into android_build_docs
DefTruth Oct 22, 2022
98a3adf
Merge branch 'android_build_docs' of https://github.com/DefTruth/Fast…
DefTruth Oct 22, 2022
c9eed32
[Doc] Update PaddleClasModel Android demo docs
DefTruth Oct 22, 2022
d37e626
[Doc] Update fastdeploy android jni docs
DefTruth Oct 22, 2022
8f837d9
[Doc] Update fastdeploy android jni usage docs
DefTruth Oct 22, 2022
dad7217
Merge branch 'develop' into android_build_docs
jiangjiajun Oct 23, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions docs/cn/build_and_install/android.md
Original file line number Diff line number Diff line change
Expand Up @@ -102,3 +102,4 @@ make install
如何使用FastDeploy Android C++ SDK 请参考使用案例文档:
- [图像分类Android使用文档](../../../examples/vision/classification/paddleclas/android/README.md)
- [目标检测Android使用文档](../../../examples/vision/detection/paddledetection/android/README.md)
- [在 Android 通过 JNI 中使用 FastDeploy C++ SDK](../../../../../docs/cn/faq/use_cpp_sdk_on_android.md)
231 changes: 231 additions & 0 deletions docs/cn/faq/use_cpp_sdk_on_android.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,231 @@
# 在 Android 中通过 JNI 使用 FastDeploy C++ SDK
本文档将以PicoDet为例,讲解如何通过JNI,将FastDeploy中的模型封装到Android中进行调用。阅读本文档,您至少需要了解C++、Java、JNI以及Android的基础知识。如果您主要关注如何在Java层如何调用FastDeploy的API,则可以不阅读本文档。

## 目录
- [新建Java类并定义native API](#Java)
- [Android Studio 生成JNI函数定义](#JNI)
- [在C++层实现JNI函数](#CPP)
- [编写CMakeLists.txt及配置build.gradle](#CMakeAndGradle)
- [更多FastDeploy Android 使用案例](#Examples)

## 新建Java类并定义native API
<div id="Java"></div>

```java
public class PicoDet {
protected long mNativeModelContext = 0; // Context from native.
protected boolean mInitialized = false;
// ...
// Bind predictor from native context.
private static native long bindNative(String modelFile,
String paramsFile,
String configFile,
int cpuNumThread,
boolean enableLiteFp16,
int litePowerMode,
String liteOptimizedModelDir,
boolean enableRecordTimeOfRuntime,
String labelFile);

// Call prediction from native context.
private static native long predictNative(long nativeModelContext,
Bitmap ARGB8888Bitmap,
boolean saved,
String savedImagePath,
float scoreThreshold,
boolean rendering);

// Release buffers allocated in native context.
private static native boolean releaseNative(long nativeModelContext);

// Initializes at the beginning.
static {
FastDeployInitializer.init();
}
}
```
这些被标记为native的接口是需要通过JNI的方式实现,并在Java层供PicoDet类调用。完整的PicoDet Java代码请参考 [PicoDet.java](../../../examples/vision/detection/paddledetection/android/app/src/main/java/com/baidu/paddle/fastdeploy/vision/detection/PicoDet.java) 。各个函数说明如下:
- `bindNative`: C++层初始化模型资源,如果成功初始化,则返回指向该模型的指针(long类型),否则返回0指针
- `predictNative`: 通过已经初始化好的模型指针,在C++层执行预测代码,如果预测成功则返回指向预测结果的指针,否则返回0指针。注意,该结果指针在当次预测使用完之后需要释放,具体操作请参考 [PicoDet.java](../../../examples/vision/detection/paddledetection/android/app/src/main/java/com/baidu/paddle/fastdeploy/vision/detection/PicoDet.java) 中的predict函数。
- `releaseNative`: 根据传入的模型指针,在C++层释放模型资源。

## Android Studio 生成JNI函数定义
<div id="JNI"></div>

Android Studio 生成 JNI 函数定义: 鼠标停留在Java中定义的native函数上,Android Studio 便会提示是否要创建JNI函数定义;这里,我们把JNI函数定义创建在一个事先创建好的c++文件`picodet_jni.cc`上;

- 使用Android Studio创建JNI函数定义:
![](https://user-images.githubusercontent.com/31974251/197341065-cdf8f626-4bb1-4a57-8d7a-80b382fe994e.png)

- 将JNI函数定义创建在picodet_jni.cc上:
![](https://user-images.githubusercontent.com/31974251/197341190-b887dec5-fa75-43c9-9ab3-7ead50c0eb45.png)

- 创建的JNI函数定义如下:
![](https://user-images.githubusercontent.com/31974251/197341274-e9671bac-9e77-4043-a870-9d5db914586b.png)

其他native函数对应的JNI函数定义的创建和此流程一样。

## 在C++层实现JNI函数
<div id="CPP"></div>

以下为PicoDet JNI层实现的示例,相关的辅助函数不在此处赘述,完整的C++代码请参考 [android/app/src/main/cpp](../../../examples/vision/detection/paddledetection/android/app/src/main/cpp/).
```C++
#include <jni.h> // NOLINT
#include "fastdeploy_jni.h" // NOLINT

#ifdef __cplusplus
extern "C" {
#endif

// 绑定C++层的模型
JNIEXPORT jlong JNICALL
Java_com_baidu_paddle_fastdeploy_vision_detection_PicoDet_bindNative(
JNIEnv *env, jclass clazz, jstring model_file, jstring params_file,
jstring config_file, jint cpu_num_thread, jboolean enable_lite_fp16,
jint lite_power_mode, jstring lite_optimized_model_dir,
jboolean enable_record_time_of_runtime, jstring label_file) {
std::string c_model_file = fastdeploy::jni::ConvertTo<std::string>(env, model_file);
std::string c_params_file = fastdeploy::jni::ConvertTo<std::string>(env, params_file);
std::string c_config_file = astdeploy::jni::ConvertTo<std::string>(env, config_file);
std::string c_label_file = fastdeploy::jni::ConvertTo<std::string>(env, label_file);
std::string c_lite_optimized_model_dir = fastdeploy::jni::ConvertTo<std::string>(env, lite_optimized_model_dir);
auto c_cpu_num_thread = static_cast<int>(cpu_num_thread);
auto c_enable_lite_fp16 = static_cast<bool>(enable_lite_fp16);
auto c_lite_power_mode = static_cast<fastdeploy::LitePowerMode>(lite_power_mode);
fastdeploy::RuntimeOption c_option;
c_option.UseCpu();
c_option.UseLiteBackend();
c_option.SetCpuThreadNum(c_cpu_num_thread);
c_option.SetLitePowerMode(c_lite_power_mode);
c_option.SetLiteOptimizedModelDir(c_lite_optimized_model_dir);
if (c_enable_lite_fp16) {
c_option.EnableLiteFP16();
}
// 如果您实现的是其他模型,比如PPYOLOE,请注意修改此处绑定的C++类型
auto c_model_ptr = new fastdeploy::vision::detection::PicoDet(
c_model_file, c_params_file, c_config_file, c_option);
// Enable record Runtime time costs.
if (enable_record_time_of_runtime) {
c_model_ptr->EnableRecordTimeOfRuntime();
}
// Load detection labels if label path is not empty.
if ((!fastdeploy::jni::AssetsLoaderUtils::IsDetectionLabelsLoaded()) &&
(!c_label_file.empty())) {
fastdeploy::jni::AssetsLoaderUtils::LoadDetectionLabels(c_label_file);
}
// WARN: need to release manually in Java !
return reinterpret_cast<jlong>(c_model_ptr); // native model context
}

// 通过传入的模型指针在C++层进行预测
JNIEXPORT jlong JNICALL
Java_com_baidu_paddle_fastdeploy_vision_detection_PicoDet_predictNative(
JNIEnv *env, jclass clazz, jlong native_model_context,
jobject argb8888_bitmap, jboolean saved, jstring saved_image_path,
jfloat score_threshold, jboolean rendering) {
if (native_model_context == 0) {
return 0;
}
cv::Mat c_bgr;
if (!fastdeploy::jni::ARGB888Bitmap2BGR(env, argb8888_bitmap, &c_bgr)) {
return 0;
}
auto c_model_ptr = reinterpret_cast<fastdeploy::vision::detection::PicoDet *>(
native_model_context);
auto c_result_ptr = new fastdeploy::vision::DetectionResult();
t = fastdeploy::jni::GetCurrentTime();
if (!c_model_ptr->Predict(&c_bgr, c_result_ptr)) {
delete c_result_ptr;
return 0;
}
// ...
return reinterpret_cast<jlong>(c_result_ptr); // native result context
}

// 在C++层释放模型资源
JNIEXPORT jboolean JNICALL
Java_com_baidu_paddle_fastdeploy_vision_detection_PicoDet_releaseNative(
JNIEnv *env, jclass clazz, jlong native_model_context) {
if (native_model_context == 0) {
return JNI_FALSE;
}
auto c_model_ptr = reinterpret_cast<fastdeploy::vision::detection::PicoDet *>(
native_model_context);
// ...
delete c_model_ptr;
return JNI_TRUE;
}

#ifdef __cplusplus
}
#endif
```
## 编写CMakeLists.txt及配置build.gradle
<div id="CMakeAndGradle"></div>

实现好的JNI代码,需要被编译成so库,才能被Java调用,为实现该目的,需要在build.gradle中添加JNI项目支持,并编写对应的CMakeLists.txt。
- build.gradle中配置NDK、CMake以及Android ABI
```java
android {
defaultConfig {
// 省略其他配置 ...
externalNativeBuild {
cmake {
arguments '-DANDROID_PLATFORM=android-21', '-DANDROID_STL=c++_shared', "-DANDROID_TOOLCHAIN=clang"
abiFilters 'armeabi-v7a', 'arm64-v8a'
cppFlags "-std=c++11"
}
}
}
// 省略其他配置 ...
externalNativeBuild {
cmake {
path file('src/main/cpp/CMakeLists.txt')
version '3.10.2'
}
}
ndkVersion '20.1.5948944'
}
```
- 编写CMakeLists.txt示例
```cmake
cmake_minimum_required(VERSION 3.10.2)
project("fastdeploy_jni")

set(FastDeploy_DIR "${CMAKE_CURRENT_SOURCE_DIR}/../../../libs/fastdeploy-android-0.4.0-shared")

find_package(FastDeploy REQUIRED)

include_directories(${CMAKE_CURRENT_SOURCE_DIR})
include_directories(${FastDeploy_INCLUDE_DIRS})

add_library(
fastdeploy_jni
SHARED
utils_jni.cc
bitmap_jni.cc
vision/results_jni.cc
vision/visualize_jni.cc
vision/detection/picodet_jni.cc
vision/classification/paddleclas_model_jni.cc)

find_library(log-lib log)

target_link_libraries(
# Specifies the target library.
fastdeploy_jni
jnigraphics
${FASTDEPLOY_LIBS}
GLESv2
EGL
${log-lib}
)
```
完整的工程示例,请参考 [android/app/src/main/cpp/CMakelists.txt](../../../examples/vision/detection/paddledetection/android/app/src/main/cpp/) 以及 [android/app/build.gradle](../../../examples/vision/detection/paddledetection/android/app/build.gradle).

## 更多FastDeploy Android 使用案例
<div id="Examples"></div>

更多FastDeploy Android 使用案例请参考以下文档:
- [图像分类Android使用文档](../../../examples/vision/classification/paddleclas/android/README.md)
- [目标检测Android使用文档](../../../examples/vision/detection/paddledetection/android/README.md)
2 changes: 2 additions & 0 deletions docs/cn/faq/use_java_sdk_on_android.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
## 在 Android 中使用 FastDeploy Java SDK
- TODO
Loading