Error in GStreamer Pipeline Syntax with OpenCV in rknn-cpp-Multithreading Project

[nerbivol]

When attempting to run the build-linux_RK3588.sh script in the rknn-cpp-Multithreading project, the following error occurs:

orangepi@orangepi5b:~/Downloads/rknn-cpp-Multithreading-main$ sh build-linux_RK3588.sh
CMake Deprecation Warning at CMakeLists.txt:1 (cmake_minimum_required):
  Compatibility with CMake < 3.5 will be removed from a future version of
  CMake.

  Update the VERSION argument <min> value or use a ...<max> suffix to tell
  CMake that the project does not need compatibility with older versions.


-- Configuring done (0.0s)
-- Generating done (0.0s)
-- Build files have been written to: /home/orangepi/Downloads/rknn-cpp-Multithreading-main/build/build_linux_aarch64
[100%] Built target rknn_yolov5_demo
[100%] Built target rknn_yolov5_demo
Install the project...
-- Install configuration: ""
-- Up-to-date: /home/orangepi/Downloads/rknn-cpp-Multithreading-main/install/rknn_yolov5_demo_Linux/./rknn_yolov5_demo
-- Up-to-date: /home/orangepi/Downloads/rknn-cpp-Multithreading-main/install/rknn_yolov5_demo_Linux/lib/librknnrt.so
-- Up-to-date: /home/orangepi/Downloads/rknn-cpp-Multithreading-main/install/rknn_yolov5_demo_Linux/lib/librga.so
-- Up-to-date: /home/orangepi/Downloads/rknn-cpp-Multithreading-main/install/rknn_yolov5_demo_Linux/.//model
-- Up-to-date: /home/orangepi/Downloads/rknn-cpp-Multithreading-main/install/rknn_yolov5_demo_Linux/.//model/RK3588
-- Up-to-date: /home/orangepi/Downloads/rknn-cpp-Multithreading-main/install/rknn_yolov5_demo_Linux/.//model/RK3588/yolov5s-640-640.rknn
-- Up-to-date: /home/orangepi/Downloads/rknn-cpp-Multithreading-main/install/rknn_yolov5_demo_Linux/.//model/coco_80_labels_list.txt
/home/orangepi/Downloads/rknn-cpp-Multithreading-main
Loading model...
sdk version: 1.5.2 (c6b7b351a@2023-08-23T15:28:22) driver version: 0.9.3
model input num: 1, output num: 3
model is NHWC input fmt
model input height=640, width=640, channel=3
Loading model...
sdk version: 1.5.2 (c6b7b351a@2023-08-23T15:28:22) driver version: 0.9.3
model input num: 1, output num: 3
model is NHWC input fmt
model input height=640, width=640, channel=3
Loading model...
sdk version: 1.5.2 (c6b7b351a@2023-08-23T15:28:22) driver version: 0.9.3
model input num: 1, output num: 3
model is NHWC input fmt
model input height=640, width=640, channel=3

(Camera FPS:21441): GStreamer-CRITICAL **: 17:21:51.985: gst_element_make_from_uri: assertion 'gst_uri_is_valid (uri)' failed
[ WARN:0] global ../modules/videoio/src/cap_gstreamer.cpp (713) open OpenCV | GStreamer warning: Error opening bin: syntax error
[ WARN:0] global ../modules/videoio/src/cap_gstreamer.cpp (480) isPipelinePlaying OpenCV | GStreamer warning: GStreamer: pipeline have not been created
Average:         nan fps/s

[leafqycc]

When attempting to run the build-linux_RK3588.sh script in the rknn-cpp-Multithreading project, the following error occurs:

orangepi@orangepi5b:~/Downloads/rknn-cpp-Multithreading-main$ sh build-linux_RK3588.sh
CMake Deprecation Warning at CMakeLists.txt:1 (cmake_minimum_required):
  Compatibility with CMake < 3.5 will be removed from a future version of
  CMake.

  Update the VERSION argument <min> value or use a ...<max> suffix to tell
  CMake that the project does not need compatibility with older versions.


-- Configuring done (0.0s)
-- Generating done (0.0s)
-- Build files have been written to: /home/orangepi/Downloads/rknn-cpp-Multithreading-main/build/build_linux_aarch64
[100%] Built target rknn_yolov5_demo
[100%] Built target rknn_yolov5_demo
Install the project...
-- Install configuration: ""
-- Up-to-date: /home/orangepi/Downloads/rknn-cpp-Multithreading-main/install/rknn_yolov5_demo_Linux/./rknn_yolov5_demo
-- Up-to-date: /home/orangepi/Downloads/rknn-cpp-Multithreading-main/install/rknn_yolov5_demo_Linux/lib/librknnrt.so
-- Up-to-date: /home/orangepi/Downloads/rknn-cpp-Multithreading-main/install/rknn_yolov5_demo_Linux/lib/librga.so
-- Up-to-date: /home/orangepi/Downloads/rknn-cpp-Multithreading-main/install/rknn_yolov5_demo_Linux/.//model
-- Up-to-date: /home/orangepi/Downloads/rknn-cpp-Multithreading-main/install/rknn_yolov5_demo_Linux/.//model/RK3588
-- Up-to-date: /home/orangepi/Downloads/rknn-cpp-Multithreading-main/install/rknn_yolov5_demo_Linux/.//model/RK3588/yolov5s-640-640.rknn
-- Up-to-date: /home/orangepi/Downloads/rknn-cpp-Multithreading-main/install/rknn_yolov5_demo_Linux/.//model/coco_80_labels_list.txt
/home/orangepi/Downloads/rknn-cpp-Multithreading-main
Loading model...
sdk version: 1.5.2 (c6b7b351a@2023-08-23T15:28:22) driver version: 0.9.3
model input num: 1, output num: 3
model is NHWC input fmt
model input height=640, width=640, channel=3
Loading model...
sdk version: 1.5.2 (c6b7b351a@2023-08-23T15:28:22) driver version: 0.9.3
model input num: 1, output num: 3
model is NHWC input fmt
model input height=640, width=640, channel=3
Loading model...
sdk version: 1.5.2 (c6b7b351a@2023-08-23T15:28:22) driver version: 0.9.3
model input num: 1, output num: 3
model is NHWC input fmt
model input height=640, width=640, channel=3

(Camera FPS:21441): GStreamer-CRITICAL **: 17:21:51.985: gst_element_make_from_uri: assertion 'gst_uri_is_valid (uri)' failed
[ WARN:0] global ../modules/videoio/src/cap_gstreamer.cpp (713) open OpenCV | GStreamer warning: Error opening bin: syntax error
[ WARN:0] global ../modules/videoio/src/cap_gstreamer.cpp (480) isPipelinePlaying OpenCV | GStreamer warning: GStreamer: pipeline have not been created
Average:         nan fps/s

I guess there is no test video in your folder, so an error is reported when running the demo. You can download the test video from https://github.com/leafqycc/rknn-cpp-Multithreading/releases/tag/v0.3, then put it in the project root directory and run it again. You can also refer to the test commands in build-linux_RK3588.sh cd install/rknn_yolov5_demo_Linux/ && ./rknn_yolov5_demo ./model/RK3588/yolov5s-640-640.rknn ../../720p60hz.mp4 to use your own video or camera for testing

[nerbivol]

After modifying the build-linux_RK3588.sh script to cd install/rknn_yolov5_demo_Linux/ && ./rknn_yolov5_demo ./model/RK3588/yolov5s-640-640.rknn ../../720p60hz.mp4 and I got the following error:

sdk version: 1.5.2 (c6b7b351a@2023-08-23T15:28:22) driver version: 0.9.3
model input num: 1, output num: 3
model is NHWC input fmt
model input height=640, width=640, channel=3
Loading model...
sdk version: 1.5.2 (c6b7b351a@2023-08-23T15:28:22) driver version: 0.9.3
model input num: 1, output num: 3
model is NHWC input fmt
model input height=640, width=640, channel=3
rga_api version 1.9.1_[4]
loadLabelName ./model/coco_80_labels_list.txt
E RKNN: [18:52:48.447] failed to submit!, op id: 1, op name: Conv:Conv_0, flags: 0x5, task start: 0, task number: 196, run task counter: 0, int status: 0, please try updating to the latest version of the toolkit2 and runtime from: https://eyun.baidu.com/s/3eTDMk6Y (PWD: rknn)
E RKNN: [18:52:48.553] failed to submit!, op id: 1, op name: Conv:Conv_0, flags: 0x5, task start: 0, task number: 196, run task counter: 0, int status: 0, please try updating to the latest version of the toolkit2 and runtime from: https://eyun.baidu.com/s/3eTDMk6Y (PWD: rknn)

[nerbivol]

When modifying the threadNum variable to 1 in the main.cc file of the rknn-cpp-Multithreading project, the program executes successfully with a frame rate of 42 fps/s. However, it's noted that the recognition quality is poor under this configuration.

When increasing threadNum to 2 or 3, the following error is encountered:

E RKNN: [18:52:48.447] failed to submit!, op id: 1, op name: Conv:Conv_0, flags: 0x5, task start: 0, task number: 196, run task counter: 0, int status: 0, please try updating to the latest version of the toolkit2 and runtime from: https://eyun.baidu.com/s/3eTDMk6Y (PWD: rknn)

[shancaidazf]

When modifying the threadNum variable to 1 in the main.cc file of the rknn-cpp-Multithreading project, the program executes successfully with a frame rate of 42 fps/s. However, it's noted that the recognition quality is poor under this configuration.

When increasing threadNum to 2 or 3, the following error is encountered:

E RKNN: [18:52:48.447] failed to submit!, op id: 1, op name: Conv:Conv_0, flags: 0x5, task start: 0, task number: 196, run task counter: 0, int status: 0, please try updating to the latest version of the toolkit2 and runtime from: https://eyun.baidu.com/s/3eTDMk6Y (PWD: rknn)

yes ,i have the same issue

[leafqycc]

When modifying the variable to in the file of the rknn-cpp-Multithreading project, the program executes successfully with a frame rate of 42 fps/s. However, it's noted that the recognition quality is poor under this configuration.threadNum``1``main.cc
When increasing to or , the following error is encountered:threadNum``2``3

E RKNN: [18:52:48.447] failed to submit!, op id: 1, op name: Conv:Conv_0, flags: 0x5, task start: 0, task number: 196, run task counter: 0, int status: 0, please try updating to the latest version of the toolkit2 and runtime from: https://eyun.baidu.com/s/3eTDMk6Y (PWD: rknn)

yes ,i have the same issue
@nerbivol @shancaidazf
Someone encountered a situation where the first NPU core occupancy rate was low. His Linux kernel was version 6.1. After switching back to version 5.10, it ran normally. Another one is like yours, where the two NPU cores have low usage. At that time, they were running normally after changing to another Linux distribution. I guess there may be compatibility issues with the NPU driver under higher versions of the Linux kernel. But I don’t have an RK3588 development board for debugging now, so if you can’t find the problem, I suggest you change to another Linux distribution.