Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Failed to run background_subtraction_demo_gapi with Media Files Available for Demos #3928

Open
kminemur opened this issue Mar 5, 2024 · 6 comments · May be fixed by #3903
Open

Failed to run background_subtraction_demo_gapi with Media Files Available for Demos #3928

kminemur opened this issue Mar 5, 2024 · 6 comments · May be fixed by #3903
Assignees

Comments

@kminemur
Copy link
Contributor

kminemur commented Mar 5, 2024

Hi team, background_subtraction_demo_gapi failed with person-bicycle-car-detection.mp4 from Media Files: https://storage.openvinotoolkit.org/data/test_data/videos.

How to download:
wget https://storage.openvinotoolkit.org/data/test_data/videos/person-bicycle-car-detection.mp4

[Environment]
OS: Ubuntu 22.04.2 LTS
Kernel: 5.15.0-86-generic
Openvino: 2023.3.0
OpenCV: 4.9.0

[Log]
$ ./background_subtraction_demo_gapi -m $HOME/workshop/kazuki/open_model_zoo/demos/background_subtraction_demo/cpp_gapi/intel/instance-segmentation-security-0002/FP32/instance-segmentation-security-0002.xml -i person-bicycle-car-detection.mp4 -at maskrcnn
[ INFO ] OpenVINO
[ INFO ] version: 2023.3.0
[ INFO ] build: 2023.3.0-13775-ceeafaf64f3
[ INFO ] The background matting model /home/yuzhang3/workshop/kazuki/open_model_zoo/demos/background_subtraction_demo/cpp_gapi/intel/instance-segmentation-security-0002/FP32/instance-segmentation-security-0002.xml is loaded to CPU device.
[ ERROR ] [ GENERAL_ERROR ] Exception from src/plugins/intel_cpu/src/infer_request.cpp:393:
Can't set the input tensor with name: image, because the model input (shape=[1,3,?,?]) and the tensor (shape=(1.432.768.3)) are incompatible

@Siddharth-Latthe-07
Copy link

@Wovchena I guess, this issue occurs due to mismatch between the expected input shape of the model and the actual input shape provided by the background_subtraction_demo_gapi demo.
you may try these steps and do let me know if it works:-

  1. resize the video frame to the required dimensions before passing it to the model.
  2. snippet for preprocessing video frames and convert to required format of [1, 3, height, width].
import cv2
import numpy as np

def preprocess_frame(frame, target_height, target_width):
    # Resize the frame
    resized_frame = cv2.resize(frame, (target_width, target_height))
    # Convert the frame to the format [1, 3, height, width]
    input_frame = np.expand_dims(resized_frame.transpose(2, 0, 1), axis=0)
    return input_frame

# Open the video file
cap = cv2.VideoCapture('person-bicycle-car-detection.mp4')

# Read the first frame
ret, frame = cap.read()
if not ret:
    print("Failed to read the video file")
    exit()

# Preprocess the frame
input_frame = preprocess_frame(frame, 432, 768)  # Replace with the target height and width

# Save the preprocessed frame as an example (optional)
np.save('preprocessed_frame.npy', input_frame)

# Release the video capture object
cap.release()
  1. update the demo script with the above change and execute it

plz let me know, if the above works or not
Thanks

@Wovchena
Copy link
Collaborator

The problem is in GAPI which is a part of OpenCV. @TolyaTalamanov and @DariaMityagina were working on fixing. Apparently they dropped this task.

@TolyaTalamanov
Copy link
Contributor

@Siddharth-Latthe-07 The problem occurs because of dynamic nature of the model.

Could you try to fix model input to "some" static shape and re-run demo, please?
https://docs.openvino.ai/2024/openvino-workflow/running-inference/dynamic-shapes.html

@Siddharth-Latthe-07
Copy link

@TolyaTalamanov , Here's my approach to address the above issue sent by you (I tried setting a static input shape for the model using OpenVINO's tools or APIs that allow input resizing.) :-

Step 1: Set the Model Input Shape Statically
explicitly set the input shape to a fixed value. can be done by using the OpenVINO Model Optimizer or directly setting it in code.
(mo.py --input_model instance-segmentation-security-0002.xml --input_shape [1,3,432,768]) OR

import openvino.runtime as ov

# Load the model
core = ov.Core()
model = core.read_model(model="instance-segmentation-security-0002.xml")
compiled_model = core.compile_model(model, device_name="CPU")

# Set a static input shape
input_tensor = compiled_model.input(0)
model.reshape({input_tensor.any_name: [1, 3, 432, 768]})

# Load the new reshaped model
compiled_model = core.compile_model(model, "CPU")

This sets the input to a fixed shape before inference.

Step 2: Update the Demo Script
input_frame = preprocess_frame(frame, 432, 768) (Should match with fixed model input)

After making these changes, re-run the background_subtraction_demo_gapi. This should eliminate the GENERAL_ERROR related to mismatched input shapes.

@TolyaTalamanov
Copy link
Contributor

@Siddharth-Latthe-07 You don't need to update the demo itself. Suggestion is to eliminate dynamism from the model by reshaping it to static shape

@Siddharth-Latthe-07
Copy link

Siddharth-Latthe-07 commented Sep 30, 2024

@TolyaTalamanov, got it

  1. Reshape the model:-
    Use the OpenVINO Python API to reshape the model to a static input shape:
from openvino.runtime import Core

# Initialize the OpenVINO runtime
core = Core()

# Load the model
model = core.read_model(model="instance-segmentation-security-0002.xml")
  1. Reshape the Model to Static Input Dimensions:
    Set the model's input shape explicitly to a static value.
    model.reshape({"image": [1, 3, 432, 768]})

  2. Compile the Model for Inference:
    After reshaping, compile the model for the target device (e.g., CPU):
    compiled_model = core.compile_model(model, device_name="CPU")

  3. Update the OpenVINO Model File
    Once you have reshaped the model using the OpenVINO API, save the updated model to use with your demo:

from openvino.runtime import serialize

serialize(model, "instance-segmentation-security-0002_reshaped.xml", "instance-segmentation-security-0002_reshaped.bin")
  1. Run the Demo with the Updated Model
    Use the reshaped model XML file (instance-segmentation-security-0002_reshaped.xml) for running the background_subtraction_demo_gapi:
    ./background_subtraction_demo_gapi -m path/to/instance-segmentation-security-0002_reshaped.xml -i person-bicycle-car-detection.mp4 -at maskrcnn

By reshaping the model to a static input shape, you eliminate the dynamic nature of the model, which should resolve the input shape compatibility issue during inference.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants