-
Notifications
You must be signed in to change notification settings - Fork 25
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
VPI_ERROR_OUT_OF_MEMORY #12
Comments
Thanks for the report! Our SQA has been able to reproduce the problem and we're qualifying a fix for this. We'll publish it as a hotfix as soon as trials are complete. |
Thanks for the quick fix, I will test this on monday! |
I can confirm that this fixed my issue. Thanks! |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Hello,
I ran into the issue already mentioned here. Full error message is:
I tried on multiple x86_64 machines with different GPUs (RTX2060S, RTX3070) running Ubuntu 20.04 LTS. Docker container is pulled from NVIDIA-ISAAC-ROS/isaac_ros_common (Docker Version 20.10.15).
I started the default pipeline like this:
ros2 run usb_cam usb_cam_node_exe --ros-args --params-file camera_params.yaml
(Webcam, 1280x960 @ 30fps, calibrated)ros2 run isaac_ros_image_proc isaac_ros_image_proc
Before subscribing to the
image_rect
topic, VRAM usage is pretty stable.nvidia-smi
(on the RTX2060S system) returns:Roughly 60 seconds after subscribing to
image_rect
withros2 topic echo /image_rect
, VPI fails to allocate the required memory:On the RTX2060S, the pipeline stops rectifying most frames but publishes the non-rectified frames instead, while on the RTX3070 the pipeline crashes after a few
VPI_ERROR_OUT_OF_MEMORY
messages. I have attached the logs of the RTX2060S setup:isaac_ros_image_proc_95128_1652183865298.log
usb_cam_node_exe_95110_1652183858680.log
After cancelling the pipeline, the memory is freed immediately.
By commenting out the nodes of the provided pipeline one-by-one, I believe the issue stems from the
rectify_node
. I tried pinpointing the issue further, but unfortunately I'm not too familiar with the VPI framework myself.I can provide more info if required, but I'm not sure what would be useful. I also have an old Quadro P2200 available that I haven't tested yet, but would be able to if that were of any help.
Kind regards,
Finn
The text was updated successfully, but these errors were encountered: