Memory leak - ros2 humble #262

firesurfer · 2023-07-12T12:14:09Z

Issue

usb_cam node allocates memory over time without freeing it until the systems hangs and the oom kills it.

[20524.624725] Out of memory: Killed process 267615 (usb_cam_node_ex) total-vm:31232704kB, anon-rss:27694492kB, file-rss:12320kB, shmem-rss:0kB, UID:1000 pgtables:59852kB oom_score_adj:0

System

Ubuntu 22.04
ROS2 Humble (binary install)
usb_cam: 0.6.0

v4l2-ctl -D

Driver Info:
	Driver name      : uvcvideo
	Card type        : HD Pro Webcam C920
	Bus info         : usb-0000:0c:00.3-2
	Driver version   : 5.19.17
	Capabilities     : 0x84a00001
		Video Capture
		Metadata Capture
		Streaming
		Extended Pix Format
		Device Capabilities
	Device Caps      : 0x04200001
		Video Capture
		Streaming
		Extended Pix Format
Media Driver Info:
	Driver name      : uvcvideo
	Model            : HD Pro Webcam C920
	Serial           : 
	Bus info         : usb-0000:0c:00.3-2
	Media version    : 5.19.17
	Hardware revision: 0x0000000c (12)
	Driver version   : 5.19.17
Interface Info:
	ID               : 0x03000002
	Type             : V4L Video
Entity Info:
	ID               : 0x00000001 (1)
	Name             : HD Pro Webcam C920
	Function         : V4L2 I/O
	Flags            : default
	Pad 0x01000007   : 0: Sink
	  Link 0x0200001c: from remote pad 0x100000a of entity 'Processing 3' (Video Pixel Formatter): Data, Enabled, Immutable

Camera yaml:

    ros__parameters:
      video_device: "/dev/video0"
      framerate: 30.0
      io_method: "mmap"
      frame_id: "camera_1"
      pixel_format: "mjpeg2rgb"
#      color_format: "yuv422p" #Will result in less then 30fps on uncompressed topic
      image_width: 1920
      image_height: 1080
      camera_name: "logitech_c920"
      camera_info_url: "package://winder_camera/config/camera_1_info.yaml"

Further info

Camera image is displayed in rviz

Let me know if you need futher information

The text was updated successfully, but these errors were encountered:

flynneva · 2023-07-13T13:41:50Z

@firesurfer thanks for reporting. I'll dig into this over the weekend when I have some time 👍🏼

flynneva · 2023-07-15T20:02:38Z

@firesurfer I've addressed the major memory leak issues with #264 (or at least a lot of them 😅 ) There are still a few smaller ones left, but I've run out of time today and need to work on some other things this weekend.

Hopefully those fixes are "good enough" for you for now - I'll leave this issue open even after #264 is merged to address the rest of the memory issues later on.

firesurfer · 2023-07-17T09:13:47Z

@flynneva I just tried #264 on my system and I am sorry to disappoint you. It still eats memory rather rapidly. After like 10min it already used like 8 GB.

flynneva · 2023-07-23T16:52:56Z

@firesurfer I believe I've addressed the last memory leak related to this drivers code in #268. There are still 3 memory leaks reported by asan but those are from lower-level ROS 2 packages I believe and not something I can address here in this repo.

I've added a SANITIZE option to the CMakelists.txt file as well so others can quickly and easily address memory leaks too.

Maybe down the road I can add a CI job that uses this option to auto-detect memory leaks in PRs

firesurfer · 2023-07-24T06:54:08Z

@flynneva I just tested #268
It looks in general way better but still starts consuming memory without freeing it (at least way slower than before).
The interesting thing is: I had to subscribe to the raw topic (didnt try the other ones) for this to happen.

I ran valgrind with leak-check=yes, perhaps the output can help you.

==14672== Invalid free() / delete / delete[] / realloc()
==14672==    at 0x484B27F: free (in /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==14672==    by 0x7B99565: av_buffer_unref (in /usr/lib/x86_64-linux-gnu/libavutil.so.56.70.100)
==14672==    by 0x693C098: av_packet_unref (in /usr/lib/x86_64-linux-gnu/libavcodec.so.58.134.100)
==14672==    by 0x6703ABA: std::_Sp_counted_ptr_inplace<usb_cam::formats::MJPEG2RGB, std::allocator<usb_cam::formats::MJPEG2RGB>, (__gnu_cxx::_Lock_policy)2>::_M_dispose() (in /home/user/workspace/webcam_ws/install/usb_cam/lib/libusb_cam.so)
==14672==    by 0x6701731: usb_cam::UsbCam::~UsbCam() (in /home/user/workspace/webcam_ws/install/usb_cam/lib/libusb_cam.so)
==14672==    by 0x6605DA3: usb_cam::UsbCamNode::~UsbCamNode() (in /home/user/workspace/webcam_ws/install/usb_cam/lib/libusb_cam_node.so)
==14672==    by 0x10C4DE: main (in /home/user/workspace/webcam_ws/install/usb_cam/lib/usb_cam/usb_cam_node_exe)
==14672==  Address 0x1b9c5000 is in a rw- mapped file /dev/video0 segment
==14672== 
==14672== 
==14672== HEAP SUMMARY:
==14672==     in use at exit: 161,055,721 bytes in 4,224 blocks
==14672==   total heap usage: 163,301 allocs, 159,078 frees, 1,892,940,843 bytes allocated
==14672== 
==14672== 16 bytes in 1 blocks are possibly lost in loss record 122 of 476
==14672==    at 0x4848899: malloc (in /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==14672==    by 0x4013E4D: malloc (rtld-malloc.h:56)
==14672==    by 0x4013E4D: allocate_dtv_entry (dl-tls.c:684)
==14672==    by 0x4013E4D: allocate_and_init (dl-tls.c:709)
==14672==    by 0x4013E4D: tls_get_addr_tail (dl-tls.c:907)
==14672==    by 0x401826B: __tls_get_addr (tls_get_addr.S:55)
==14672==    by 0x1980A147: ???
==14672==    by 0x40021A1: call_destructors (dl-close.c:129)
==14672==    by 0x4E68C84: _dl_catch_exception (dl-error-skeleton.c:182)
==14672==    by 0x4002635: _dl_close_worker.part.0.isra.0 (dl-close.c:292)
==14672==    by 0x40032A1: _dl_close_worker (dl-close.c:150)
==14672==    by 0x40032A1: _dl_close (dl-close.c:818)
==14672==    by 0x4E68C27: _dl_catch_exception (dl-error-skeleton.c:208)
==14672==    by 0x4E68CF2: _dl_catch_error (dl-error-skeleton.c:227)
==14672==    by 0x4D841AD: _dlerror_run (dlerror.c:138)
==14672==    by 0x4D83ED7: dlclose@@GLIBC_2.34 (dlclose.c:31)
==14672== 
==14672== 64 bytes in 1 blocks are definitely lost in loss record 225 of 476
==14672==    at 0x4848899: malloc (in /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==14672==    by 0x6703F09: usb_cam::formats::MJPEG2RGB::MJPEG2RGB(int const&, int const&) (in /home/user/workspace/webcam_ws/install/usb_cam/lib/libusb_cam.so)
==14672==    by 0x670489C: usb_cam::UsbCam::set_pixel_format_from_string(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) (in /home/user/workspace/webcam_ws/install/usb_cam/lib/libusb_cam.so)
==14672==    by 0x67014B8: usb_cam::UsbCam::configure(usb_cam::parameters_t&, usb_cam::utils::io_method_t const&) (in /home/user/workspace/webcam_ws/install/usb_cam/lib/libusb_cam.so)
==14672==    by 0x660A04D: usb_cam::UsbCamNode::init() (in /home/user/workspace/webcam_ws/install/usb_cam/lib/libusb_cam_node.so)
==14672==    by 0x660BE6A: usb_cam::UsbCamNode::UsbCamNode(rclcpp::NodeOptions const&) (in /home/user/workspace/webcam_ws/install/usb_cam/lib/libusb_cam_node.so)
==14672==    by 0x661399D: rclcpp_components::NodeFactoryTemplate<usb_cam::UsbCamNode>::create_node_instance(rclcpp::NodeOptions const&) (in /home/user/workspace/webcam_ws/install/usb_cam/lib/libusb_cam_node.so)
==14672==    by 0x10BDB4: main (in /home/user/workspace/webcam_ws/install/usb_cam/lib/usb_cam/usb_cam_node_exe)
==14672== 
==14672== 128 bytes in 2 blocks are definitely lost in loss record 360 of 476
==14672==    at 0x4848899: malloc (in /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==14672==    by 0x400F21D: malloc (rtld-malloc.h:56)
==14672==    by 0x400F21D: resize_scopes (dl-open.c:295)
==14672==    by 0x400F21D: dl_open_worker_begin (dl-open.c:707)
==14672==    by 0x4E68C27: _dl_catch_exception (dl-error-skeleton.c:208)
==14672==    by 0x400DF99: dl_open_worker (dl-open.c:782)
==14672==    by 0x4E68C27: _dl_catch_exception (dl-error-skeleton.c:208)
==14672==    by 0x400E34D: _dl_open (dl-open.c:883)
==14672==    by 0x4D846BB: dlopen_doit (dlopen.c:56)
==14672==    by 0x4E68C27: _dl_catch_exception (dl-error-skeleton.c:208)
==14672==    by 0x4E68CF2: _dl_catch_error (dl-error-skeleton.c:227)
==14672==    by 0x4D841AD: _dlerror_run (dlerror.c:138)
==14672==    by 0x4D84747: dlopen_implementation (dlopen.c:71)
==14672==    by 0x4D84747: dlopen@@GLIBC_2.34 (dlopen.c:81)
==14672==    by 0x4A9D8CB: rcutils_load_shared_library (in /opt/ros/humble/lib/librcutils.so)
==14672== 
==14672== 143 (64 direct, 79 indirect) bytes in 1 blocks are definitely lost in loss record 366 of 476
==14672==    at 0x4849013: operator new(unsigned long) (in /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==14672==    by 0x50E3A23: ??? (in /opt/ros/humble/lib/librosidl_typesupport_cpp.so)
==14672==    by 0x55FC888: ???
==14672==    by 0x55DA167: ???
==14672==    by 0x55DA366: ???
==14672==    by 0x55DBFEF: ???
==14672==    by 0x4F47849: rcl_node_init (in /opt/ros/humble/lib/librcl.so)
==14672==    by 0x496408C: rclcpp::node_interfaces::NodeBase::NodeBase(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::shared_ptr<rclcpp::Context>, rcl_node_options_s const&, bool, bool) (in /opt/ros/humble/lib/librclcpp.so)
==14672==    by 0x495BFD6: rclcpp::Node::Node(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, rclcpp::NodeOptions const&) (in /opt/ros/humble/lib/librclcpp.so)
==14672==    by 0x495D207: rclcpp::Node::Node(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, rclcpp::NodeOptions const&) (in /opt/ros/humble/lib/librclcpp.so)
==14672==    by 0x660B12A: usb_cam::UsbCamNode::UsbCamNode(rclcpp::NodeOptions const&) (in /home/user/workspace/webcam_ws/install/usb_cam/lib/libusb_cam_node.so)
==14672==    by 0x661399D: rclcpp_components::NodeFactoryTemplate<usb_cam::UsbCamNode>::create_node_instance(rclcpp::NodeOptions const&) (in /home/user/workspace/webcam_ws/install/usb_cam/lib/libusb_cam_node.so)
==14672== 
==14672== 151 (64 direct, 87 indirect) bytes in 1 blocks are definitely lost in loss record 373 of 476
==14672==    at 0x4849013: operator new(unsigned long) (in /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==14672==    by 0x50E3A23: ??? (in /opt/ros/humble/lib/librosidl_typesupport_cpp.so)
==14672==    by 0x55FC888: ???
==14672==    by 0x55DA167: ???
==14672==    by 0x55DA366: ???
==14672==    by 0x55DA61F: ???
==14672==    by 0x4F468B2: rcl_publisher_init (in /opt/ros/humble/lib/librcl.so)
==14672==    by 0x49B48AD: rclcpp::PublisherBase::PublisherBase(rclcpp::node_interfaces::NodeBaseInterface*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, rosidl_message_type_support_t const&, rcl_publisher_options_s const&) (in /opt/ros/humble/lib/librclcpp.so)
==14672==    by 0x1B041D0B: ???
==14672==    by 0x1B0423B0: ???
==14672==    by 0x4971F4B: rclcpp::node_interfaces::NodeTopics::create_publisher(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, rclcpp::PublisherFactory const&, rclcpp::QoS const&) (in /opt/ros/humble/lib/librclcpp.so)
==14672==    by 0x1B040C6F: ???
==14672== 
==14672== 296 (192 direct, 104 indirect) bytes in 1 blocks are definitely lost in loss record 410 of 476
==14672==    at 0x4849013: operator new(unsigned long) (in /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==14672==    by 0x55D20E9: ???
==14672==    by 0x55DFEE7: ???
==14672==    by 0x4F4DBF7: rcl_wait_set_init (in /opt/ros/humble/lib/librcl.so)
==14672==    by 0x4944ABD: rclcpp::Executor::Executor(rclcpp::ExecutorOptions const&) (in /opt/ros/humble/lib/librclcpp.so)
==14672==    by 0x494832C: rclcpp::executors::SingleThreadedExecutor::SingleThreadedExecutor(rclcpp::ExecutorOptions const&) (in /opt/ros/humble/lib/librclcpp.so)
==14672==    by 0x10BA1E: main (in /home/user/workspace/webcam_ws/install/usb_cam/lib/usb_cam/usb_cam_node_exe)
==14672== 
==14672== 327 (16 direct, 311 indirect) bytes in 1 blocks are definitely lost in loss record 411 of 476
==14672==    at 0x4849013: operator new(unsigned long) (in /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==14672==    by 0x1AFBEE0C: ???
==14672==    by 0x400647D: call_init.part.0 (dl-init.c:70)
==14672==    by 0x4006567: call_init (dl-init.c:33)
==14672==    by 0x4006567: _dl_init (dl-init.c:117)
==14672==    by 0x4E68C84: _dl_catch_exception (dl-error-skeleton.c:182)
==14672==    by 0x400DFF5: dl_open_worker (dl-open.c:808)
==14672==    by 0x400DFF5: dl_open_worker (dl-open.c:771)
==14672==    by 0x4E68C27: _dl_catch_exception (dl-error-skeleton.c:208)
==14672==    by 0x400E34D: _dl_open (dl-open.c:883)
==14672==    by 0x4D846BB: dlopen_doit (dlopen.c:56)
==14672==    by 0x4E68C27: _dl_catch_exception (dl-error-skeleton.c:208)
==14672==    by 0x4E68CF2: _dl_catch_error (dl-error-skeleton.c:227)
==14672==    by 0x4D841AD: _dlerror_run (dlerror.c:138)
==14672== 
==14672== 329 (16 direct, 313 indirect) bytes in 1 blocks are definitely lost in loss record 412 of 476
==14672==    at 0x4849013: operator new(unsigned long) (in /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==14672==    by 0x1AFBEFF1: ???
==14672==    by 0x400647D: call_init.part.0 (dl-init.c:70)
==14672==    by 0x4006567: call_init (dl-init.c:33)
==14672==    by 0x4006567: _dl_init (dl-init.c:117)
==14672==    by 0x4E68C84: _dl_catch_exception (dl-error-skeleton.c:182)
==14672==    by 0x400DFF5: dl_open_worker (dl-open.c:808)
==14672==    by 0x400DFF5: dl_open_worker (dl-open.c:771)
==14672==    by 0x4E68C27: _dl_catch_exception (dl-error-skeleton.c:208)
==14672==    by 0x400E34D: _dl_open (dl-open.c:883)
==14672==    by 0x4D846BB: dlopen_doit (dlopen.c:56)
==14672==    by 0x4E68C27: _dl_catch_exception (dl-error-skeleton.c:208)
==14672==    by 0x4E68CF2: _dl_catch_error (dl-error-skeleton.c:227)
==14672==    by 0x4D841AD: _dlerror_run (dlerror.c:138)
==14672== 
==14672== 345 (16 direct, 329 indirect) bytes in 1 blocks are definitely lost in loss record 416 of 476
==14672==    at 0x4849013: operator new(unsigned long) (in /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==14672==    by 0x1B03A5AC: ???
==14672==    by 0x400647D: call_init.part.0 (dl-init.c:70)
==14672==    by 0x4006567: call_init (dl-init.c:33)
==14672==    by 0x4006567: _dl_init (dl-init.c:117)
==14672==    by 0x4E68C84: _dl_catch_exception (dl-error-skeleton.c:182)
==14672==    by 0x400DFF5: dl_open_worker (dl-open.c:808)
==14672==    by 0x400DFF5: dl_open_worker (dl-open.c:771)
==14672==    by 0x4E68C27: _dl_catch_exception (dl-error-skeleton.c:208)
==14672==    by 0x400E34D: _dl_open (dl-open.c:883)
==14672==    by 0x4D846BB: dlopen_doit (dlopen.c:56)
==14672==    by 0x4E68C27: _dl_catch_exception (dl-error-skeleton.c:208)
==14672==    by 0x4E68CF2: _dl_catch_error (dl-error-skeleton.c:227)
==14672==    by 0x4D841AD: _dlerror_run (dlerror.c:138)
==14672== 
==14672== 347 (16 direct, 331 indirect) bytes in 1 blocks are definitely lost in loss record 417 of 476
==14672==    at 0x4849013: operator new(unsigned long) (in /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==14672==    by 0x1AF3E1CC: ???
==14672==    by 0x400647D: call_init.part.0 (dl-init.c:70)
==14672==    by 0x4006567: call_init (dl-init.c:33)
==14672==    by 0x4006567: _dl_init (dl-init.c:117)
==14672==    by 0x4E68C84: _dl_catch_exception (dl-error-skeleton.c:182)
==14672==    by 0x400DFF5: dl_open_worker (dl-open.c:808)
==14672==    by 0x400DFF5: dl_open_worker (dl-open.c:771)
==14672==    by 0x4E68C27: _dl_catch_exception (dl-error-skeleton.c:208)
==14672==    by 0x400E34D: _dl_open (dl-open.c:883)
==14672==    by 0x4D846BB: dlopen_doit (dlopen.c:56)
==14672==    by 0x4E68C27: _dl_catch_exception (dl-error-skeleton.c:208)
==14672==    by 0x4E68CF2: _dl_catch_error (dl-error-skeleton.c:227)
==14672==    by 0x4D841AD: _dlerror_run (dlerror.c:138)
==14672== 
==14672== 348 (16 direct, 332 indirect) bytes in 1 blocks are definitely lost in loss record 418 of 476
==14672==    at 0x4849013: operator new(unsigned long) (in /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==14672==    by 0x1B03A791: ???
==14672==    by 0x400647D: call_init.part.0 (dl-init.c:70)
==14672==    by 0x4006567: call_init (dl-init.c:33)
==14672==    by 0x4006567: _dl_init (dl-init.c:117)
==14672==    by 0x4E68C84: _dl_catch_exception (dl-error-skeleton.c:182)
==14672==    by 0x400DFF5: dl_open_worker (dl-open.c:808)
==14672==    by 0x400DFF5: dl_open_worker (dl-open.c:771)
==14672==    by 0x4E68C27: _dl_catch_exception (dl-error-skeleton.c:208)
==14672==    by 0x400E34D: _dl_open (dl-open.c:883)
==14672==    by 0x4D846BB: dlopen_doit (dlopen.c:56)
==14672==    by 0x4E68C27: _dl_catch_exception (dl-error-skeleton.c:208)
==14672==    by 0x4E68CF2: _dl_catch_error (dl-error-skeleton.c:227)
==14672==    by 0x4D841AD: _dlerror_run (dlerror.c:138)
==14672== 
==14672== 350 (16 direct, 334 indirect) bytes in 1 blocks are definitely lost in loss record 419 of 476
==14672==    at 0x4849013: operator new(unsigned long) (in /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==14672==    by 0x1AF3E3B1: ???
==14672==    by 0x400647D: call_init.part.0 (dl-init.c:70)
==14672==    by 0x4006567: call_init (dl-init.c:33)
==14672==    by 0x4006567: _dl_init (dl-init.c:117)
==14672==    by 0x4E68C84: _dl_catch_exception (dl-error-skeleton.c:182)
==14672==    by 0x400DFF5: dl_open_worker (dl-open.c:808)
==14672==    by 0x400DFF5: dl_open_worker (dl-open.c:771)
==14672==    by 0x4E68C27: _dl_catch_exception (dl-error-skeleton.c:208)
==14672==    by 0x400E34D: _dl_open (dl-open.c:883)
==14672==    by 0x4D846BB: dlopen_doit (dlopen.c:56)
==14672==    by 0x4E68C27: _dl_catch_exception (dl-error-skeleton.c:208)
==14672==    by 0x4E68CF2: _dl_catch_error (dl-error-skeleton.c:227)
==14672==    by 0x4D841AD: _dlerror_run (dlerror.c:138)
==14672== 
==14672== 364 (16 direct, 348 indirect) bytes in 1 blocks are definitely lost in loss record 422 of 476
==14672==    at 0x4849013: operator new(unsigned long) (in /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==14672==    by 0x137958DC: ???
==14672==    by 0x400647D: call_init.part.0 (dl-init.c:70)
==14672==    by 0x4006567: call_init (dl-init.c:33)
==14672==    by 0x4006567: _dl_init (dl-init.c:117)
==14672==    by 0x4E68C84: _dl_catch_exception (dl-error-skeleton.c:182)
==14672==    by 0x400DFF5: dl_open_worker (dl-open.c:808)
==14672==    by 0x400DFF5: dl_open_worker (dl-open.c:771)
==14672==    by 0x4E68C27: _dl_catch_exception (dl-error-skeleton.c:208)
==14672==    by 0x400E34D: _dl_open (dl-open.c:883)
==14672==    by 0x4D846BB: dlopen_doit (dlopen.c:56)
==14672==    by 0x4E68C27: _dl_catch_exception (dl-error-skeleton.c:208)
==14672==    by 0x4E68CF2: _dl_catch_error (dl-error-skeleton.c:227)
==14672==    by 0x4D841AD: _dlerror_run (dlerror.c:138)
==14672== 
==14672== 367 (16 direct, 351 indirect) bytes in 1 blocks are definitely lost in loss record 423 of 476
==14672==    at 0x4849013: operator new(unsigned long) (in /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==14672==    by 0x13795AC1: ???
==14672==    by 0x400647D: call_init.part.0 (dl-init.c:70)
==14672==    by 0x4006567: call_init (dl-init.c:33)
==14672==    by 0x4006567: _dl_init (dl-init.c:117)
==14672==    by 0x4E68C84: _dl_catch_exception (dl-error-skeleton.c:182)
==14672==    by 0x400DFF5: dl_open_worker (dl-open.c:808)
==14672==    by 0x400DFF5: dl_open_worker (dl-open.c:771)
==14672==    by 0x4E68C27: _dl_catch_exception (dl-error-skeleton.c:208)
==14672==    by 0x400E34D: _dl_open (dl-open.c:883)
==14672==    by 0x4D846BB: dlopen_doit (dlopen.c:56)
==14672==    by 0x4E68C27: _dl_catch_exception (dl-error-skeleton.c:208)
==14672==    by 0x4E68CF2: _dl_catch_error (dl-error-skeleton.c:227)
==14672==    by 0x4D841AD: _dlerror_run (dlerror.c:138)
==14672== 
==14672== 496 (16 direct, 480 indirect) bytes in 1 blocks are definitely lost in loss record 428 of 476
==14672==    at 0x4849013: operator new(unsigned long) (in /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==14672==    by 0x6610F79: void class_loader::impl::registerPlugin<rclcpp_components::NodeFactoryTemplate<usb_cam::UsbCamNode>, rclcpp_components::NodeFactory>(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) (in /home/user/workspace/webcam_ws/install/usb_cam/lib/libusb_cam_node.so)
==14672==    by 0x660538A: __static_initialization_and_destruction_0(int, int) [clone .constprop.0] (in /home/user/workspace/webcam_ws/install/usb_cam/lib/libusb_cam_node.so)
==14672==    by 0x400647D: call_init.part.0 (dl-init.c:70)
==14672==    by 0x4006567: call_init (dl-init.c:33)
==14672==    by 0x4006567: _dl_init (dl-init.c:117)
==14672==    by 0x4E68C84: _dl_catch_exception (dl-error-skeleton.c:182)
==14672==    by 0x400DFF5: dl_open_worker (dl-open.c:808)
==14672==    by 0x400DFF5: dl_open_worker (dl-open.c:771)
==14672==    by 0x4E68C27: _dl_catch_exception (dl-error-skeleton.c:208)
==14672==    by 0x400E34D: _dl_open (dl-open.c:883)
==14672==    by 0x4D846BB: dlopen_doit (dlopen.c:56)
==14672==    by 0x4E68C27: _dl_catch_exception (dl-error-skeleton.c:208)
==14672==    by 0x4E68CF2: _dl_catch_error (dl-error-skeleton.c:227)
==14672== 
==14672== 608 bytes in 1 blocks are possibly lost in loss record 438 of 476
==14672==    at 0x4848899: malloc (in /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==14672==    by 0x4013FF0: malloc (rtld-malloc.h:56)
==14672==    by 0x4013FF0: _dl_resize_dtv (dl-tls.c:498)
==14672==    by 0x4014C0F: _dl_update_slotinfo (dl-tls.c:812)
==14672==    by 0x4014D0B: update_get_addr (dl-tls.c:919)
==14672==    by 0x401826B: __tls_get_addr (tls_get_addr.S:55)
==14672==    by 0x1980A147: ???
==14672==    by 0x40021A1: call_destructors (dl-close.c:129)
==14672==    by 0x4E68C84: _dl_catch_exception (dl-error-skeleton.c:182)
==14672==    by 0x4002635: _dl_close_worker.part.0.isra.0 (dl-close.c:292)
==14672==    by 0x40032A1: _dl_close_worker (dl-close.c:150)
==14672==    by 0x40032A1: _dl_close (dl-close.c:818)
==14672==    by 0x4E68C27: _dl_catch_exception (dl-error-skeleton.c:208)
==14672==    by 0x4E68CF2: _dl_catch_error (dl-error-skeleton.c:227)
==14672== 
==14672== 2,616 (32 direct, 2,584 indirect) bytes in 1 blocks are definitely lost in loss record 456 of 476
==14672==    at 0x4848899: malloc (in /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==14672==    by 0x56FFBAC: ???
==14672==    by 0x56F860D: ???
==14672==    by 0x56E03D3: ???
==14672==    by 0x56D6A7C: ???
==14672==    by 0x56E5DD1: ???
==14672==    by 0x55DFD81: ???
==14672==    by 0x4F31E05: ??? (in /opt/ros/humble/lib/librcl.so)
==14672==    by 0x49531CF: rclcpp::GuardCondition::GuardCondition(std::shared_ptr<rclcpp::Context>, rcl_guard_condition_options_s) (in /opt/ros/humble/lib/librclcpp.so)
==14672==    by 0x4944617: rclcpp::Executor::Executor(rclcpp::ExecutorOptions const&) (in /opt/ros/humble/lib/librclcpp.so)
==14672==    by 0x494832C: rclcpp::executors::SingleThreadedExecutor::SingleThreadedExecutor(rclcpp::ExecutorOptions const&) (in /opt/ros/humble/lib/librclcpp.so)
==14672==    by 0x10BA1E: main (in /home/user/workspace/webcam_ws/install/usb_cam/lib/usb_cam/usb_cam_node_exe)
==14672== 
==14672== 8,264 bytes in 1 blocks are possibly lost in loss record 464 of 476
==14672==    at 0x4848899: malloc (in /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==14672==    by 0x56FFBAC: ???
==14672==    by 0x56C5A8C: ???
==14672==    by 0x56D6A77: ???
==14672==    by 0x56E5DD1: ???
==14672==    by 0x55DFD81: ???
==14672==    by 0x4F31E05: ??? (in /opt/ros/humble/lib/librcl.so)
==14672==    by 0x49531CF: rclcpp::GuardCondition::GuardCondition(std::shared_ptr<rclcpp::Context>, rcl_guard_condition_options_s) (in /opt/ros/humble/lib/librclcpp.so)
==14672==    by 0x4944617: rclcpp::Executor::Executor(rclcpp::ExecutorOptions const&) (in /opt/ros/humble/lib/librclcpp.so)
==14672==    by 0x494832C: rclcpp::executors::SingleThreadedExecutor::SingleThreadedExecutor(rclcpp::ExecutorOptions const&) (in /opt/ros/humble/lib/librclcpp.so)
==14672==    by 0x10BA1E: main (in /home/user/workspace/webcam_ws/install/usb_cam/lib/usb_cam/usb_cam_node_exe)
==14672== 
==14672== 11,296 bytes in 1 blocks are definitely lost in loss record 466 of 476
==14672==    at 0x4848899: malloc (in /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==14672==    by 0x181AC51B: ???
==14672==    by 0x18183989: ???
==14672==    by 0x15A49C77: ???
==14672==    by 0x15A49D7A: ???
==14672==    by 0x400647D: call_init.part.0 (dl-init.c:70)
==14672==    by 0x4006567: call_init (dl-init.c:33)
==14672==    by 0x4006567: _dl_init (dl-init.c:117)
==14672==    by 0x4E68C84: _dl_catch_exception (dl-error-skeleton.c:182)
==14672==    by 0x400DFF5: dl_open_worker (dl-open.c:808)
==14672==    by 0x400DFF5: dl_open_worker (dl-open.c:771)
==14672==    by 0x4E68C27: _dl_catch_exception (dl-error-skeleton.c:208)
==14672==    by 0x400E34D: _dl_open (dl-open.c:883)
==14672==    by 0x4D846BB: dlopen_doit (dlopen.c:56)
==14672== 
==14672== 38,848 (14,568 direct, 24,280 indirect) bytes in 607 blocks are definitely lost in loss record 472 of 476
==14672==    at 0x484DE30: memalign (in /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==14672==    by 0x484DF92: posix_memalign (in /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==14672==    by 0x7BBC9E4: av_malloc (in /usr/lib/x86_64-linux-gnu/libavutil.so.56.70.100)
==14672==    by 0x7BBCDC1: av_mallocz (in /usr/lib/x86_64-linux-gnu/libavutil.so.56.70.100)
==14672==    by 0x7B99347: av_buffer_create (in /usr/lib/x86_64-linux-gnu/libavutil.so.56.70.100)
==14672==    by 0x693B502: av_packet_from_data (in /usr/lib/x86_64-linux-gnu/libavcodec.so.58.134.100)
==14672==    by 0x6703B98: usb_cam::formats::MJPEG2RGB::convert(char const*&, char*&, int const&) (in /home/user/workspace/webcam_ws/install/usb_cam/lib/libusb_cam.so)
==14672==    by 0x6700434: usb_cam::UsbCam::process_image(char const*, char*&, int const&) (in /home/user/workspace/webcam_ws/install/usb_cam/lib/libusb_cam.so)
==14672==    by 0x67007C0: usb_cam::UsbCam::read_frame() (in /home/user/workspace/webcam_ws/install/usb_cam/lib/libusb_cam.so)
==14672==    by 0x6701852: usb_cam::UsbCam::grab_image() (in /home/user/workspace/webcam_ws/install/usb_cam/lib/libusb_cam.so)
==14672==    by 0x6608A3C: usb_cam::UsbCamNode::take_and_send_image() (in /home/user/workspace/webcam_ws/install/usb_cam/lib/libusb_cam_node.so)
==14672==    by 0x6609117: usb_cam::UsbCamNode::update() (in /home/user/workspace/webcam_ws/install/usb_cam/lib/libusb_cam_node.so)
==14672== 
==14672== 3,791,587 bytes in 16 blocks are possibly lost in loss record 474 of 476
==14672==    at 0x48487A9: malloc (in /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==14672==    by 0x7B998C7: av_buffer_realloc (in /usr/lib/x86_64-linux-gnu/libavutil.so.56.70.100)
==14672==    by 0x693B22D: av_new_packet (in /usr/lib/x86_64-linux-gnu/libavcodec.so.58.134.100)
==14672==    by 0x6703B84: usb_cam::formats::MJPEG2RGB::convert(char const*&, char*&, int const&) (in /home/user/workspace/webcam_ws/install/usb_cam/lib/libusb_cam.so)
==14672==    by 0x6700434: usb_cam::UsbCam::process_image(char const*, char*&, int const&) (in /home/user/workspace/webcam_ws/install/usb_cam/lib/libusb_cam.so)
==14672==    by 0x67007C0: usb_cam::UsbCam::read_frame() (in /home/user/workspace/webcam_ws/install/usb_cam/lib/libusb_cam.so)
==14672==    by 0x6701852: usb_cam::UsbCam::grab_image() (in /home/user/workspace/webcam_ws/install/usb_cam/lib/libusb_cam.so)
==14672==    by 0x6608A3C: usb_cam::UsbCamNode::take_and_send_image() (in /home/user/workspace/webcam_ws/install/usb_cam/lib/libusb_cam_node.so)
==14672==    by 0x6609117: usb_cam::UsbCamNode::update() (in /home/user/workspace/webcam_ws/install/usb_cam/lib/libusb_cam_node.so)
==14672==    by 0x660C894: rclcpp::GenericTimer<std::_Bind<void (usb_cam::UsbCamNode::*(usb_cam::UsbCamNode*))()>, (void*)0>::execute_callback() (in /home/user/workspace/webcam_ws/install/usb_cam/lib/libusb_cam_node.so)
==14672==    by 0x4943650: rclcpp::Executor::execute_any_executable(rclcpp::AnyExecutable&) (in /opt/ros/humble/lib/librclcpp.so)
==14672==    by 0x494AF3F: rclcpp::executors::SingleThreadedExecutor::spin() (in /opt/ros/humble/lib/librclcpp.so)
==14672== 
==14672== 156,904,367 (14,592 direct, 156,889,775 indirect) bytes in 608 blocks are definitely lost in loss record 476 of 476
==14672==    at 0x484DE30: memalign (in /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==14672==    by 0x484DF92: posix_memalign (in /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==14672==    by 0x7BBC9E4: av_malloc (in /usr/lib/x86_64-linux-gnu/libavutil.so.56.70.100)
==14672==    by 0x7BBCDC1: av_mallocz (in /usr/lib/x86_64-linux-gnu/libavutil.so.56.70.100)
==14672==    by 0x7B99347: av_buffer_create (in /usr/lib/x86_64-linux-gnu/libavutil.so.56.70.100)
==14672==    by 0x7B998EB: av_buffer_realloc (in /usr/lib/x86_64-linux-gnu/libavutil.so.56.70.100)
==14672==    by 0x693B22D: av_new_packet (in /usr/lib/x86_64-linux-gnu/libavcodec.so.58.134.100)
==14672==    by 0x6703B84: usb_cam::formats::MJPEG2RGB::convert(char const*&, char*&, int const&) (in /home/user/workspace/webcam_ws/install/usb_cam/lib/libusb_cam.so)
==14672==    by 0x6700434: usb_cam::UsbCam::process_image(char const*, char*&, int const&) (in /home/user/workspace/webcam_ws/install/usb_cam/lib/libusb_cam.so)
==14672==    by 0x67007C0: usb_cam::UsbCam::read_frame() (in /home/user/workspace/webcam_ws/install/usb_cam/lib/libusb_cam.so)
==14672==    by 0x6701852: usb_cam::UsbCam::grab_image() (in /home/user/workspace/webcam_ws/install/usb_cam/lib/libusb_cam.so)
==14672==    by 0x6608A3C: usb_cam::UsbCamNode::take_and_send_image() (in /home/user/workspace/webcam_ws/install/usb_cam/lib/libusb_cam_node.so)
==14672== 
==14672== LEAK SUMMARY:
==14672==    definitely lost: 41,144 bytes in 1,232 blocks
==14672==    indirectly lost: 156,920,038 bytes in 1,868 blocks
==14672==      possibly lost: 3,800,475 bytes in 19 blocks
==14672==    still reachable: 292,048 bytes in 1,084 blocks
==14672==         suppressed: 0 bytes in 0 blocks
==14672== Reachable blocks (those to which a pointer was found) are not shown.
==14672== To see them, rerun with: --leak-check=full --show-leak-kinds=all
==14672== 
==14672== For lists of detected and suppressed errors, rerun with: -s
==14672== ERROR SUMMARY: 23 errors from 23 contexts (suppressed: 0 from 0)

flynneva · 2023-07-25T02:43:32Z

@firesurfer ah you are right! I only tested without subscribing. Let me try again with subscribing and see what else I can address 👍

flynneva · 2023-07-25T03:29:55Z

@firesurfer well I cant seem to get valgrind to work for me to show the same exact memory leaks as you, but I think from your output I'm able to fix the leaks (fingers crossed). I just pushed one more commit to #268 so give it a go and let me know what you get.

Could you share with me how you run valgrind? I've tried the following but I dont see the same memory leaks as you:

 valgrind --leak-check=full \
   --show-leak-kinds=all \
   --track-origins=yes  \
   --verbose \
   --log-file=valgrind-out.txt ./install/usb_cam/lib/usb_cam/usb_cam_node_exe

flynneva · 2023-07-25T03:53:58Z

@firesurfer oh nevermind 😓 I was using yuyv instead of mjpeg. Ignore my stupidity 😅 I see your valgrind output now

flynneva · 2023-07-25T05:43:42Z

@firesurfer I was able to track down most of the issues in the mjpeg2rgb conversion I think. Now that I can reproduce locally though I should be able to address the rest of them over the weekend

flynneva · 2023-07-25T06:21:20Z

@firesurfer actually wasn't able to solve this just yet. Give me a few days and ill track it dow!

firesurfer · 2023-08-14T06:04:00Z

@flynneva hope you are doing fine! Did you manage to dig further into the issue by now ?

Relates to ros-drivers#262 Signed-off-by: Evan Flynn <evanflynn.msu@gmail.com>

flynneva · 2023-08-24T23:10:30Z

@firesurfer still working on it a bit and plan to do more work over the weekend.

I'll keep you posted 👍

flynneva · 2023-08-30T05:08:49Z

@firesurfer small update here: I've narrowed the major leakages down to the av_packet_from_data call. We'll need to come up with an alternative way of initializing the ffmpeg AVPacket, or figuring out what is leaking there.

Sorry this has taken so long...I had a friends wedding last week and things have been pretty busy for me the last few weeks 😅

firesurfer · 2023-08-30T14:22:19Z

@flynneva No worries. Thanks for maintaining this package!

I just took a quick look myself into the code and the FFmpeg documentation.

So if I understand the docs correctly (https://ffmpeg.org/doxygen/2.7/group__lavc__packet.html#ga7ca877e1f0ded89a27199b65e9a077dc) One should only pass a data pointer which has been allocated with av_malloc. Which isn't the case here if I am right.

See also this stackoverflow question:
https://stackoverflow.com/questions/63617241/how-to-load-data-to-an-ffmpeg-avbuffer

flynneva · 2023-08-31T02:21:32Z

@firesurfer I think I got it: bd6bb91

now the only leaks that are left are outside of this package it looks like, and they dont grow over time - which is good

firesurfer · 2023-09-04T12:16:36Z

@flynneva Sounds great. I hope I find some time this week to test it!

flynneva · 2023-09-08T01:46:54Z

@firesurfer just an fyi the update is now on the main ros2 branch, so just use that one for your tests 👍🏼

firesurfer · 2023-09-18T12:49:50Z

@flynneva Just tested the latest ros-iron release and it seems to fix the issue!
Thanks a lot

flynneva · 2023-09-18T13:09:47Z

@firesurfer awesome thanks for confirming! I'll go ahead and close this issue then 👍

flynneva mentioned this issue Jul 15, 2023

Address multiple memory leak issues after ros2 rewrite #264

Merged

flynneva mentioned this issue Jul 23, 2023

262 fix memory leak #268

Merged

flynneva self-assigned this Aug 16, 2023

flynneva added a commit to flynneva/usb_cam that referenced this issue Aug 22, 2023

Fix some minor memorly leaks for mjpeg

da715c9

Relates to ros-drivers#262 Signed-off-by: Evan Flynn <evanflynn.msu@gmail.com>

flynneva closed this as completed Sep 18, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Memory leak - ros2 humble #262

Memory leak - ros2 humble #262

firesurfer commented Jul 12, 2023

flynneva commented Jul 13, 2023

flynneva commented Jul 15, 2023

firesurfer commented Jul 17, 2023

flynneva commented Jul 23, 2023

firesurfer commented Jul 24, 2023 •

edited

Loading

flynneva commented Jul 25, 2023

flynneva commented Jul 25, 2023

flynneva commented Jul 25, 2023 •

edited

Loading

flynneva commented Jul 25, 2023

flynneva commented Jul 25, 2023

firesurfer commented Aug 14, 2023

flynneva commented Aug 24, 2023

flynneva commented Aug 30, 2023

firesurfer commented Aug 30, 2023

flynneva commented Aug 31, 2023 •

edited

Loading

firesurfer commented Sep 4, 2023

flynneva commented Sep 8, 2023

firesurfer commented Sep 18, 2023

flynneva commented Sep 18, 2023

Memory leak - ros2 humble #262

Memory leak - ros2 humble #262

Comments

firesurfer commented Jul 12, 2023

Issue

System

Camera yaml:

Further info

flynneva commented Jul 13, 2023

flynneva commented Jul 15, 2023

firesurfer commented Jul 17, 2023

flynneva commented Jul 23, 2023

firesurfer commented Jul 24, 2023 • edited Loading

flynneva commented Jul 25, 2023

flynneva commented Jul 25, 2023

flynneva commented Jul 25, 2023 • edited Loading

flynneva commented Jul 25, 2023

flynneva commented Jul 25, 2023

firesurfer commented Aug 14, 2023

flynneva commented Aug 24, 2023

flynneva commented Aug 30, 2023

firesurfer commented Aug 30, 2023

flynneva commented Aug 31, 2023 • edited Loading

firesurfer commented Sep 4, 2023

flynneva commented Sep 8, 2023

firesurfer commented Sep 18, 2023

flynneva commented Sep 18, 2023

firesurfer commented Jul 24, 2023 •

edited

Loading

flynneva commented Jul 25, 2023 •

edited

Loading

flynneva commented Aug 31, 2023 •

edited

Loading