enabling single/multi video feed to the lerobot over lan with ip:port

[eDeveloperOZ]

Enable LAN Video Feed Support for OpenCV Implementation

Problem

Currently, the OpenCV implementation in lerobot is limited to accessing built-in cameras (like MacBook's webcam) and connected iOS devices. This limitation prevents users from accessing video feeds from network cameras, restricting the development of distributed robotic setups where cameras need to be placed at different locations on the local network to recored/monitor the robotic device.

Solution

Enhanced the OpenCV implementation to support video feeds over LAN connections, enabling:

• Single or multiple network camera feeds via IP:port configuration
• Seamless integration with existing camera support
• Flexible configuration through both YAML files and command-line interface
• Automated handling of network camera connections and integration with existing camera management system

Key features:

• Support for multiple network cameras with different resolutions/FPS
• Backward compatibility with existing camera functionality
• Easy-to-use command line interface for testing network cameras
• Configurable through standard lerobot YAML configuration files

Implementation Details

• Extended find_cameras() and _find_cameras() to support network camera discovery
• Added URL-based camera initialization in OpenCVCamera class
• Implemented multi-camera support with comma-separated IP:port specification
• Verification of correct commands provided by users with explanatory help
• Extended parser with appropriate flags
• Added validation for network camera connections
• Maintained existing functionality for built-in cameras like -cameras--id etc'

Usage Examples

Command Line Interface

Before (local cameras only):

python lerobot/common/robot_devices/cameras/opencv.py --images-dir outputs/images_from_opencv_cameras

After (with network cameras):

python lerobot/common/robot_devices/cameras/opencv.py --LAN -ip '192.168.1.173,192.168.1.238' -port '8080,8080' --images-dir outputs/images_from_opencv_cameras

YAML Configuration

recored bash commands stays the same
Added network camera support in configuration:

cameras:
  network_camera:
    target: lerobot.common.robot_devices.cameras.opencv.OpenCVCamera
    camera_index: 0
    fps: 25
    width: 1920
    height: 1080
    url: "192.168.1.238:8080"

Testing

Hardware Configurations Tested

• Built-in MacBook camera (backward compatibility)
• Android "IP Webcam" app on multiple smartphones
• Generic IP cameras with HTTP stream support
• Various resolutions: 640x480, 1280x720, 1920x1080
• Different frame rates: 15fps, 25fps, 30fps

Functionality Verification

• Single and multi-camera network configurations
• Backward compatibility with existing camera functionality
• Error handling for invalid network connections
• Command line interface with all flags
• YAML configuration support

Error Handling

The implementation includes error handling for:

• Invalid IP address format: Provides clear error message with correct format example
• Mismatched IP/port counts: Validates number of IPs matches number of ports
• Network timeouts: Handles unreachable cameras gracefully
• Invalid camera configurations: Verifies fps, width, and height settings

IMPORTANT Notes

• Currently tested on MacOS; while implementation should work on Linux/Windows, additional testing would be valuable
• Network cameras must be accessible on the local network, tested on Android with app like "IP webcam" as well as Aliexpress bought cameras that provide ip:port for their feed
• Users should ensure proper network permissions and firewall settings