[Enhancement] Simplify multiple webcam handling and number of mjpg-streamer instances

[SilverWingedHawk]

mjpg-streamer is able to handle multiple input and/or output streams (by specifying multiple -i or -o flags). In order to reduce the overhead created by spawning multiple instances of mjpg-streamer, as well as simplifying the reverse proxy configuration (either HAProxy or other reverse proxy software), I've modified the webcamd script to be able to allow for multiple input streams.

If multiple input streams are available, a setting in octopi.txt or any supplementary *.txt config file is introduced, which will cap the maximum input streams to 2 so that the pi doesn't suffer too much strain. The default value is now 2, but that can be reduced or increased, I just thought it would be a sensible default.

Also, this modification removes the limitation of offering either USB or Raspicam support, both should be available with this modification. The script is now able to add multiple stream input automatically, not limited to the first /dev/video device encountered. It also relies on the v4l2-ctl command (available through the v4l-utils package) to determine whether a /dev/video* device offers Video Capture capabilities and so can be used as an input stream for mjpg-streamer.

Unfortunately, I don't own an Rpi device nor a Raspicam, so I'm not able to test those functionalities on real hardware, though the script runs the appropriate commands in a similar Linux environment.

This would actually solve #677.

I welcome any feedback you might have regarding this PR.

Best regards

[guysoft]

Nice! have you tested it?
If not what got you to write this?

[SilverWingedHawk]

Unfortunately I haven't tested it on real hardware as stated in my message, I don't own a Raspberry Pi, nor a RaspiCam. I've just tested that the script calls the correct commands with different situations I'm able to reproduce on a Linux (Fedora) machine.

What prompted this PR was that I'd seen my father struggle with his own Rpi setting up multiple webcams months ago. He switched to a full blown computer with Fedora, and I installed OctoPrint on it, and fiddled with mjpg-streamer a bit. Going to the forums I saw that multicam support on OctoPi was a bit hard for newcomers to set up, and I thought that I could write something up to ease the configuration of multiple webcams, hence this PR

[cp2004]

This would make my guide https://community.octoprint.org/t/setting-up-multiple-webcams-in-octopi-the-right-way/32669 easier, as we could skip the haproxy step, one less thing to break for sure!

I will try and test/review this at some point, but I'm not a script-person so it would mostly be from a potential user's point of view.

[guysoft]

@cp2004 Any chance you could test it before I merge?
Or alternatively , i can merge and you can test the nightly?

I just dont want to ship it unless someone tests it

[cp2004]

I can try and give it a go over this weekend.

[cp2004]

Sorry, haven't been able to test this yet. Slightly forgot about it and made a new OctoPrint plugin instead. Will get on to it!

[cp2004]

Hmm, this doesn't entirely work on my Pi. Currently, it has no webcams plugged in, and with and without nb_max_camera=2 set in the config. I have two config files setup, as I would normally for multiple webcams.

webcamd output

Starting up webcamDaemon...

--- Configuration: ----------------------------
cfg_file:      /boot/octopi.txt
camera:        usb
usb options:   -r 640x480 -f 10 -d /dev/video0
raspi options: -fps 10
http options:  -w ./www-octopi -n --listen 127.0.0.1

Explicitly set USB device: /dev/video0
-----------------------------------------------

cfg_file:      /boot/octopi.conf.d/camera1.txt
camera:        auto
usb options:   -r 640x480 -f 10
raspi options: -fps 10
http options:  -w ./www-octopi -n -p 8081

Explicitly set USB device:
-----------------------------------------------

Found video devices:
/dev/video10
/dev/video11
/dev/video12
/dev/video13
/dev/video14
/dev/video15
/dev/video16
config file='/boot/octopi.conf.d/camera1.txt':USB device was not set in options, adding input stream to MJPG-streamer with found video device: /dev/video10
<13>Aug 10 21:55:53 pi: Starting USB webcam
config file='/boot/octopi.conf.d/camera1.txt':USB device was not set in options, adding input stream to MJPG-streamer with found video device: /dev/video11
<13>Aug 10 21:55:53 pi: Starting USB webcam
Cannot add /dev/video12 to input streams as it does not offer video capture capability
Cannot add /dev/video13 to input streams as it does not offer video capture capability
Cannot add /dev/video14 to input streams as it does not offer video capture capability
Cannot add /dev/video15 to input streams as it does not offer video capture capability
Cannot add /dev/video16 to input streams as it does not offer video capture capability
Checking for VL805 (Raspberry Pi 4)...
  - It seems that you don't have VL805 (Raspberry Pi 4).
    There should be no problems with USB (a.k.a. select() timeout)
Running ./mjpg_streamer -o output_http.so -w ./www-octopi -n -p 8081 -i 'input_uvc.so -r 640x480 -f 10 -d /dev/video10' -i 'input_uvc.so -r 640x480 -f 10 -d /dev/video11'
MJPG Streamer Version: git rev: 5554f42c352ecfa7edaec6fc51e507afce605a34
ERROR: could not find input plugin
       Perhaps you want to adjust the search path with:
       # export LD_LIBRARY_PATH=/path/to/plugin/folder
       dlopen:  'input_uvc.so -r 640x480 -f 10 -d /dev/video10' -i 'input_uvc.so -r 640x480 -f 10 -d /dev/video11' : cannot open shared object file: No such file or directory

As you can see in the final line, it seems to pick up both /dev/video10 and /dev/video11 - those are always there on most Pis, so this issue impacts everyone. I currently don't recommend merging this PR as it is, as I have a few issues with the way it works. Basically, I don't entirely understand the logic so I think it will be hard to explain to the majority of OctoPi users as well.

• The nb_max_webcam setting conflicts with the idea of creating multiple files in /boot/octopi.conf.d for each webcam. I would like to define multiple webcams manually (by device ID) but this doesn't seem to be possible with this method - if I explicitly set usb options as -r 640x480 -f 10 -d /dev/video0, it is ignored.
• The auto detecting video device code doesn't work. On my Pi, I have 7 /dev/video* devices, and no cameras connected - but it picks up 2 devices to add to mjpg streamer (video10 & video11). Mjpg streamer crashes, as it does when it can't open a camera.
• OctoPi should not try and run more than one camera at once - the default should be set to 1.

[SilverWingedHawk]

Thanks a lot for the test feedback.

• As for the fact that it ignores the device provided, it shouldn't. So I will look deeper into this so that it does not.

• In order to be able to filter out /dev/video10 and /dev/video11 devices that are always present on a Pi, could you please provide the output of the following commands v4l2-ctl --list-formats -d /dev/video10 and v4l2-ctl --list-formats -d /dev/video11? There's always the possibility of "blacklisting" video10 and video11 based on their name by default, but I'd rather have another way of doing this, so that should it change with future releases of the Pi, the script wouldn't need modifications.

• As for the default, I have no issue whatsoever reducing it to one.

[cp2004]

Adding multiple webcams to a single mjpg-streamer process does and doesn't make sense, here's what I thought of.

For:

• It means that users don't have to edit haproxy configuration if they want /webcam/ rather than ports. This is a big plus for usability.

Against:

• Are we sacrificing multi-core support, by putting all webcams in one process?
• Does mjpg streamer stop all cameras when one of the cameras goes down?
• This PR doesn't currently work as it is.

Might also need clarification where camera_http_options comes in. It's confusing how adding multiple USB options works in multiple files, but if it should all be merged into a single process then we only have one camera_http_options.

[cp2004]

v4l2-ctl --list-formats -d /dev/video10

ioctl: VIDIOC_ENUM_FMT
        Type: Video Capture Multiplanar

        [0]: 'YU12' (Planar YUV 4:2:0)
        [1]: 'YV12' (Planar YVU 4:2:0)
        [2]: 'NV12' (Y/CbCr 4:2:0)
        [3]: 'NV21' (Y/CrCb 4:2:0)
        [4]: 'RGBP' (16-bit RGB 5-6-5)

v4l2-ctl --list-formats -d /dev/video11

ioctl: VIDIOC_ENUM_FMT
        Type: Video Capture Multiplanar

        [0]: 'H264' (H.264, compressed)
        [1]: 'MJPG' (Motion-JPEG, compressed)

[SilverWingedHawk]

Thanks for the command's output. As for the current logic of the script, it's expected behavior those devices get added to the list of input streams... I'm relying on this command to output something which contains more than 3 lines (v4l2-ctl --list-formats still outputs the ioctl lines whether the device has video capture or not). But I can't even filter out based on MJPG format, as first, this might falsely ignore a valid camera not offering this kind of stream, second, it wouldn't filter out /dev/video11... So I'd have to resort to hard blacklisting both of those devices, hoping they never change of /dev/video node... Are there udev rules that are in place to ensure this ?

Concerning the multi-core support, a cursory look at the mjpg-streamer source code shows the use of pthreads so it would stand to reason that we're not sacrificing multi-core support, but I'd like a second look to be sure.

As for the event of one camera stopping working, would it stop the entire streaming process, my guess would be no, as both streams are presented by differents URLs... but I'd rather have a real test confirming that.

As for the camera_http_options, yes, it stands to reason that it should only appear once.

[cp2004]

I have no idea of the first part - it's out of my knowledge of how this stuff works.

Concerning the multi-core support, a cursory look at the mjpg-streamer source code shows the use of pthreads so it would stand to reason that we're not sacrificing multi-core support, but I'd like a second look to be sure.

That's definitely good to hear, 👍

As for the event of one camera stopping working, would it stop the entire streaming process, my guess would be no, as both streams are presented by differents URLs... but I'd rather have a real test confirming that.

I don't currently have the cameras wired up to test that... but I know the process ends when one camera stops streaming, if you are just streaming one camera.

[SilverWingedHawk]

Could you be so kind as to provide the output of udevadm info -a /dev/video10 and udevadm info -a /dev/video11 ? Be aware, the output of those commands can be rather lengthy...

Looking deeper in the source code of mjpg-streamer, especially in input_uvc.c file, line 494, the input_run function which is responsible for starting the related input stream makes use of pthread_create and pthread_detach so the multi-core support would still be there.

It also seems that each camera stream can be stopped at will, at least from the source code point of view... So perhaps the removal of one webcam wouldn't impact the streaming process... But as said earlier, a test in real conditions is the only proof I'd find acceptable

[cp2004]

1st one

(oprint) pi@octopi:~/mjpg-streamer $ udevadm info -a /dev/video10

Udevadm info starts with the device specified by the devpath and then
walks up the chain of parent devices. It prints for every device
found, all possible attributes in the udev rules key format.
A rule to match, can be composed by the attributes of the device
and the attributes from one single parent device.

  looking at device '/devices/platform/soc/fe00b840.mailbox/bcm2835-codec/video4linux/video10':
    KERNEL=="video10"
    SUBSYSTEM=="video4linux"
    DRIVER==""
    ATTR{dev_debug}=="0"
    ATTR{index}=="0"
    ATTR{name}=="bcm2835-codec-decode"

  looking at parent device '/devices/platform/soc/fe00b840.mailbox/bcm2835-codec':
    KERNELS=="bcm2835-codec"
    SUBSYSTEMS=="platform"
    DRIVERS=="bcm2835-codec"
    ATTRS{driver_override}=="(null)"

  looking at parent device '/devices/platform/soc/fe00b840.mailbox':
    KERNELS=="fe00b840.mailbox"
    SUBSYSTEMS=="platform"
    DRIVERS=="bcm2835_vchiq"
    ATTRS{driver_override}=="(null)"

  looking at parent device '/devices/platform/soc':
    KERNELS=="soc"
    SUBSYSTEMS=="platform"
    DRIVERS==""
    ATTRS{driver_override}=="(null)"

  looking at parent device '/devices/platform':
    KERNELS=="platform"
    SUBSYSTEMS==""
    DRIVERS==""

2nd one:

(oprint) pi@octopi:~/mjpg-streamer $ udevadm info -a /dev/video11

Udevadm info starts with the device specified by the devpath and then
walks up the chain of parent devices. It prints for every device
found, all possible attributes in the udev rules key format.
A rule to match, can be composed by the attributes of the device
and the attributes from one single parent device.

  looking at device '/devices/platform/soc/fe00b840.mailbox/bcm2835-codec/video4linux/video11':
    KERNEL=="video11"
    SUBSYSTEM=="video4linux"
    DRIVER==""
    ATTR{name}=="bcm2835-codec-encode"
    ATTR{index}=="0"
    ATTR{dev_debug}=="0"

  looking at parent device '/devices/platform/soc/fe00b840.mailbox/bcm2835-codec':
    KERNELS=="bcm2835-codec"
    SUBSYSTEMS=="platform"
    DRIVERS=="bcm2835-codec"
    ATTRS{driver_override}=="(null)"

  looking at parent device '/devices/platform/soc/fe00b840.mailbox':
    KERNELS=="fe00b840.mailbox"
    SUBSYSTEMS=="platform"
    DRIVERS=="bcm2835_vchiq"
    ATTRS{driver_override}=="(null)"

  looking at parent device '/devices/platform/soc':
    KERNELS=="soc"
    SUBSYSTEMS=="platform"
    DRIVERS==""
    ATTRS{driver_override}=="(null)"

  looking at parent device '/devices/platform':
    KERNELS=="platform"
    SUBSYSTEMS==""
    DRIVERS==""

[SilverWingedHawk]

This is getting somewhere, I hadn't thought that the hardware encoder/decoder would be presented as a video device... Well... sorry to ask again, but what is the output of v4l2-ctl -D -d /dev/video10 and v4l2-ctl -D -d /dev/video11. I do hope they offer some capabilty that a camera doesn't or vice-versa...

[cp2004]

(oprint) pi@octopi:~/mjpg-streamer $ v4l2-ctl -D -d /dev/video10
Driver Info:
        Driver name      : bcm2835-codec
        Card type        : bcm2835-codec-decode
        Bus info         : platform:bcm2835-codec
        Driver version   : 5.10.17
        Capabilities     : 0x84204000
                Video Memory-to-Memory Multiplanar
                Streaming
                Extended Pix Format
                Device Capabilities
        Device Caps      : 0x04204000
                Video Memory-to-Memory Multiplanar
                Streaming
                Extended Pix Format
Media Driver Info:
        Driver name      : bcm2835-codec
        Model            : bcm2835-codec
        Serial           : 0000
        Bus info         : platform:bcm2835-codec
        Media version    : 5.10.17
        Hardware revision: 0x00000001 (1)
        Driver version   : 5.10.17
Interface Info:
        ID               : 0x0300000c
        Type             : V4L Video
Entity Info:
        ID               : 0x00000001 (1)
        Name             : bcm2835-codec-decode-source
        Function         : V4L2 I/O
        Pad 0x01000002   : 0: Source
          Link 0x02000008: to remote pad 0x1000004 of entity 'bcm2835-codec-decode-proc': Data, Enabled, Immutable

(oprint) pi@octopi:~/mjpg-streamer $ v4l2-ctl -D -d /dev/video11
Driver Info:
        Driver name      : bcm2835-codec
        Card type        : bcm2835-codec-encode
        Bus info         : platform:bcm2835-codec
        Driver version   : 5.10.17
        Capabilities     : 0x84204000
                Video Memory-to-Memory Multiplanar
                Streaming
                Extended Pix Format
                Device Capabilities
        Device Caps      : 0x04204000
                Video Memory-to-Memory Multiplanar
                Streaming
                Extended Pix Format
Media Driver Info:
        Driver name      : bcm2835-codec
        Model            : bcm2835-codec
        Serial           : 0000
        Bus info         : platform:bcm2835-codec
        Media version    : 5.10.17
        Hardware revision: 0x00000001 (1)
        Driver version   : 5.10.17
Interface Info:
        ID               : 0x0300001a
        Type             : V4L Video
Entity Info:
        ID               : 0x0000000f (15)
        Name             : bcm2835-codec-encode-source
        Function         : V4L2 I/O
        Pad 0x01000010   : 0: Source
          Link 0x02000016: to remote pad 0x1000012 of entity 'bcm2835-codec-encode-proc': Data, Enabled, Immutable

[SilverWingedHawk]

OK, this is something that I can work with to filter both of this devices... The Video Memory-to-Memory Multiplanar capability seems to be specific to both these devices, and is not something that appears on a regular webcam. I just have to find a way to query for that capability specifically, or craft a regexp as a last resort.

The nb_max_webcam setting conflicts with the idea of creating multiple files in /boot/octopi.conf.d for each webcam. I would like to define multiple webcams manually (by device ID) but this doesn't seem to be possible with this method - if I explicitly set usb options as -r 640x480 -f 10 -d /dev/video0, it is ignored.

Looking back at the script, that's a behavior I didn't change... The /dev/video0 node doesn't exist as evidenced by the list of found video devices. Rather early in the script, checks are in place to find the real device node behind a device path. As yours is inexistent, the script simply doesn't process it. This has nothing to do with code introduced by this PR... At least in these conditions.

I'll try to commit changes to assess what has been discussed here, but I might not be able to do it before next week.

Thanks again @cp2004 for the help provided, and for taking the time to test this.

[cp2004]

Looking back at the script, that's a behavior I didn't change... The /dev/video0 node doesn't exist as evidenced by the list of found video devices. Rather early in the script, checks are in place to find the real device node behind a device path. As yours is inexistent, the script simply doesn't process it. This has nothing to do with code introduced by this PR... At least in these conditions.

OK, that makes sense - I should really have tested it with something that actually existed! Seems like you've got a good idea of how we can get this into shape & ready to merge. Let me know if you need any more info.

[SilverWingedHawk]

OK, that makes sense - I should really have tested it with something that actually existed! Seems like you've got a good idea of how we can get this into shape & ready to merge. Let me know if you need any more info.

As an improvement, a message stating that the device doesn't exist can be added to the script. I can do it in this PR, but it does seem that it would need its own PR as it is not related to the problem this PR is trying to solve...

[SilverWingedHawk]

@cp2004, sorry to bother you again, but could you please provide the output of the following commands:

• v4l2-ctl -I -d /dev/video10
• v4l2-ctl -I -d /dev/video11
• v4l2-ctl -O -d /dev/video10
• v4l2-ctl -O -d /dev/video11
• v4l2-ctl -P -d /dev/video10
• v4l2-ctl -P -d /dev/video11

They might be easier to parse than v4l2-ctl -D -d /dev/video(10|11)

[foosel]

I saw that a review from me was requested here, however I can't do a proper one at the moment. A quick look over the code however raised my suspicions regarding this possibly running into issues with the encoding devices present on all Pis (which btw also caused the issues I recently fixed with the endlessly looping webcamd process), which @cp2004 confirmed.

[SilverWingedHawk]

Hi everyone,

I have changed the logic behind device detection. It now relies on a regex matching for the output of v4l2-ctl -D -d <device>, matching the capabilities hexadecimal representation which should end with a 1 if the device offer video capture, which the hardware encoders do not, as per this previous comment, while webcams should offer this capability.

I also took the liberty to introduce a message telling if a device isn't found, instead of silently skipping the non-existent device, so that configuration errors may be easier to find.

Please feel free to review these new changes and test them.

[SilverWingedHawk]

Hi,

Has anyone been able to review of test the change I made ?

[guysoft]

Hey, sorry for this hanging, didn't get to it, and no one else seems to be testing it. I will try find time but I am really swamped with other stuff.
It would be really helpful if anyone can test this

[guysoft]

Hey,
So I built flashed and tested it here on my printer.

For an initial test I started it with my pi camera that works out of the box when I flash mainstream.
The camera does not show up and I get this:

output of ls /dev/video* is:

/dev/video0  /dev/video10  /dev/video11  /dev/video12  /dev/video13  /dev/video14  /dev/video15  /dev/video16

raspistill -o img.jpg works so the camera is indeed working on this build.

dmesg output:

[   13.209037] bcm2835-codec bcm2835-codec: Device registered as /dev/video10
[   13.209111] bcm2835-codec bcm2835-codec: Loaded V4L2 decode
[   13.209200] bcm2835-isp bcm2835-isp: Device node output[0] registered as /dev/video13
[   13.210094] bcm2835-isp bcm2835-isp: Device node capture[0] registered as /dev/video14
[   13.210918] bcm2835-isp bcm2835-isp: Device node capture[1] registered as /dev/video15
[   13.211533] bcm2835-isp bcm2835-isp: Device node stats[2] registered as /dev/video16
[   13.211570] bcm2835-isp bcm2835-isp: Register output node 0 with media controller
[   13.211603] bcm2835-isp bcm2835-isp: Register capture node 1 with media controller
[   13.211730] bcm2835-isp bcm2835-isp: Register capture node 2 with media controller
[   13.211831] bcm2835-isp bcm2835-isp: Register capture node 3 with media controller
[   13.212195] bcm2835-isp bcm2835-isp: Loaded V4L2 bcm2835-isp
[   13.219793] bcm2835-codec bcm2835-codec: Device registered as /dev/video11
[   13.219866] bcm2835-codec bcm2835-codec: Loaded V4L2 encode
[   13.226297] bcm2835-codec bcm2835-codec: Device registered as /dev/video12
[   13.226396] bcm2835-codec bcm2835-codec: Loaded V4L2 isp
[   13.241534] bcm2835-v4l2-0: scene mode selected 0, was 0
[   13.246327] bcm2835-v4l2-0: V4L2 device registered as video0 - stills mode > 1280x720
[   13.251686] bcm2835-v4l2-0: Broadcom 2835 MMAL video capture ver 0.0.2 loaded.
[   13.360764] brcmfmac: brcmf_fw_alloc_request: using brcm/brcmfmac43455-sdio for chip BCM4345/6
[   13.373812] brcmfmac: brcmf_c_preinit_dcmds: Firmware: BCM4345/6 wl0: Jan  4 2021 19:56:29 version 7.45.229 (617f1f5 CY) FWID 01-2dbd9d2e                                                                                                            

[SilverWingedHawk]

Hi, thanks for the test.

In order to see what went wrong, I would also need the contents of the /var/log/webcamd.log file, as the contents you've provided, although helpful, do not point me to what went wrong with the mjpg_streamer invocation.

[guysoft]

Hey, sorry I didn't get back to you with a log, life got in the way. I think that month in particular I was a week away.
Has there been any other progress? Since I expect we will have an RC soon.