Some V4L2 settings are missing/broken compared to the available MMAL settings

[dgsiegel]

Describe the bug

I would expect that all raspistill (which uses MMAL directly) would work for the V4L2 interface as well: https://www.raspberrypi.org/documentation/raspbian/applications/camera.md

To reproduce
Compare https://www.raspberrypi.org/documentation/raspbian/applications/camera.md with v4l2-ctl -L

Expected behaviour
Ideally all MMAL settings would be exposed to V4L2 as well

Actual behaviour

Compared to MMAL/raspistill/raspivid, v4l2-ctl is missing the following settings:

Exposure mode (scene_mode)

Only 3 values available:

• 0: None
• 8: Night
• 11: Sports

Whereas raspistill offers:

• auto: use automatic exposure mode
• night: select setting for night shooting
• nightpreview:
• backlight: select setting for backlit subject
• spotlight:
• sports: select setting for sports (fast shutter etc.)
• snow: select setting optimised for snowy scenery
• beach: select setting optimised for beach
• verylong: select setting for long exposures
• fixedfps: constrain fps to a fixed value
• antishake: antishake mode
• fireworks: select setting optimised for fireworks

Color effects (color_effects)

The following effects don't work:

• 4: Emboss
• 5: Sketch
• 6: Sky Blue (pastel)
• 7: Grass Green (watercolour)
• 9: Vivid (saturation)
• 11: Art Freeze (hatch)
• 12: Silhouette

The following effects are missing:

• posterise: posterise the image
• whiteboard: whiteboard effect
• blackboard: blackboard effect
• denoise: denoise the image
• hatch: hatch sketch effect
• gpen: graphite sketch effect
• watercolour: watercolour effect
• blur: blur the image

Colour effect (color_effects_cbcr)

Does not seem to have any effect

JPEG Quality (compression_quality)

Does not have any effect on MJPEG stream

Video stabilisation (image_stabilization)

Does not seem to have any effect

Metering (exposure_metering_mode)

Matrix metering is missing

Exposure mode (auto_exposure)

The following setting are missing:

• night: select setting for night shooting
• nightpreview:
• backlight: select setting for backlit subject
• spotlight:
• sports: select setting for sports (fast shutter etc.)
• snow: select setting optimised for snowy scenery
• beach: select setting optimised for beach
• verylong: select setting for long exposures
• fixedfps: constrain fps to a fixed value
• antishake: antishake mode
• fireworks: select setting optimised for fireworks

Shutter speed (exposure_time_absolute)

Doesn't seem to have any effect above 800

Dynamic range compression

Does not exist in V4L2

Analog gain

Does not exist in V4L2

Digital gain

Does not exist in V4L2

Sensor mode

Does not exist in V4L2. Could be interesting for higher resolutions than 1080p with 2x2 binning

System

Affects all systems with 5.4.73 or later and the Raspberry HQ Camera.

The following kernel modules are affected:

• bcm2835_v4l2
• bcm2835_mmal_vchiq

[6by9]

In a large number of those cases "Does not exist in V4L2" is exactly right.

V4L2 is a standard Linux API with documentation at https://www.kernel.org/doc/html/latest/userspace-api/media/v4l/v4l2.html. Many of the controls from raspistill do not exist within the V4L2 API.

Whilst it is possible to add custom controls, doing so is
a) platform specific
b) difficult to do in a stable way

All the CID (Control ID) values are defined in https://elixir.bootlin.com/linux/latest/source/include/uapi/linux/v4l2-controls.h#L56. Add any downstream and there's a high chance that something else adds controls upstream and now shares your CID values.

Note:

• Exposure mode and scene_mode are not synonymous. Scene mode sets up exposure mode and metering mode
• V4L2_CID_COLORFX_CBCR requires that you set V4L2_CID_COLORFX to V4L2_COLORFX_SET_CBCR first.
• Bitrate control for video codecs (H264 and MJPEG) are set via the bitrate (V4L2_CID_MPEG_VIDEO_BITRATE), not by setting a quality.

[dgsiegel]

Thanks, this makes sense! What would be the best way however to expose all MMAL settings to the V4L2 API?

It would be very helpful for https://github.com/showmewebcam/showmewebcam which transforms the Pi into a UVC webcam.

[lategoodbye]

At least matrix metering should be available since Linux 5.8:
93a8883#diff-d501fb5d054c24c31bf3a2ee9869f60877469d9ba611599eab3580d04af26419

[lategoodbye]

Here is a patch against Linux 5.10 to add more scene modes (not sure about the right settings):
https://gist.github.com/lategoodbye/f10e4cce20406bea901b168f244be916

[Malvineous]

I recently opened #4715 to add the sensor mode as a V4L2 custom control, but further discussion is needed so I'm going to reuse this issue as it covers the same ground.

I am quite keen to at least get support for changing the sensor mode implemented in V4L2, because using the Pi as a security camera this gives a much sharper image and is worth it despite the lower framerate.

In the PR, @6by9 was not keen on using V4L2 custom controls, so I would like to discuss what alternatives are available. For example I could add a new module option to set the value, or co-opt an unrelated V4L2 standard control instead. Neither of these seem that great to me, but I'm not entirely clear on the drawback of V4L2 custom controls.

6by9 pointed me to #4440, and according to that it is somewhat of a problem to keep BCM2835_ISP_BASE in sync between the Pi fork of the kernel and the upstream one. However my understanding is that this is already the situation, and this value now needs to be kept in sync whether we add more V4L2 custom controls or not, assuming we base them off the same constant. So if I am understanding correctly, it won't introduce any additional maintenance effort beyond what is already there.

Am I misunderstanding this? Is there more beyond this single definition that needs to be kept in sync? I am a little confused as the discussion in that issue makes it sound like libcamera is shipping its own version of v4l2-controls.h instead of using the version that matches whatever kernel is running on the system (typically located in /usr/include/linux/v4l2-controls.h and copied here from the kernel source code as part of the kernel installation process). If this is the case, libcamera will always risk using the wrong constant if it's built against a different v4l2-controls.h than the kernel was. I must be misunderstanding something, because if libcamera is using /usr/include/linux/v4l2-controls.h then it will always pick up the correct constant so #4440 wouldn't be an issue.

I am also wondering why this value needs to be kept in sync between the Pi fork and upstream. Is it possible to submit a preliminary upstream PR to allocate this constant permanently, and then just use the assigned value from that point on? Wouldn't this remove the hassle of having to always keep updating the value in the Pi fork every time upstream gets a new custom control allocation?

Sorry for all the questions, I'm just trying to understand the issues in the hope that there is some way to work around them!