UVC Video encoder (zephyr,videoenc) + H264 support + STM32 STM32N6 conf files #97425
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
zephyrbot
added
platform: STM32
zephyrbot
requested review from
FRASTM,
GeorgeCGV,
cfriedt,
djiatsaf-st,
erwango,
etienne-lms,
gautierg-st,
jfischer-no,
jhedberg,
josuah,
loicpoulain,
marwaiehm-st,
mathieuchopstm,
ngphibang and
tmon-nordic
|
Moving assignee to Video subsystem maintainer, as it seems more appropriate to the content of the RFC |
|
Adding a DNM flag for PRs dependencies. |
josuah
previously approved these changes
Oct 14, 2025
avolmat-st
force-pushed
the
pr-uvc-videoenc-support
branch
from
75eec12 to
3edf0b3
Compare
|
Had to push a new version to correct an issue in the |
josuah
previously approved these changes
Oct 23, 2025
There was a problem hiding this comment.
This looks like nothing but it saves a lot of variable definition in the samples 👍
jfischer-no
reviewed
Oct 23, 2025
| uint32_t dwFrameInterval[CONFIG_USBD_VIDEO_MAX_FRMIVAL]; | ||
| } __packed; | ||
|
|
||
| struct uvc_frame_based_continuous_descriptor { |
There was a problem hiding this comment.
@josuah IMO uvc_frame_based_continuous_descriptor and uvc_frame_based_discrete_descriptor are a bit too long names.
There was a problem hiding this comment.
Maybe the _based_ part can be left out? uvc_frame_continuous_descriptor + uvc_frame_discrete_descriptor should be non-ambiguous:
It could fit well in the list of descriptors:
struct uvc_format_uncomp_descriptor fmt_uncomp;
struct uvc_format_mjpeg_descriptor fmt_mjpeg;
struct uvc_format_frame_descriptor fmt_frame;
struct uvc_frame_descriptor frm;There was a problem hiding this comment.
btw, how Linux does it? I cannot find them in https://github.com/torvalds/linux/blob/master/include/uapi/linux/usb/video.h.
added 2 commits
October 24, 2025 09:24
Replace video_dequeue / video_enqueue buffer exchange code by video_transfer_buffer helper function. Signed-off-by: Alain Volmat <alain.volmat@foss.st.com>
In preparation for the introduction of video encoder support add an indirection for handling of the buffers of the UVC source device. Currently this is only video_dev however it can also be an encoder device when encoder is introduced between video capture device and the UVC device. Signed-off-by: Alain Volmat <alain.volmat@foss.st.com>
avolmat-st
force-pushed
the
pr-uvc-videoenc-support
branch
from
1ab716d to
d0462ef
Compare
erwango
previously approved these changes
Oct 24, 2025
avolmat-st
force-pushed
the
pr-uvc-videoenc-support
branch
from
d0462ef to
d3a72d3
Compare
jfischer-no
reviewed
Oct 24, 2025
added 8 commits
October 24, 2025 11:18
Allow creating a pipeline as follow camera receiver -> encoder -> uvc If the chosen zephyr,videoenc is available, the sample will pipe the camera receiver to the encoder and then the UVC device instead of directly the camera receiver to the UVC. Current implementation has several points hardcoded for the time being: 1. intermediate pixel format between the camera receiver and encoder is set to NV12. This shouldn't be hardcoded and should instead be discovered as a commonly capable format from the encoder / video dev 2. it is considered that encoder device do NOT perform any resolution change and that encoder output resolution is directly based on the camera receiver resolution. Thanks to this, UVC exposed formats are thus the encoder output pixel format & camera receiver resolutions. Signed-off-by: Alain Volmat <alain.volmat@foss.st.com>
Add rough estimate of a worth case H264 output size. The video_estimate_fmt_size would need more information such as quality, profile in order to give a better estimate for each formats so for the time being just stick to 16bpp based size, same as for JPEG. Signed-off-by: Alain Volmat <alain.volmat@foss.st.com>
This commit prepares introduction of the UVC Frame Based support by using the struct uvc_frame_descriptor as parameter of most of the UVC functions. struct uvc_frame_descriptor contains the common fields for all supported frame type and then depending on the DescriptorSubtype the pointer is casted in the correct struct definition. Signed-off-by: Alain Volmat <alain.volmat@foss.st.com>
The frame_based descriptors differ from the frame descriptors in that there is no dwMaxVideoFrameBufferSize field. In order to do that, add a new uvc_frame_based_discrete_descriptor structure to be used to fill in proper information into the frame descriptor. In addition to that, a new format descriptor is also added for frame based transfer. Signed-off-by: Alain Volmat <alain.volmat@foss.st.com>
Add proper check of the return value of video_enqueue / video_dequeue. Signed-off-by: Alain Volmat <alain.volmat@foss.st.com>
Add overlay files in order to enable usage of
the encoder in the UVC sample.
This work with platform defining node label
zephyr_jpegenc
zephyr_h264enc
Mode can be selected by using -DFILE_SUFFIX="jpegenc" or
-DFILE_SUFFIX="h264enc" when building the sample while
also adding -DCONFIG_VIDEO_ENCODER_JPEG or
-DCONFIG_VIDEO_ENCODER_H264 as well in the command line.
Signed-off-by: Alain Volmat <alain.volmat@foss.st.com>
Add zephyr_h264enc and zephyr_jpegenc labels on node in order to be able to use VENC and JPEG codec from samples. Signed-off-by: Alain Volmat <alain.volmat@foss.st.com>
Add entries in sample.yaml for enabling h264enc / jpegenc uvc based test on the stm32n6570_dk/stm32n657xx/sb platform. Signed-off-by: Alain Volmat <alain.volmat@foss.st.com>
avolmat-st
force-pushed
the
pr-uvc-videoenc-support
branch
from
d3a72d3 to
431233f
Compare
jfischer-no
approved these changes
Oct 24, 2025
ngphibang
approved these changes
Oct 24, 2025
josuah
approved these changes
Oct 24, 2025
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
8 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Allow creating a pipeline as follow
camera receiver -> encoder -> uvc
I post this PR while there are still some points to improve (cf hardcoded stuff detailed below) in order to get some first feedbacks. Since this depends on the UVC PR (PR #93192) and DCMIPP UVC PR (PR #94562), there are lots of commits in this PR. However only the LAST 8 COMMITS is relevant for this PR.
If the chosen zephyr,videoenc is available, the sample will pipe the camera receiver to the encoder and then the UVC device instead of directly the camera receiver to the UVC.
In order to making the change as simple as possible, the source device of the UVC device is renamed uvc_src_dev instead of video_dev previously since, depending on the configuration, the UVC source might be either the video_dev or the encoder_dev.
Current implementation has several points hardcoded for the time being:
1. intermediate pixel format between the camera receiver and encoder
is set to NV12. This is temporary until proper analysis of video_dev
caps and encoder caps is done, allowing to select the common
format of the two devices.
2. it is considered that encoder device do NOT perform any resolution
change and that encoder output resolution is directly based on the
camera receiver resolution. Thanks to this, UVC exposed formats
are thus the encoder output pixel format & camera receiver
resolutions.
This has been tested using the STM32N6-DK and the JPEG codec, leading to the following pipe:
IMX335 -> CSI/DCMIPP -> JPEG -> UVC
compiled via:
And also VENC codec, leading to the following pipe:
IMX335 -> CSI/DCMIPP -> VENC -> UVC
compiled via: