Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Prep HEVC decoder for upstreaming #6519

Draft
wants to merge 713 commits into
base: rpi-6.12.y
Choose a base branch
from

Conversation

6by9
Copy link
Contributor

@6by9 6by9 commented Dec 6, 2024

Adds the driver with the multiplanar SAND format support, and then adds the minimal patch to reintroduce the legacy single planar format.

Working with both stock FFmpeg, and John's new multi-planar branch https://github.com/jc-kynesim/rpi-ffmpeg/tree/dev/5.1.6/sandm_1

Mesa does still need updating to support the SAND height of 0.

naushir and others added 29 commits December 10, 2024 14:46
Add version 4.17.1 of the Hailo PCIe device drivers.
Sourced from https://github.com/hailo-ai/hailort-drivers/

Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
An attempt to fix the include paths - they look reasonable, but the
GitHub auto-builds fail.

Signed-off-by: Phil Elwell <phil@raspberrypi.com>
Signed-off-by: Phil Elwell <phil@raspberrypi.com>
Add helpers to set and get vchiq driver data. vchiq_set_drvdata() and
vchiq_get_drvdata() wraps dev_set_drvdata() and dev_get_drvdata()
respectively.

Signed-off-by: Umang Jain <umang.jain@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
The vchiq_connected.h header was removed in f875976 ("staging:
vc04_services: Drop vchiq_connected.[ch] files") to simplify the
implementation.

Update the vc_sm driver accordingly which can still use the same
functions through the vchiq_arm.h header declarations.

Fixes: b1ab7a0 ("staging: vc04_services: Add new vc-sm-cma driver")
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Drop the include directive. They can break the build, when one only
wants to build a subdirectory. Replace with "../" for the includes in
the vc_sm files instead.

The fix is equivalent to the four patches between 29d49a7
("staging: vc04_services: bcm2835-audio: Drop include Makefile
directive")...2529ca2 ("staging: vc04_services: interface: Drop
include Makefile directive")

Fixes: b1ab7a0 ("staging: vc04_services: Add new vc-sm-cma driver")
Suggested-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Register the vcsm rive with the vchiq_bus_type instead of useing the
platform driver/device.

Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Phil Elwell <phil@raspberrypi.org>
After the decision to use bcm2711 compatible for upstream, we should
switch all accepted compatibles to bcm2711. So we can boot with
one DTB the down- and the upstream kernel.

Signed-off-by: Stefan Wahren <wahrenst@gmx.net>
Now that V4L2_PIX_FMT_Y12P is defined, allow passing raw 12bit
mono packed data through the peripheral.

Signed-off-by: Dave Stevenson <dave.stevenson@raspberrypi.com>
Now that V4L2_PIX_FMT_Y14 and V4L2_PIX_FMT_Y14P are defined,
allow passing 14bit mono data through the peripheral.

Signed-off-by: Dave Stevenson <dave.stevenson@raspberrypi.com>
Now that the 14bit non-packed Bayer formats are defined, add them
into the supported formats lookup table.

Signed-off-by: Dave Stevenson <dave.stevenson@raspberrypi.com>
Add two new DT properties:
* microchip,eee-enabled  - a boolean to enable EEE
* microchip,tx-lpi-timer - time in microseconds to wait before entering
                           low power state

Signed-off-by: Phil Elwell <phil@raspberrypi.com>
Disabling tx_lpi or eee should not cause the value of tx_lpi_timer to
be lost, even though it is not useful until they are re-enabled.

Signed-off-by: Phil Elwell <phil@raspberrypi.com>
The platform model originally handled the DMA mask. Now that
we are on the vchiq_bus we need to explicitly set this.

Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Backport to rpi-6.6.y the mainline version of the PiSP BE driver.

The backported version of the driver corresponds to the one available
at:
https://lore.kernel.org/all/20240626181440.195137-1-jacopo.mondi@ideasonboard.com/

Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Re-introduce multi-context support that was dropped from the
mainline driver version.

Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Offset the backend dev-nodes starting at /dev/video20
onwards to maintain backward compatibility with the
pre-upstreamed kernel driver.

Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Add YAML device tree bindings for the Raspberry Pi RP2040 GPIO Bridge.

Signed-off-by: Richard Oliver <richard.oliver@raspberrypi.com>
The Raspberry Pi RP2040 GPIO bridge is an I2C-attached device exposing
both a Tx-only SPI controller, and a GPIO controller.

Due to the relative difference in transfer rates between standard-mode
I2C and SPI, the GPIO bridge makes use of 12 MiB of non-volatile storage
to cache repeated transfers. This cache is arranged in ~8 KiB blocks and
is addressed by the MD5 digest of the data contained therein.

Optionally, this driver is able to take advantage of Raspberry Pi RP1
GPIOs to achieve faster than I2C data transfer rates.

Signed-off-by: Richard Oliver <richard.oliver@raspberrypi.com>

spi: rp2040-gpio-bridge: Add debugfs progress indicator

Useful for tracking upload progress via userspace.

Signed-off-by: Naushir Patuck <naush@raspberrypi.com>

spi: rp2040-gpio-bridge: add missing MD5 dependency

rp2040-gpio-bridge relies on the md5 crypto driver. This dependency
cannot be determined automatically as rp2040-gpio-bridge does not
use any of md5's symbols directly.

Declare a soft 'pre' dependency on md5 to ensure that it is included and
loaded before rp2040-gpio-bridge.

Signed-off-by: Richard Oliver <richard.oliver@raspberrypi.com>
The snps,block-size DT property declares the maximum block size for each
channel of the dw-axi-dmac. However, the driver ignores these when
setting max_seg_size and uses MAX_BLOCK_SIZE (4096) instead.

To take advantage of the efficiencies of larger blocks, calculate the
minimum block size across all channels and use that instead.

See: raspberrypi#6256

Signed-off-by: Phil Elwell <phil@raspberrypi.com>
The firmware advertises its features as a string of words separated by
spaces. Ensure that feature names are only matched in their entirety.

Signed-off-by: Phil Elwell <phil@raspberrypi.com>
The Cypress firmwares use "extsae" to indicate wpa_supplicant-hosted
SAE/WPA3.

Signed-off-by: Phil Elwell <phil@raspberrypi.com>
support sae executed in wpa_supplicant and offload 4-way handshake offload.

Signed-off-by: Chien-Chia Chen <carella.chen@infineon.com>

JIRA: SWWLAN-142424
Define two new RP1 clocks, representing the MIPI DSI byteclock
sources for the dividers used to generate MIPI[01] DPI pixel clocks.
(Previously they were represented by "fake" fixed clocks sources).

Signed-off-by: Nick Hollinghurst <nick.hollinghurst@raspberrypi.com>
Whilst the Unicam driver has now been upstreamed it only supports
configuration via Media Controller (not driven from the /dev/videoN
node), which makes life significantly harder for simple devices such
as mono sensors, and HDMI or analogue video to CSI2 bridge chips
(eg TC358743 and ADV7282M).

Fix up the downstream driver so that it builds, reinstate the links
from Kconfig and Makefile to it, and give it a new Kconfig name
(VIDEO_BCM2835_UNICAM_LEGACY).

Signed-off-by: Dave Stevenson <dave.stevenson@raspberrypi.com>
The upstream Unicam driver needs a set of userland changes to get
libcamera to run, and those aren't written or merged yet.

Move the "brcm,bcm2835-unicam" compatible from the upstream driver
to the old downstream version so that users can run libcamera
against 6.10.
Once the libcamera changes have been merged this can be reverted
to use the upstream driver.

If using the non-legacy compatible then assume we want to use
media-controller API for configuration.

Signed-off-by: Dave Stevenson <dave.stevenson@raspberrypi.com>
pelwell and others added 12 commits December 10, 2024 14:46
A dts file should have exactly one /dts-v1/ tag, and overlays should
also have one /plugin/ tag. Through careless inclusion of other files,
some Device Trees and overlays end up with duplicated tags - this
commit removes them.

The change is largely cosmetic, unless using an old version of dtc.

Signed-off-by: Phil Elwell <phil@raspberrypi.com>
…sing pimidi overlay.

This change removes the forced 1MHz clock frequency, so it can be overridden using `i2c_arm_baudrate`.
Some NVME drives seem to request significant amounts of DMA coherent
memory - enough to exhaust our standard 64MB CMA allocation.

Try disabling the feature to see what effect it has - drives should
continue to function without it.

Link: raspberrypi#6504

Signed-off-by: Phil Elwell <phil@raspberrypi.com>
A dev_info call in rpi_fw_uart_configure causes kernel log output every
time one opens the UART. Demote it to dev_dbg.

Signed-off-by: Phil Elwell <phil@raspberrypi.com>
The Arducam module is slow starting up, so add an override
to slow the regulator down.
https://forums.raspberrypi.com/viewtopic.php?t=380236

Signed-off-by: Dave Stevenson <dave.stevenson@raspberrypi.com>
Signed-off-by: eng33 <eng33@waveshare.com>
Signed-off-by: eng33 <eng33@waveshare.com>
Signed-off-by: eng33 <eng33@waveshare.com>
Signed-off-by: Phil Elwell <phil@raspberrypi.com>
The upstream addition of the kernel parameter cgroup_disable makes it
possible to configure cgroups at boot time. In theory, re-enabling a
disabled cgroup is simply a case of removing the relevant cgroup_disable
setting, but this is difficult if the setting comes from Device Tree.

Re-introduce cgroup_enable as a way around the problem.

Signed-off-by: Phil Elwell <phil@raspberrypi.com>
The datasheet lists that "Maximum exposure time is frame
length -25 row periods, where frame length is set by
registers {0x380E, 0x380F}".
However this driver had OV9282_EXPOSURE_OFFSET set to 12
which allowed that restriction to be violated, and would
result in very under-exposed images.

Correct the offset.

Fixes: 14ea315 ("media: i2c: Add ov9282 camera sensor driver")
Signed-off-by: Dave Stevenson <dave.stevenson@raspberrypi.com>
There was a report that 6.12 kernel has lower benchmark
scores than 6.6.

I can confirm, and found it started with 6.8 kernel
which moved some code into a new file (v3d_submit.c)
and in two places the change to the clock api were missed.

The effect of the bug is the v3d clock sometimes
unwantedly drops to a lower rate.

With this patch the benchmark scores are good again.

Fixes: 8696303

Signed-off-by: Dom Cobley <popcornmix@gmail.com>
6by9 and others added 14 commits December 12, 2024 16:20
Adds a binding for the HEVC decoder found on the BCM2711 / Raspberry Pi 4,
and BCM2712 / Raspberry Pi 5.

Signed-off-by: Dave Stevenson <dave.stevenson@raspberrypi.com>
The Raspberry Pi HEVC decoder uses a tiled format based on
columns for 8 and 10 bit YUV images, so document them as
NV12MT_COL128 and NV12MT_10_COL128.

Signed-off-by: Dave Stevenson <dave.stevenson@raspberrypi.com>
Add V4L2_PIXFMT_NV12MT_COL128 and V4L2_PIXFMT_NV12MT_10_COL128
to describe the Raspberry Pi HEVC decoder NV12 multiplanar formats.

Signed-off-by: Dave Stevenson <dave.stevenson@raspberrypi.com>
The BCM2711 and BCM2712 SoCs used on Rapsberry Pi 4 and Raspberry
Pi 5 boards include an HEVC decoder block. Add a driver for it.

Signed-off-by: John Cox <john.cox@raspberrypi.com>
Signed-off-by: Dave Stevenson <dave.stevenson@raspberrypi.com>
The SAND handling had been using what was believed to be a runtime
parameter in the modifier, however that has been clarified that
all permitted variants of the modifier must be advertised, so
making it variable wasn't practical.

With a rationalisation of how the producers of this format are
configured, we can switch to a variant that doesn't have as much
variation, and can be configured such that only 2 options are
required.

Add a modifier with value 0 to denote that the height of the luma
column matches the buffer height, and chroma column will be half
that due to YUV420.
A modifier of 1 denotes that the height of the luma column still
matches the buffer height, but the chroma column height is the same.
This can be used to replicate the previous behaviour.

Signed-off-by: Dave Stevenson <dave.stevenson@raspberrypi.com>
Signed-off-by: Dave Stevenson <dave.stevenson@raspberrypi.com>
Signed-off-by: Dave Stevenson <dave.stevenson@raspberrypi.com>
Adds the relevant node for the HEVC decoder on BCM2711 / Pi4.

Signed-off-by: Dave Stevenson <dave.stevenson@raspberrypi.com>
For downstream only, add back in the legacy single planar
SAND formats.

Signed-off-by: Dave Stevenson <dave.stevenson@raspberrypi.com>
Upstream will take the multi-planar SAND format, but add back
in the downstream single planar variant for backwards compatibility

Signed-off-by: Dave Stevenson <dave.stevenson@raspberrypi.com>
@6by9 6by9 force-pushed the rpi-6.12.y-hevc_dec branch from 3406809 to 6168cbb Compare December 12, 2024 17:36
@popcornmix popcornmix force-pushed the rpi-6.12.y branch 2 times, most recently from 2accb17 to 49679ed Compare December 20, 2024 13:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.