WIP: vc4: Avoid hdmi audio underrun with heavy sdram traffic

[popcornmix]

We use 8 channel 192kHz audio for HD passthrough formats.
We've found with v4l2 h.264 video decode that we get fifo underruns in the audio dma.
(That decode path triggers a vpu based frame copy/convert)

See: https://forum.kodi.tv/showthread.php?tid=359450

[popcornmix]

I think we should perhaps use DMA_WIDE_SOURCE / DMA_WIDE_DEST more often for memory source/dest.
Without it we are generating 4 times the bus traffic and (I think) 4 times the sdram bandwidth.

[pelwell]

Doing wide transfers for SPI might be quite a saving where it is being used to drive displays. And I2S, to a lesser extent. It's unfortunate that we can't use the same trick for those devices that share a channel for TX and RX as it stands.

[HiassofT]

quick heads up: we got a report that ext4 filesystems got trashed on RPi2 LibreELEC/LibreELEC.tv#4903

Reverting the bcm2835-dma: Add bcm2835-dma: Add DMA_WIDE_SOURCE and DMA_WIDE_DEST flags commit fixes this

[pelwell]

It might be wise to put some parentheses around the x here (and WAIT_RESP and WIDE_DEST), just in case.

[pelwell]

Is the moving of WAIT_RESP to every transfer (and not just the last) and considered change?

And the new WIDE flags won't get applied to DMA40 channels until to_bcm2711_srci and to_bcm2711_dsti are also changed.

[popcornmix]

It seemed to make more sense to me that way although was intended for discussion (PR got merged before that though). @HiassofT did report that reverting this change didn't avoid the corruption. And excluding this change the commit should be a no-op on Pi2/3 with c->dreq not including bits 28/29, so I'm a bit confused exactly where the cause of corruption is.

[pelwell]

Yes, so am I.

[HiassofT]

I'm quite puzzled why these changes would break anything on RPi2 as well, I've looked over them several times and changing things I thought I spotted didn't help - obviously I'm missing something so a second (or third) pair of eyes might help.

Here's the stuff I tested so far - it's not all combinations and better redo the tests yourself in case I might have messed up:

Report of breakage was with 5.10.5 commit a7eebc0 - which I could confirm.

Going back to previous builds showed that 5.10.4 commit b323545 plus the first 2 commits in this PR (the patch here LibreELEC/LibreELEC.tv@d2ea7e0) also corrupted SD cards.

I then tested with above mentioned 5.10.5 with the following results:

reverting eb092d9 fixed the issue

reverting the WAIT_RESP change didn't help

--- a/drivers/dma/bcm2835-dma.c
+++ b/drivers/dma/bcm2835-dma.c
@@ -861,9 +861,9 @@ static struct dma_async_tx_descriptor *bcm2835_dma_prep_dma_memcpy(
 {
        struct bcm2835_chan *c = to_bcm2835_dma_chan(chan);
        struct bcm2835_desc *d;
-       u32 info = BCM2835_DMA_D_INC | BCM2835_DMA_S_INC | WAIT_RESP(c->dreq) |
+       u32 info = BCM2835_DMA_D_INC | BCM2835_DMA_S_INC |
                   WIDE_SOURCE(c->dreq) | WIDE_DEST(c->dreq);
-       u32 extra = BCM2835_DMA_INT_EN;
+       u32 extra = BCM2835_DMA_INT_EN | WAIT_RESP(c->dreq);
        size_t max_len = bcm2835_dma_max_frame_length(c);
        size_t frames;

reverting the arch/arm/boot/dts/bcm2835-common.dtsi change in b561d64 didn't help either

--- a/arch/arm/boot/dts/bcm2835-common.dtsi
+++ b/arch/arm/boot/dts/bcm2835-common.dtsi
@@ -123,7 +123,7 @@
                        clocks = <&clocks BCM2835_PLLH_PIX>,
                                 <&clocks BCM2835_CLOCK_HSM>;
                        clock-names = "pixel", "hdmi";
-                       dmas = <&dma (17|(1<<27)|(1<<28))>;
+                       dmas = <&dma (17|(1<<27))>;
                        dma-names = "audio-rx";
                        status = "disabled";
                };

I haven't tested these changes with 5.10.4 with just the first 2 commits (so leaving out the CS flags commit 32de3b3) - can try to do that the next days unless someone beats me to it

[graysky2]

@HiassofT - Quite a bit to follow here and in the 5.10.y thread in the forums.

1. Is RPi4 unaffected by this? There is at least one users in the forum thread stating otherwise, possibly two.
2. If not, what is the last-known-good 5.10.y kernel commit and what is the corresponding commit of the firmware in order to avoid the FS corruption?

Are these correct:
Kernel: ff8d655
Firmware: raspberrypi/firmware@481ccbb

[HiassofT]

@graysky2 kernel commit 5837bdb (the one before the HDMI audio DMA series) should be fine - I haven't tested this though, in LibreELEC we simply went back to the 5.10.4 version we were using before.

The firmware commit you referenced is fine - newer firmwares might be fine,too, I haven't tested this yet though.

I didn't notice any issues on RPi4, I and several forum users were testing earlier and the latest version of the HDMI audio patch successfully the last 1.5 weeks

[graysky2]

@HiassofT - Thanks for the info. I will just downgrade to 5837bdb until I see the issue has been fixed. Should someone create a new issue for this/I did not see one referencing the FS corruption? It would help users follow the upstream progress in fixing it and help us understand when it has been (officially) fixed.

[HiassofT]

@pelwell this is most likely the culprit https://github.com/raspberrypi/linux/blob/rpi-5.10.y/arch/arm/boot/dts/bcm270x.dtsi#L187

&sdhost {
        dmas = <&dma (13|(1<<29))>;

setting DREQ bit 29 seems to be an old debugging leftover added in commit 0c7dd5c

[popcornmix]

@HiassofT good spot. Also arch/arm/boot/dts/bcm2835-common.dtsi has
dmas = <&dma (17|(1<<27)|(1<<28))>;
Looks like bit 28 can mean WAIT_FOR_OUTSTANDING_WRITES and bit 29 can mean DISDEBUG in DMA_CS register (but we've been using them to pass other flags in). Spec says in DMA_CS:
15:09 Reserved
27:24 Reserved
I don't see 24/25 being used, so it may be safe to move to there.

[popcornmix]

@HiassofT does a896a00 avoid the issue?

[HiassofT]

@popcornmix thanks, very quick test on RPi2 looks good, LE started up fine. Will do more testing (also on RPi4) the next days

[popcornmix]

@HiassofT thanks for testing. I've pushed this to rpi-update.
Booted on a Pi3 without issue.

[HiassofT]

testing on RPi2, RPi3B+ and RPi4 looks good. No issues on RPi2/3 and HD audio passthrough on RPi4 worked fine