Missing video signal(?) when running hello_triangle and omxplayer at the same time

[dividuum]

While developing an omx based video player for the Pi I observed that sometimes the video signal gets lost when I play videos. I've been able to reproduce it by running these two programs at the same time:

/opt/vc/src/hello_pi/hello_triangle/hello_triangle.bin
omxplayer /opt/vc/src/hello_pi/hello_video/test.h264 --win '0 0 1920 200' --layer 10 --blank

I'm using a NOOBs installed raspbian with all packages updated to the latest version. I updated the firmware files using rpi-update. I observed the above behavior on different screens:

• Dell U2713HM using a HDMI cable with (1920x1080 resolution)
• ACER_V243HQ using DVI with an adapter (1920x1080 resolution)
• ASUS_VS228 using DVI with an adapter (1920x1080 resolution)
• L90D__DVI using DVI with an adapter (1280x1024 resolution). See note

On the last screen it works when I'm using the win parameters above. But once I change them to 0 0 1280 100 it stops working. I tried different things but I keep getting a blank screen/lost video signal:

• I tried it on a Pi2 and B+.
• I bought a new power adapter with 2.1A. No success.
• I tried different values (0, 4 and 7) of config_hdmi_boost in /boot/config.txt
• I tried an older firmware version (d6e004c61a7a749897c482c860d0b2c28196437e)
• I tried different gpu_mem settings (192 and 384MB)

Using the complete screen space in the win parameter (or simply omitting it) works. It sometime also works with other win sizes, but I did not yet find any reliable pattern. It also works when I get rid of the blank layer.

There is also a corresponding thread on http://www.raspberrypi.org/forums/viewtopic.php?f=67&t=105906

It sounds a bit like #286, so I tried running tvservice -s in a loop while the screen goes blank: tvservice keeps reporting the correct resolution. This thread sounds like it might have the same root cause: http://www.raspberrypi.org/forums/viewtopic.php?f=67&t=50126, sadly there is no solution.

Firmware version is

Mar 27 2015 18:51:06 
Copyright (c) 2012 Broadcom
version b2109b1c1c8ae8717df455cfaf312a6c1801f45a (clean) (release)

Kernel is

Linux raspberrypi 3.18.10-v7+ #774 SMP PREEMPT Wed Mar 25 14:10:30 GMT 2015 armv7l GNU/Linux

omxplayer versions tested are (from raspbian):

omxplayer - Commandline multimedia player for the Raspberry Pi
        Build date: Tue, 10 Feb 2015 01:49:30 +0000
        Version   : 337004e [master]
        Repository: https://github.com/popcornmix/omxplayer.git

and latest version from omxplayer.sconde.net

omxplayer - Commandline multimedia player for the Raspberry Pi
        Build date: Thu, 19 Mar 2015 01:49:46 +0000
        Version   : 7c752d3 [master]
        Repository: https://github.com/popcornmix/omxplayer.git

[popcornmix]

Does adding dispmanx_offline=1 to config.txt avoid the issue (most likely with reduced framerate and increased gpu memory)?

[dividuum]

That does indeed help but as you said greatly reduces the framerate when the video is then played in fullscreen. I didn't know about dispmanx_offline before, and I found this thread on the forum: http://www.raspberrypi.org/forums/viewtopic.php?f=67&t=80218. I have a few questions now:

• omxplayer uses a 1x1 pixels blank dispmanx layer to get rid of all console output. Is there maybe another way to archive this? Is it possible to completely remove the console layer so the HVS doesn't have to touch it?
• Is there any way to detect this problem other than looking at the screen?
• Would using a resizer component help? That way the HVS doesn't have to scale the layer?

Thanks for your quick response!

[popcornmix]

You currently have 4 overlays on the screen:
FB console
3D framebuffer from hello_triangle
Video from omxplayer
Black screen from --blank

The HVS (hardware video scalar) can work in an online or offline mode.
The default is the online mode, where for each source overlay it much fetch the source pixels, format convert, resize and composite (alpha blend) and then output that row of hdmi pixels "on the fly". If the time taken to do this is too great then we underrun the pixels output to display, and get dropouts.

The alternative is offline mode. We composite to an offscreen buffer in memory, and then output that to the display. This removes the real time constraint, but it is much less efficient, so you tend to halve the framerate.

I'd suggest you remove the framebuffer, and remove the --blank option (which is just hiding the unwanted framebuffer).

Easiest way to remove framebuffer is
tvservice -o && tvservice -p
and get it back with:
fbset -depth 8 && fbset -depth 16

You can also remove framebuffer with:
echo 0 > /sys/class/vtconsole/vtcon1/bind
but you have to be more careful with that one.

[dividuum]

Thanks for the explanation. Echoing 0 to vtcon1/bind seems to stops the framebuffer from being updated but its current content is still visible. So I suspect the layer is still composed by the hardware? Is there a way to see how many layers are currently active?

Your proposed solution would of course work for that setup, but that's just the easiest way to replicate a problem I having with my own program. I might not be able to predict layer sizes and counts there so it might be helpful to better understand the hardware limits. I didn't have problems displaying 4 videos (non-FullHD) at the same time. Each of them rendering using their own render component. So I guess it all depends on how expensive resizing and memory bandwidth is? Do you think it might help to resize the video (using a resize component) to the layer size?

[popcornmix]

Unscaled images are cheaper than scaled.
RGB is cheaper than YUV (YUV is 3 layers, two have an implicit scaling)
horizontal scaling is cheaper than vertical scaling.
layers which overlap on a single horizontal line are more expensive (i.e. four short wide videos stacked vertically is cheaper than 4 tall thin videos stacked horizontally)
Overclocking (core_freq and sdram_freq) helps.

[dividuum]

Thanks again. One last questions (for now): I guess there is no way to detect an HVS underrun other than looking at the screen? Because then I might at least throw some error instead of producing broken output.

[popcornmix]

It will likely show in "sudo vcdbg log assert" output, but exactly how it shows depends on which resource ran out.
It may be possible to report this through a more convenient means (e.g. a vcgencmd).

[dividuum]

It does:

3088307.371: assert( !(irq_status & (SCALER_ESFRAME1 | SCALER_ESLINE1 | SCALER_EUFLOW1 )) || hvs_assert_fn(1) ) failed; ../../../../../middleware/hvs/hvs.c::hvs_isr line 1240

Having a way to query this somehow using vcgencmd would be nice.

[dividuum]

Add command for querying if hvs asserts have occurred

Thanks for that. I assume

# vcgencmd get_hvs_asserts
hvs_asserts=1

returns the number of times the HVS had an underrun and resets the internal counter. Right?

[popcornmix]

Correct.

[dividuum]

Thanks again. While this doesn't really "fix the problem" I now at least know what's going on and might work around it. Closing this issue.

[julianscheel]

@popcornmix Is there a vcdbg command to get a list of currently active overlays that the HVS has to process? I just stumbled into this issue again (loosing HDMI sync for a second or two) and am wondering what might cause the HVS to fail here... Having an overview of active layers might help.

[popcornmix]

There isn't currently.

I agree a vcdbg/vcgencmd that displays the formats, source/dest rectangles and transform flags currently on the display, would be useful for debugging.
I guess it would also need to iterate over the displays (e.g. there may be an additional DSI/DPI display).

vcdbg makes sense logically, but vcdbg works by directly reading the GPU's data structures. Need to check if doing this without locking and/or cache coherency produces useful information. A vcgencmd would parse the info on the GPU side, so will be cache coherent and can lock the structures.

No promises when, but I'll add this to my list.

[julianscheel]

@popcornmix Cool, thank you

[popcornmix]

@julianscheel
You can try: https://dl.dropboxusercontent.com/u/3669512/temp/firmware_displist.zip
and run:
vcgencmd dispmanx_list

[julianscheel]

@popcornmix That was fast, thanks :) - I'll be able to test it on thursday.

[dividuum]

@popcornmix thanks for adding that feature! It's will be really useful to see what's going on. How is 'cost' calculated? What is 'lbm'? What's the meaning of the transform value? I guess it's the DISPMANX_TRANSFORM_T value that is used when creating a layer, except I pass DISPMANX_NO_ROTATE (0) for my GL layer and I see 20000. Or are GL layers always implicitly using DISPMANX_FLIP_VERT (2 << 17)? Would it be possible to include the layer number?

display:0 format:RGBA32 transform:20000 src:0,0,800,480 dst:0,0,800,480 cost:534 lbm:0
display:0 format:YUV_UV transform:0 src:0,0,560,320 dst:0,0,560,320 cost:461 lbm:9216

[popcornmix]

cost is an estimate of the number of cycles of HVS processing required for that element. If the sum of costs gets too high you may get HVS underruns and HDMI dropouts.

lbm is the line buffer memory. That is the context required for vertical resizing. That is a dedicated block of memory in the HVS hardware of 96K. Go above that and elements can't be displayed.

Adding the layer number can be done.

The transform values are from DISPMANX_TRANSFORM_T. GL buffers do get an implicit vertical flip. There may be other bits set for internal status (like 3D left/right flags).

[julianscheel]

@popcornmix Do you have an estimated threshold where cost will be too high?

[popcornmix]

No. It is going to vary with hdmi mode (and the pixel clock used). It will vary with overclock (core_freq mostly, but sdram_freq will have an effect). It will vary with the busyness off the system (sdram latency may mean more cycles are required to process an element).
And it is only an estimate - things like clock domain crossing mean the number of cycles taken can vary with phases of the actual clocks.
For a specific use case you could add elements until you get a failure to estimate the limit.

[julianscheel]

Ok, makes sense. I am wondering a bit that in my case the dropout seem to happen randomly and last for 1-2 seconds. I do not see any hvs_asserts in this case though (so far still running with firmware from May 18th, will update tomorrow). It might be that my setup is right at the edge of what can be handled and the pixelclock adjustments done by the mmal latency target cause it to collapse from time to time. Does this sound sane to you?

[popcornmix]

It's possibly, but I'd say sdram being busier periodically means the composition sometimes takes a bit longer is a more likely reason.

[julianscheel]

Seems more likely indeed. Varying stream bandwidth might make the difference... I'll investigate it further.

[julianscheel]

@popcornmix I'm having a bit of trouble running that firmware (in fact it's not a problem with the test firmware, but with any recent firmware. I probably have to bisect what's the first broken one for me). USB devices do not work anymore on a Model B Pi. I can see that bcm2708_usb is initialised by the kernel but no detected peripheral including the smsc network chip is detected. I've upgrade my kernel to 4.1.9 now, but without any change. I'm not using a device-tree enabled kernel yet, though. Do you have a clue what change might be related and what needs to be done to get USB back alive?

[julianscheel]

@popcornmix f6ce0b0 is the breaking one for me. I presume the relevant change is: firmware: Replacing board rev functions with board_info library - Any hint what I need to change in my system/kernel to cope with that?

[popcornmix]

No immediate answer. Can you create a new issue for this as it's in danger of being lost in this closed issue.

[julianscheel]

@popcornmix You're right, it's a bit misplaced in here. See #479

[julianscheel]

@popcornmix dispmanx_list works quite well for me. One thing that would be nice to have would be a pid of the owning process of that layer. But probably that's not easily achievable, is it?

[popcornmix]

Not from where the command is implemented (dispmanx). I think the layer above (display_server) may have that info, but I'll have to see how ugly it would be to get hold of that.