Add OpenGL as source pad capability

[nuess0r]

Hi

Thank you very much for the gstreamer plugin. It makes my life to play around with libprojectm a lot easier and opens up lots of possibilities.

For some use cases it's convenient, that gst-projectm spits out a video stream but I was a little bit surprised to see that there is no OpenGL available as source capability. In gstreamer pipelines there are lots of OpenGL modules available that would avoid copying the gst-projectm output to the CPU and later back to the GPU.

Source pad capabilities of gst-projectm:

  SRC template: 'src'
    Availability: Always
    Capabilities:
      video/x-raw
                 format: { (string)RGBA }
              framerate: [ 0/1, 2147483647/1 ]
                  width: [ 1, 2147483647 ]
                 height: [ 1, 2147483647 ]

Here an example pipeline that works with the current gst-projectm:

gst-launch-1.0 audiotestsrc ! queue ! audioconvert ! projectm ! video/x-raw,width=640,height=480,framerate=30/1 ! videoconvert ! glupload ! glimagesink

Here how it should work when the GL texture can be directly used:

gst-launch-1.0 audiotestsrc ! queue ! audioconvert ! projectm ! video/x-raw,width=640,height=480,framerate=30/1 ! glimagesink

I don't know if it is feasible or wished to combine it with the fbo rendering patch #12

[hack-s]

This is indeed how the fbo rendering patch works. It allocates gl memory and passes gl buffers to the pipeline. The plugin does not copy gpu memory to main memory anymore, and leaves memory transfers to be handled by gstreamer.

[nuess0r]

I already applied the fbo rendering patch as otherwise the plugin was not working at all on my machine.

You mean, in my case I would not see a performance impact even if I have to add the ! videoconvert ! glupload ! elements to the pipeline?

[hack-s]

you shouldn't need glupload, since the projectm plugin is already writing to gl memory. What happens when you run it without it ? I run it like this, ex.
... projectm preset=prese-dir preset-duration=5 mesh-size=48,32 texture-dir=presets-milkdrop-texture-pack ! video/x-raw,width=1920,height=1080,framerate=60/1 ! glimagesink sync=true

Or did I not configure the caps right for the data now being in gl memory ?

[hack-s]

Ha, I did not realize I had to change the caps. I have pushed an update to the branch now.

[hack-s]

now, with the new caps definition in place, it would look like this: ... ! projectm preset=presets-dir preset-duration=5 mesh-size=48,32 texture-dir=presets-milkdrop-texture-pack ! "video/x-raw(memory:GLMemory),width=1920,height=1080,framerate=60/1" ! glimagesink sync=true

[hack-s]

There is an updated fbo rendering patch available, it has additional bug fixes and performance improvements: #17