mpeg4 audio support

[totaam]

Issue migrated from trac ticket # 1194

component: sound | priority: major | resolution: fixed

2016-05-12 10:00:09: antoine created the issue

---

Based on mp4mux and qtdemux, we should be able to forward sound in an mpeg container.

Example pipelines found here: [https://github.com/pedwo/gst-sh-mobile/blob/master/examples/examples.sh]:

# Encode microphone input as mp4 AAC file.
gst-launch -e \
 alsasrc ! audio/x-raw-int, rate=16000, channels=2 \
 ! queue ! audioconvert ! faac profile=LC bitrate=192000 \
 ! queue ! mp4mux ! filesink location=./test.mp4

# mp4 AAC file playback
gst-launch \
 filesrc location=./test.mp4 \
 ! qtdemux ! queue ! faad ! audioconvert ! audioresample ! autoaudiosink

[totaam]

2016-05-12 13:31:56: antoine uploaded file add-aac-mpeg4.patch (3.5 KiB)

work in progress patch: try to add aac codec and mpeg4 muxer

[totaam]

2016-05-12 15:59:19: antoine changed owner from antoine to afarr

[totaam]

2016-05-12 15:59:19: antoine commented

---

Implemented in r12565: we now have aac+gdp and aac+mpeg4.
Both work OK on Fedora if you have all the required plugins installed (if not, run xpra/sound/gstreamer-util.py -v for diagnostics, then find the required rpm packages to install.

This is what I get:

$ xpra attach --speaker-codec=help
speaker codecs available: opus+gdp, opus, vorbis, vorbis+mka, flac+gdp, mp3, aac+gdp, aac+mpeg4, wav, wavpack, speex+gdp, speex

Since this seems to work well, I've ranked it higher than wav and wavpack, just after mp3.
r12574 adds support for those new formats to the win32 and osx clients and servers.
Minor improvement follow up in #1196.

[totaam]

2016-05-16 22:08:05: afarr changed owner from afarr to antoine

[totaam]

2016-05-16 22:08:05: afarr commented

---

Well, I'm getting the same response client side (using the 0.18.0 r12587 osx client in your repo), but I can't seem to find the rpms I need for a fedora 23 vm 0.18.0 server.

Google has directed me to rpmfusion (gstremaer-plugins-good/bad/ugly), gstreamer1-libav, gstreamer-ffmpeg, gstreamer-plugins-bad-nonfree, gstreamer1-plugins-bad-free-extras, vlc, & rygel... but I continue to get this message:

missing ['mp3parse'] from ('lamemp3enc', None, 'mp3parse ! mad', None)
missing ['faac', 'faad'] from ('faac', 'gdppay', 'faad', 'gdpdepay')
missing ['avenc_aac', 'avdec_aac'] from ('avenc_aac', 'gdppay', 'avdec_aac', 'gdpdepay')
missing ['faac', 'faad'] from ('faac', 'mp4mux', 'faad', 'qtdemux')
missing ['avenc_aac', 'avdec_aac'] from ('avenc_aac', 'mp4mux', 'avdec_aac', 'qtdemux')

I even tried a sudo dnf install gstreamer* --exclude=*devel*,*doc* - still no luck.

I presume that I'm just not finding the right things to search for... so I'll assign this back to you and leave you to decide to close (and I can take some more stabs at finding the right rpms when I get the chance) or perhaps provide a hint where to try next.

[totaam]

2016-05-17 04:27:45: antoine changed owner from antoine to afarr

[totaam]

2016-05-17 04:27:45: antoine commented

---

Odd, the aac codec is part of libav (see: [https://www.rpmfind.net/linux/rpm2html/search.php?query=gstreamer1(element-avdec_aac)]), which should have been installed from your gstreamer* glob, try explicitly:

sudo dnf install gstreamer1-libav

Then verify with

gst-inspect-1.0 avenc_aac
gst-inspect-1.0 avdec_aac

[totaam]

2016-05-20 00:15:38: afarr commented

---

I feel a little less like I'm just missing something... or maybe I'm missing something unbelievably obvious?

sudo dnf install gstreamer1-libav
Last metadata expiration check: 0:34:17 ago on Thu May 19 15:48:42 2016.
Package gstreamer1-libav-1.6.3-1.fc23.x86_64 is already installed, skipping.
Dependencies resolved.
Nothing to do.
Complete!

...

[jimador@jimador bin]$ gst-inspect-1.0 avenc_aac
No such element or plugin 'avenc_aac'
[jimador@jimador bin]$ gst-inspect-1.0 avdec_aac
No such element or plugin 'avdec_aac'

Uhhh?

[totaam]

2016-05-20 04:31:23: antoine commented

---

NOT obvious at all.
Trying again in a clean virtual machine after installing rpmfusion and gstreamer1-libav:

gst-inspect-1.0 /usr/lib64/gstreamer-1.0/libgstlibav.so | grep aac

Shows the aac plugins, but they aren't present with a plain gst-inspect-1.0 | grep aac.

So then I wrote a script (run as root) to delete all the plugins and then re-add them one by one to find out which one is preventing us from loading the libav plugin:

#!/usr/bin/bash
#stash them away:
mkdir /usr/lib64/gstreamer-1.0.bak/
mv /usr/lib64/gstreamer-1.0/* /usr/lib64/gstreamer-1.0.bak/
#restore the libav one:
cp /usr/lib64/gstreamer-1.0.bak/*libav* /usr/lib64/gstreamer-1.0/
for x in `ls /usr/lib64/gstreamer-1.0.bak/`; do
        mv /usr/lib64/gstreamer-1.0.bak/$x /usr/lib64/gstreamer-1.0/
        #check if aac is still found:
        gst-inspect-1.0 avenc_aac >& /dev/null
        if [ "$?" != "0" ]; then
                echo "aac missing after adding $x"
                exit 1
        fi
        echo "* $x"
done

And to my surprise, it ran without a hitch.
After running this script, the aac plugins are found without problems every time.
There must be something broken with the gstreamer libav plugin packaging, or with the plugin probing.
You may be able to fix your installation with just:

touch /usr/lib64/gstreamer-1.0/*

If not, run the script. If that fixes things for you, please file an upstream ticket for this.

[totaam]

2016-05-21 01:39:11: afarr commented

---

Touch, unfortunately, seems to fail...

[jimador@jimador bin]$ sudo touch /usr/lib64/gstreamer-1.0/*
[jimador@jimador bin]$ gst-inspect-1.0 avenc_aac
sys:1: PyGIWarning: Gst was imported without specifying a version first. Use gi.require_version('Gst', '1.0') before import to ensure that the right version gets loaded.

(gst-plugin-scanner:10366): GStreamer-WARNING **: Failed to load plugin '/usr/lib64/gstreamer-1.0/libgstlibav.so': libx264.so.148: cannot open shared object file: No such file or directory
DtsGetHWFeatures: Create File Failed
sys:1: Warning: cannot register existing type 'GstSmooth'
sys:1: Warning: g_once_init_leave: assertion 'result != 0' failed

(gst-plugin-scanner:10366): GStreamer-CRITICAL **: gst_element_register: assertion 'g_type_is_a (type, GST_TYPE_ELEMENT)' failed
No such element or plugin 'avenc_aac'

Running the script... I get aac missing after adding libgst1394.so ... but trying to run gst-inspect again, I get more No such element or plugin 'avenc_aac'.

I'll see about filing a ticket upstream about the libgst1394.so... after a quick tweak to the script to be sure that's not the first library and all of them are giving me issues.

[totaam]

2016-05-21 04:00:00: antoine commented

---

This is an upstream bug in rpmfusion:

(gst-plugin-scanner:10366): GStreamer-WARNING **: Failed to load plugin '/usr/lib64/gstreamer-1.0/libgstlibav.so': libx264.so.148: cannot open shared object file: No such file or directory

You need to install x264-libs, which rpmfusion should include as a dependency of the gstreamer-libav package, as libav will not load without it.

Maybe aac will work after that, or maybe you'll need to run touch again, or run the script again.
libgst1394 is probably a false positive, as the libav and its aac codecs will not work until x264-libs is installed.

[totaam]

2016-05-23 23:35:24: afarr changed status from new to closed

[totaam]

2016-05-23 23:35:24: afarr set resolution to fixed

[totaam]

2016-05-23 23:35:24: afarr commented

---

Ok... looks like sudo dnf install x264-libs was "all" I needed to do (though, it occurs to me I've done all of the above on the same vm, so not sure that it was indeed all that was needed).

Went ahead and grabbed a clean vm to confirm the mess of steps:

• Install rpmfusion.

• Install gstreamer1-libav
• Install x264-libs
• touch /usr/lib64/gstreamer-1.0/*

And that's "all" I had to do to get something when inspecting.

Closing.

[totaam]

2016-07-12 17:52:23: antoine commented

---

Milestone renamed

[totaam]

2016-11-10 08:34:10: antoine commented

---

r14401 adds "mp3+mpeg4" which may help with #1341, r14410 + r14415 tries to make the stream more "compatible".
The html5 client should now use mpeg4, see #1341#comment:6 and #845.