Audio usability improvements

[totaam]

Issue migrated from trac ticket # 662

component: sound | priority: minor | resolution: wontfix

2014-08-23 21:50:00: onlyjob created the issue

---

Please allow audio codec selection from menu (similar to video encodings).
Also please introduce per-codec audio buffering config options.
Also it will be useful to have audio codec priority order config option to change hardcoded CODEC_ORDER = [MP3, WAVPACK, WAV, FLAC, SPEEX].
Thanks.

[totaam]

2014-08-24 05:12:26: totaam changed status from new to assigned

[totaam]

2014-08-24 05:12:26: totaam changed owner from antoine to totaam

[totaam]

2014-08-24 05:12:26: totaam commented

---

Please allow audio codec selection from menu (similar to video encodings).
[[BR]]
The codecs are defined in the order that has been most tested, I fear that flac and speex are likely to not work very well out of the box, if at all. Exposing them via the gui would make it more likely that the user would end up with a non functional sound setup.
[[BR]]
Also please introduce per-codec audio buffering config options.
[[BR]]
Hmm could do. I would prefer if we chose the right buffering options for each encoding automatically, which might be do-able, just needs time spent testing all the platforms and network combinations. Of which there are many.
And since it works well enough with mp3 on most platforms, and that wavpack works well enough as a fallback... there is little incentive to work on this.
[[BR]]

That said, you can play with:

XPRA_SOUND_QUEUE_TIME=450 xpra attach ...

And if you find a better value, re-assign to afarr for further testing and maybe we can tweak the defaults. Per-codec, and per-platform if needed.
[[BR]]

The one thing that is in favour of this change is usage over very jittery connections, like 3G, but since we have statistical data on the connection latency we could take this into account to calculate an appropriate buffer delay / size automatically.
[[BR]]

Note: the reason why the default buffer queue is so long is because on some platforms (OSX: #379), overruns and underruns are expensive to deal with.
Other related tickets: #362, #400

[[BR]]

useful to have audio codec priority order config option to change hardcoded
[[BR]]
To some extent, you can already affect this order by omitting some codecs from the speaker-codec and microphone-codec lists.

[totaam]

2016-06-17 09:01:59: antoine changed status from assigned to new

[totaam]

2016-06-17 09:01:59: antoine changed owner from totaam to onlyjob

[totaam]

2016-06-17 09:01:59: antoine commented

---

The codec order will honour the speaker-codec and microphone-codec options, trivial fix in r12849 (which I may backport).
It will still default to the pre-defined codec order if those command line / config file options are left unset. You can see these default values by running xpra _sound_query.

---

As for the buffering suggestion, that's unlikely to change because:

• no-one has suggested a better value for the default initial buffer size
• queue levels are now kept as low as possible: sound improvements: refactorings, cleanups, #849, this was possible thanks to the switch to gstreamer 1.x on OSX (switch to gstreamer 1.x on osx #970) and win32 (switch to gstreamer 1.x on win32 #1041)
• FYI, we now have many formats supported: add new sound container formats: webm / matroska #1090, uncompressed sound / stream compressed sound #1212, split dependency between sound encoders and decoders #1204, ie:

$ xpra attach --speaker-codec=help
speaker codecs available: opus+gdp, opus+ogg, vorbis+gdp, vorbis+mka, \
    flac+gdp, flac+ogg, mp3, aac+gdp, aac+mpeg4, raw+gdp+lz4, raw+gdp+lzo, raw+gdp, \
    wav+lz4, wav+lzo, wav, wavpack, speex+gdp, speex+ogg

and unless proven otherwise, the defaults (vorbis or opus) are the best codecs for almost every possible use case - both in terms of quality, compression ratio and latency: #1074

• the focus is on av-sync: synchronize sound with video frames #835, av-sync finer tuning #1164

[totaam]

2016-07-12 17:51:51: antoine commented

---

Milestone renamed

[totaam]

2016-08-21 10:55:50: antoine commented

---

Milestone renamed

[totaam]

2017-02-15 11:35:27: antoine changed status from new to closed

[totaam]

2017-02-15 11:35:27: antoine set resolution to wontfix

[totaam]

2017-02-15 11:35:27: antoine commented

---

As per comment:3, closing.