Multi Broadcasting
Google Summer of Code project by Stéphane Lepin
Current State: final coding period
This project for Mixxx aims to add several features potentially useful for users willing to broadcast live with Mixxx. Two new features will be implemented : broadcasting profiles and multiple broadcasting outputs. These features will be implemented in two parts, matching the GSoC phases schedule.
Broadcasting profiles allow management of several sets of server settings/credentials and encoder settings. These will be managed through dedicated controls in Mixxx's existing Live Broadcasting preferences UI. Profiles have the standard Icecast/Shoutcast and encoder settings currently available in the Live Broadcasting panel. The goal of Phase 1 is to replace the existing Live Broadcasting settings subsystem (in mixxx.cfg) with a new XML subsystem of profiles within the existing UI. The new UI along with multi-broadcasting is the goal of Phase 2.
- Profiles are saved as *.bcp.xml files under a "broadcast_profiles" subdirectory in the settings folder. This file nature is hidden from the user, and the profile list in the preferences UI is built by listing files in that folder.
- Create a new BroadcastProfile class:
- Move all properties from BroadcastSettings to BroadcastProfile
- The BroadcastProfile class takes care of its serialization/deserialization from/to XML, using Qt's built-in XML features
* save() method to save the profile
* Static method to build an instance based on an existing .bcp.xml file
* The XML document generated by an instance of BroadcastProfile has the profile's name as its filename (no need to store it inside the profile's XML document)
* Refactor the BroadcastSettings class:
* Move all properties to BroadcastProfile
* Keep a QMap of broadcast profiles (for fast name-based lookup)
* During at construct time, fill the QList of broadcast profiles by listing the contents of the "broadcast_profiles" settings subdirectory
* Add several signals for profile addition/removal/rename. Will be useful for phase 2
| [[/media/wiki/gsoc_palakis_phase1_umlbefore.png | ]] |
| Class structure before modifications | Class structure after modifications |
This feature allows live audio streaming to several servers simultaneously and relies on the broadcasting profiles added in Phase 1. Management of the streaming connections is done through a new “Live Broadcasting” settings panel, which has a list of broadcasting outputs with the profile settings form under the connections table. Selecting a profile/connection in the list shows its settings in the aforementioned form (which is also editable).
UI mockup of the original design
- The current libshout logic in EngineBroadcast must be separated from it and moved to a new class ShoutOutput (with QThread inheritance)
- ShoutOutput features:
* Broadcasting profile (instance of BroadcastProfile) as constructor parameter
* Has its own FIFO buffer filled by the engine
* Has its own thread (base on EngineBroadcast's) to process frames available in the FIFO buffer
* The EngineBroadcast sidechain filter must be refactored to only act as a "broadcast manager" that receives audio samples and pushes them to output instances
* Has an internal list of ShoutOutput instances, kept in sync with BroadcastSettings' profile list using signals and slots
* The Live Broadcasting settings UI must be updated (see UI mockup above)
There are several aspects to cover:
- Changes in LB preferences UI:
- Complete the "connection removal" user experience
- Show each connection state in the profile list
- Error reporting: show an error message when one or more active connections failed to connect
- Addition: Live Broadcasting Status dialog (a read-only list of each output connection)
In XML broadcasting profiles, two fields are considered sensitive: "Login" (username) and "Password" Currently, these sensitive fields are stored in plaintext. These should be optionally (enabled/disabled by a user setting) encrypted to avoid privacy and/or security issues. One way of securely storing credentials would be to use the OS' keychain using the 3rd-party QtKeychain library, which is compatible with Windows, Linux and OS X. The broadcasting profile subsystem would then put and fetch sensitive information from the keychain instead of the profile's XML document. Broadcasting profiles are currently not meant for import/export and sharing, so storing values outside of the XML document is fine. Users doing manual transfers of profiles from one system to another should then see empty values for the sensitive fields.
Entries stored through QtKeychain have three attributes
- Service: application name
- Key: generally used as the account's name
- Data: generally used as the account's password
The schema for entries meant for broadcasting profiles look like this:
- Service: broadcasting profile name, prefixed by "Mixxx - "
- Key: server username (called Login in profiles)
- Data: server password
- Confirmed on the wishlist: https://bugs.launchpad.net/mixxx/+bug/726991
- Libshout support: see link above
- First solution: legally, it is possible to use FFmpeg's built-in AAC encoder, which is covered by LGPLv2.1 (see source code), that license being seemingly compatible with Mixxx's GPLv2.
- Not possible for AAC+ with FFmpeg, because it relies on non-free libfdkaac.
- FFmpeg is already a Mixxx dependency
- Second solution: use dynamic loading of libfdkaac
- Both AAC (LC) and AAC+ (HE-AAC and HE-AAC v2) would be supported
- Windows: extract libfdk-aac-1.dll from a BUTT installation
- OS X: install fdk-aac from Homebrew (maybe too technical?)
- Linux: install libfdkaac package
- Confirmed on the wishlist: https://bugs.launchpad.net/mixxx/+bug/1338413
- libopus is quite easy to use
- The Opus datastream must be muxed in an Ogg stream
- School project
- Libshout streaming logic separate from EngineBroadcast to new class ShoutOutput
- Add QList<ShoutOutput> instance list to EngineBroadcast
- Keep BroadcastSettings' profile list in sync with EngineBroadcast's instance list
- Only on "Apply" button click on Live Broadcasting preferences
- Active connections shouldn't be restarted
- Begin work on settings UI refactor
- BroadcastSettings: add "rename profile" method
- Continue work on settings UI refactor
- Bug hunting in the Live Broadcasting engine
- Write PoC for secure credential storage
- Implement secure password storage for Broadcasting profiles
- Live Broadcasting: UI polishing
- Live Broadcasting preferences: actually apply settings to profiles only when clicking "Apply" or "OK"
- Work on preferences UI
- Bug fixing in multi-broadcasting
- Add support for AAC/AAC+ streaming with fdkaac
- Add support for Opus streaming
- Fix remaining UI bugs
- Rework parts of the wiki page
I wrote the implementation for the BroadcastProfile class: it currently has its XML load/save code that needs to be tested and the get/set behaviour of settings is a bit different compared to the settings management in BroadcastSettings: settings get/set methods in BroadcastSettings (that will be removed) "directly talk" with the mixxx.cfg file, while BroadcastProfile stores values temporarily in private members until save() is called to write the values to the profile's XML file. Also, methods beginning with "getDefault" don't exist in BroadcastProfile because these are set on instanciation of the class.
I got a bit late because of dev setup issues on my Windows system: I tried to work on Mixxx with Eclipse on Windows by using the build.bat provided in the Windows Dev Setup guide from the wiki. Building works, cleaning too (with a small addition to the Batch script) but Eclipse's "code checker" throws a lot of errors about undefined symbols (for the record: include paths for Qt and Mixxx's source code where added in the project's settings, along with appropriate search paths). In the end, I switched to Qt Creator, and it suits me for now.
For the next week of coding, I expect to have results on the XML profiles load/save mechanisms and UI by Friday.
I refactored BroadcastSettings as a manager of BroadcastProfile instances. It's integrated in the current UI, which saves settings to a file named "Default Profile.bcp.xml" in $SETTINGS_DIR/broadcast_profiles.
Actual implementation of BroadcastProfile differs a bit from the original design: file management (create/rename/delete/open) was initially planned to be done in a profile instance itself, but was eventually moved to BroadcastSettings to have this responsibility handled in a single class instead of several parts across several classes. However, the XML save/load code is still the responsibility of BroadcastProfile.
I got late with UI, so next week's plan is:
- First, make sure BroadcastSettings is instancianted only once
- Replace instances in DlgPrefBroadcast and EngineBroadcast with a pointer to a single instance
- Work on UI from Monday to Wednesday
- Testing during the remaining days.
I had progress on the UI but there was still a lot of work left to do to make sure everything is functioning properly. Instead of implementing the new Broadcasting Profiles UI, Daniel and I agreed on a slimmed-down schedule for the remaining of phase 1. The plan is to focus on the XML profile subsystem to have a good foundation for multiple broadcasting outputs (a.k.a multi-broadcasting, each profile being a connection) for phase 2, with proper instance reference passing and unit tests. See this Pull Request.
The UI will be left untouched, and the work already done is kept separately for later re-use.
Work went as planned for this week: unit tests for BroadcastProfile were written, pointers to BroadcastProfile instances are now QSharedPointers, turned BroadcastSettings into a Qt object to be able to implement basic read-only model support and profile name synchronization with signals & slots between BroadcastSettings' hashmap and the instances' name attribute. getCurrentProfile and setCurrentProfile were deleted because no longer useful, and were instead replaced with a more future-proof call to profileAt(int) that returns a profile based on its list index. Seems like work on Phase 1 is over. Further additions will come during phase 2, where those will be testable.
School project
So here starts Phase 2, where I get to implement multi-broadcasting! The streaming code has been separated from EngineBroadcast into a new class (ShoutOutput) that can be instanciated as needed. A ShoutOutput instance is linked with a profile pointer (BroadcastProfilePtr). Values are passed to libshout on connection start and when settings are applied from the Live Broadcasting preferences panel. Frames are polled from EngineBroadcast's thread like before, but are then passed to each ShoutOutput's process() method. Connections within EngineBroadcast are kept in sync with BroadcastSettings' profiles list using several signals/slots.
Currently I see one design flaw in my code: instead of passing a BroadcastProfilePtr to a ShoutOutput, ShoutOutput should be a child class of BroadcastProfile. Since the profile is never changed during the lifetime of a ShoutOutput instance, this would make the link between the two more obvious and logical.
Week 7's work is for the Preferences UI. Testing is already possible with the current UI, and the first WIP of the new UI will allow for testing on several streaming outputs.
In this past two weeks, I managed to work my way mostly through UI work. The QTableView used as a profile list is now properly formatted, and removing profiles/connections is now possible. Working with QTableView and Qt models was a bit confusing at first, but I managed to find solutions for my problems.
Engine-wise, I've migrated most of EngineBroadcast's code to a separate class called ShoutOutput. The initial design planned to pass each frame received by EngineBroadcast to every instance of ShoutOutput with ShoutOutput::process(), but this caused crackling audio (due to excessive latency between calls to process()) when too many connections were involved, with crackling increasing with the number of active connections. In the end, each ShoutOutput now has a FIFO buffer where EngineBroadcast puts new frames available to read. An idea to try out would be to implement a "slow start" for the ShoutOutput threads, where each thread waits for a defined number of available frames to be available before processing them.
Still engine-wise but related to settings, I've made a lot of improvements and bugfixes to the sync process between BroadcastSettings and EngineBroadcast. There are still improvements to be made in that area, though.
I finally managed to have working multi-broadcasting! The initial design was broken and lacked testing. Each ShoutOutput has its own FIFO buffer, fed by EngineBroadcast's thread which is in turn fed by EngineNetworkStream (responsible for overall buffer sync management). The final version must get rid of EngineBroadcast's thread and have proper buffer management as in EngineNetworkStream.
Regarding the idea of having secure password storage in XML profiles, I managed to have a working implementation using QtKeychain, so that's one less thing to worry about.
The secure password storage implementation is now complete, along with a proper compile flag to disable it at compile time. The preferences UI for Live Broadcasting has its design complete too. Work remains of minor behaviour fixes and the ability to cancel changes made to profiles. Engine-wise, work has been done to remove EngineBroadcast's thread and have proper buffer management with multiple streaming connections as SoundDeviceNetwork and EngineNetworkStream did before with a single connection. Most of the heavy thinking has been done on this, so what remains to do is putting proper structure into the latest changes.
Secure password storage now has error messages in case something goes wrong when reading from or writing to the OS keychain. The preferences UI now has proper separation between temporary settings and "live" settings. To achieve this, the model parts of BroadcastSettings were moved to a separate class, with a sync mechanism between the two triggered only when applying settings. With this and the colorful connection states, the preferences UI is near feature-complete. The audio engine part is now complete: work on the audio engine refactor for multi-broadcasting has been merged into the main work branch, and the remaining bits to tidy up (input of NetworkStreamWorker, proper structure) were addressed.
There's little much left to do on multi-broadcasting: bug hunting/fixing and fixing minor UI aspects still in discussion. Next week's focus will be to address these, as well as implementing new streaming encoders.
Writing in progress, publishing ASAP
Work in progress
This is the Mixxx Wiki - 
Mixxx is a free and open-source DJ software.
Manual
Hardware Compatibility
Reporting Bugs
Getting Involved
Contribution Guidelines
Coding Guidelines
Using Git
Developer Guide
Contributing Mappings
Mixxx Controls
MIDI Scripting
Components JS