Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Just curious (I don't have an Android 12L device with such decoding capabilities at my disposal): have you seen what Spatializer.canBeSpatialized() returns, if we pass an AudioFormat with the channel mask set to AudioFormat.CHANNEL_OUT_7POINT1POINT4?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It depends on the spatializer capability on that device. For example, as long as one device integrates new Dolby DD+JOC decoder (multichannel output support), one Dolby 7.1.4 spatializer is integrated on that device too. So Spatializer.canBeSpatialized() will return true if you pass an AudioFormat with the channel mask set to AudioFormat.CHANNEL_OUT_7POINT1POINT4. The related code is in AudioPolicyManager::canBeSpatialized().
Our Dolby new DD+JOC decoder logic is,
ExoPlayer sets this key value to 99 in current release. So DD+JOC decoder will output 7.1.4ch for DD+JOC content.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thank you for the answer. How many channels will the compressed DD+JOC stream have before we decode it? Will it be 12 channels or something else?
For context: ExoPlayer will call the Spatializer.canBeSpatialized() and use a channel mask based on the channel count of the compressed audio. This will happen before decoding, while the player decides which audio track to select.
For example, some DD+JOC content we use is 6 channels and ExoPlayer will call Spatializer.canBeSpatialized() with channel mask AudioFormat.CHANNEL_OUT_5POINT1. I wonder if it is possible to
Do you see this scenario possible?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Excellent question!
The original DD+JOC stream can be 11.1ch (11 is object number) or 15.1ch (15 is object number). The decoded DD+JOC content should go through Object Audio Renderer (OAR, a component inside decoder) to be converted from object based audio to channel based audio (from 11.1/15.1 to 7.1.4 currently). OAR can output 2.0/5.1/7.1/5.1.2/7.1.4ch PCM based on configuration. Currently, for DD+JOC content, we always config it to output 7.1.4ch PCM.
As far as your example "some DD+JOC content we use is 6 channels", I don't think that is DD+JOC content. It should be legacy DD+ content with 5.1 channel mask. In that case, ExoPlayer can call Spatializer.canBeSpatialized() with channel mask AudioFormat.CHANNEL_OUT_5POINT1 and TRUE is returned in this case. Then ExoPlayer will set MediaFormat.KEY_MAX_OUTPUT_CHANNEL_COUNT to 99 (or 6, any case is OK.). For our decoder, it will output 6 channel PCM since this content is legacy 5.1 DD+ content rather than DD+JOC content. The 5.1 channel PCM goes to spatializer thread. According to CURRENT Android spatializer thread logic, this 5.1 channel PCM will be upmix to 7.1.4 by adding "0" to left rear / right rear and all four top channels. Finally, this 12 channel (6 of them are zeros) signal is spatialized by spatializer.
As far as DD+JOC content, your method will success on 11.1 DD+JOC content but fail on 15.1 DD+JOC.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
After internal discussion, our team's suggestion is to call Spatializer.canBeSpatialized() with AudioFormat.CHANNEL_OUT_7POINT1POINT4 always for DD+JOC content if Util.SDK_INT >= 32.
Two more things:
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I would like to clarify what might happen with 15.1 DD+JOC:
Util.getAudioTrackChannelConfig()
will returnAudioFormat.CHANNEL_INVALID
.AudioFormat.CHANNEL_INVALID
. Therefore, on an Android 12L device that supports Spatialization, we will try to select an alternative audio track, e.g. if the content offers a stereo track.SPATIALIZER_IMMERSIVE_AUDIO_LEVEL_NONE
.Util.getAudioTrackChannelConfig(16)
to get a channel mask to configure AudioTrack, which will beAudioFormat.CHANNEL_INVALID
and not AudioFormat.CHANNEL_OUT_9POINT1POINT6. I believe playback will fail.I'm just bringing this case to your attention in case you want 15.1 DD+JOC to work too.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for your detailed clarification. I think the key point here is that DD+JOC is unique and its stream channel count (16) is different from its decoder output channel count (12). The reason is DD+JOC is object based stream rather than legacy channel based stream.
As far as your questions,
Decoder will output 7.1.4 (12 channels) if device is Android 12L and later. Decoder will output 2 channels if device is Android 12 and before.
YES, we need to make 15.1 DD+JOC work too because most of DD+JOC contents in market are this format.
So, is it possible to implement this proposal on your side?
That is to say, the first two steps would be
Util.getAudioTrackChannelConfig()
will returnAudioFormat.CHANNEL_INVALID
.Spatializer.canBeSpatialized()
with channel maskAudioFormat.CHANNEL_OUT_7POINT1POINT4
rather thanAudioFormat.CHANNEL_INVALID
. On an Android 12L device that integrates Dolby Spatializer, I expect it returns true. Therefore, DD+JOC track is selected and user can get better immersive effect.The corresponding pseudocode is
int channelConfig = Util.getAudioTrackChannelConfig(16);
if (Util.SDK_INT >= 32 && MimeTypes.AUDIO_E_AC3_JOC == format.sampleMimeType)
channelConfig = AudioFormat.CHANNEL_OUT_7POINT1POINT4;
// Call Spatializer.canBeSpatialized()