Playing IEC 61937 audio as stereo PCM not decoded at HDMI amp

[tghewett]

Playing 5.1 S/PDIF (IEC 61937) audio, i.e. AC3 and DTS encapsulated as stereo PCM, is not detected as encoded audio at the amp and so only noise is heard. Playing the same audio as stereo PCM to the same amp from the HDMI output of a Mac running MacOS 10.8 (full volume setting, i.e. x 1.0) is decoded without any problems.

The device volume setting was suspected as disrupting the IEC 61937 bitstream, i.e. not x 1.0, but no setting seems to fix the problem and allow the bitstream through unmodified.

The amp receives AC3 and DTS fine if sent in raw unencapsulated form by setting the OMX audio coding appropriately (OMX_AUDIO_CodingDDP and OMX_AUDIO_CodingDTS).

[popcornmix]

Can you be clear what you are doing?
Are you using omxplayer? Your own code?
How are you producing/submitting "AC3 and DTS encapsulated as stereo PCM"?

[tghewett]

To convert a raw AC3 file to S/PDIF format (adapt if AC3 is sampled at 44100Hz):

ffmpeg -i file.ac3 -acodec copy -f spdif - | ffmpeg -f s16le -ac 2 -ar 48000 -i - -acodec copy -f wav file.wav

I modified hello_audio to play raw DDP and DTS from a file. Also using omxplayer to play raw AC3 and DTS files:

pi@raspberrypi ~ $ omxplayer -p -o hdmi 1.ac3
Audio codec ac3 channels 2 samplerate 48000 bitspersample 16
Subtitle count: 0, state: off, index: 1, delay: 0
strg-c caught

have a nice day ;)
pi@raspberrypi ~ $ omxplayer -p -o hdmi 1.dts
Audio codec dca channels 2 samplerate 48000 bitspersample 16
Subtitle count: 0, state: off, index: 1, delay: 0
strg-c caught

have a nice day ;)
pi@raspberrypi ~ $

[EDIT: also using omxplayer to play the wav files created using ffmpeg as well as aplay using the ALSA driver, the amp only plays noise. The motivation for keeping AC3 and DTS in IEC 61937 form is for importing into iTunes, iTunes doesn't recognised raw AC3 and DTS].

[popcornmix]

Sorry, are you playing the wav file with omxplayer, or a dts/ac3 file?
How are you playing this file on Mac?

[tghewett]

See edit in previous comment.

[popcornmix]

"omxplayer -p" will be doing the spdif packetisation, which I assume the ffmpeg stage has already done.
I imagine you'd have more luck with playing a wav file and outputting PCM, although I'm not sure of the format required to get the samples out unmolested.

It may be possible to add a new passthrough format (or hack dts in a test build) to skip the spdif stage, but I don't know standard well enough to know if that's all that needs doing, and whether there's any endian, packet size or timing issues that may complicate things.

[tghewett]

I have checked the passthrough option of omxplayer, it is just to effectively annul the operation of the decoder component. The hello_audio example doesn't use a decoder component and so the PCM, DDP and DTS bitstream effectively is passed through unmodified. I've also tried playing a IEC 61937 raw PCM file (not wav) with an adapted hello_audio copy, again just noise is heard. Playing the file to a USB S/PDIF device connected to the 'Pi and to the same amp, using aplay, works fine.

It is as if there is a flag set on the Pi's HDMI PCM transport which is stopping the amp from decoding the audio, a flag which isn't there in the Mac's HDMI output. I have no way of seeing what samples the amp is receiving to be able to tell if they have had any (even if very small) gain or other modifications applied to them.

I don't believe that omxplayer -p packetises to IEC 61937 format for transmission over HDMI, certainly there isn't anything within its own code doing that, instead sending the encoded audio in raw form in the same way as MLP/Dolby TrueHD/DTS HD etc.. The amp works fine with that, but it is clearly also capable of detecting and decoding IEC 61937 encoded audio when received in a HDMI stereo PCM stream.

For your information, the amp's EDID audio capabilities are listed as follows:

HDMI:EDID found audio format 2 channels PCM, sample rate: 32|44|48|88|96|176|192 kHz, sample size: 16|20|24 bits
HDMI:EDID found audio format 8 channels PCM, sample rate: 32|44|48|88|96|176|192 kHz, sample size: 16|20|24 bits
HDMI:EDID found audio format 8 channels AC3, sample rate: 32|44|48 kHz, bitrate: 640 kbps
HDMI:EDID found audio format 8 channels DTS, sample rate: 44|48 kHz, bitrate: 1536 kbps
HDMI:EDID found audio format 6 channels One Bit Audio, sample rate: 44 kHz, codec define: 0
HDMI:EDID found audio format 8 channels Dobly Digital+, sample rate: 44|48 kHz, codec define: 0
HDMI:EDID found audio format 8 channels DTS-HD, sample rate: 44|48|88|96|176|192 kHz, codec define: 1
HDMI:EDID found audio format 8 channels MLP, sample rate: 48|96|192 kHz, codec define: 0

[popcornmix]

omxplayer does (amongst other things):
m_omx_decoder.SetParameter(OMX_IndexParamBrcmDecoderPassThrough, &boolType);
which turns on the spdif packetisation on the GPU.

Can you put the hello_audio code on pastebin (or similar) and I'll check the code path, and see whether the samples are modified.

[tghewett]

No problem, it is in a .tar.bz2, uuencoded, at http://pastebin.com/Cb1ebdP6

Currently compiled to support PCM but can be changed for DDP (currently EAC3) or DTS.

I say that S/PDIF framing isn't used by the Pi's HDMI audio because the amp continues to show that AC3 or DTS is being received even when no audio is being sent, until the port is reconfigured to output a different kind of audio. Also there is no such thing as S/PDIF encapsulation for MLP/Dolby TrueHD, DTS HD, DSD etc..

[popcornmix]

On the GPU before dma-ing to the hdmi hardware all data is formatted and packetised.
Even PCM data goes through this sequence:

; Reformat "mid-justified" LPCM audio buffer in-place into IEC 60958 subframes
; fetch sample
; clear top 4 bits (flag bits)
; clear bottom 4 bits (preamble sync)
; add in spdif mask bits (flags, etc)
; test if count of bits in the frame is even or odd.
; set parity bit if count is odd.

The output of this sequence looks like the bytes in your wav file. (0x4e1ff872, 0x38000001)
I'm guessing this has already been done by ffmpeg and we want some means of disabling this processing to do a "raw" passthrough (not the same as PCM passthrough).

[tghewett]

Ok I see. That doesn't happen for PCM when sent over a physical S/PDIF connection, it is sent raw except for some very low level 32-bit framing usually hidden from view inside the device hardware, this is what I believe IEC 60958 to be, i.e. the lowest level of an S/PDIF link handing matters of synchronisation etc.. From what you have said it looks like PCM is being encapsulated in the IEC 61937 frames, which seems unusual in that the standard exists for conveying encoded audio over PCM - if it is doing this then it is using PCM to convey encapsulated PCM. It may be the case that the Mac isn't doing that hence the amp is able to decode it. It may also be the root of my sense that there is a flag telling the amp to only decode the stream as PCM, since the IEC 61937 frames may be indicating that to it.

I have seen the ffmpeg (actually libavformat) code for S/PDIF mixing and as you suspect it is that which is adding the frame header sequence you mention as part of the S/PDIF muxing.

[tghewett]

I am wrong about Dolby True HD and DTS HD having no relevance to IEC 61937, in fact they are listed in the coder enum in libavformat's spdif.h. They don't fit over an S/PDIF connection though. It seems to me that this is one (the) way audio is sent over HDMI.

enum IEC61937DataType {
IEC61937_AC3 = 0x01, ///< AC-3 data
IEC61937_MPEG1_LAYER1 = 0x04, ///< MPEG-1 layer 1
IEC61937_MPEG1_LAYER23 = 0x05, ///< MPEG-1 layer 2 or 3 data or MPEG-2 without extension
IEC61937_MPEG2_EXT = 0x06, ///< MPEG-2 data with extension
IEC61937_MPEG2_AAC = 0x07, ///< MPEG-2 AAC ADTS
IEC61937_MPEG2_LAYER1_LSF = 0x08, ///< MPEG-2, layer-1 low sampling frequency
IEC61937_MPEG2_LAYER2_LSF = 0x09, ///< MPEG-2, layer-2 low sampling frequency
IEC61937_MPEG2_LAYER3_LSF = 0x0A, ///< MPEG-2, layer-3 low sampling frequency
IEC61937_DTS1 = 0x0B, ///< DTS type I (512 samples)
IEC61937_DTS2 = 0x0C, ///< DTS type II (1024 samples)
IEC61937_DTS3 = 0x0D, ///< DTS type III (2048 samples)
IEC61937_ATRAC = 0x0E, ///< Atrac data
IEC61937_ATRAC3 = 0x0F, ///< Atrac 3 data
IEC61937_ATRACX = 0x10, ///< Atrac 3 plus data
IEC61937_DTSHD = 0x11, ///< DTS HD data
IEC61937_WMAPRO = 0x12, ///< WMA 9 Professional data
IEC61937_MPEG2_AAC_LSF_2048 = 0x13, ///< MPEG-2 AAC ADTS half-rate low sampling frequency
IEC61937_MPEG2_AAC_LSF_4096 = 0x13 | 0x20, ///< MPEG-2 AAC ADTS quarter-rate low sampling frequency
IEC61937_EAC3 = 0x15, ///< E-AC-3 data
IEC61937_TRUEHD = 0x16, ///< TrueHD data
};

[popcornmix]

I've got a suspicion the Mac is parsing the PCM for headers and skipping the encapsulation when it already appears to be validly encapsulated.

I've tried sending various types of raw data over hdmi, and without the header bytes, you don't get any audio out.

[tghewett]

So if you send ffmpeg-encapsulated AC3/DTS-in-PCM, you don't get audio?

[popcornmix]

I thought I had omxplayer working, which played my ffmpeg-encapsulated AC3 wav file correctly, but somehow ffmpeg is extracting the ac3 packets and they are just going through the normal AC3 passthough route.

Attempting to pass the ffmpeg-encapsulated AC3 data through raw hasn't made any noises so far (not even white noise).

[tghewett]

I think you might be right about the Mac parsing the incoming audio for the IEC headers.

Just re-tested the Mac, the amp also continues to show the received audio as DD 5.1 etc after having received some then stopping it.

[tghewett]

I thought I had omxplayer working, which played my ffmpeg-encapsulated AC3 wav file correctly, but somehow ffmpeg is extracting the ac3 packets and they are just going through the normal AC3 passthough route.

I have seen VLC do that, which also uses the ffmpeg libraries like omxplayer. My modified hello_audio program may be helpful.

[tghewett]

Further, I'm wondering if the comment text you pasted in earlier is, as it says, referring to the low level S/PDIF IEC 60958 subframes I was mentioning (I described them as 32-bit based but it seems there is a longer term framing format according to the AES/EBU). It seems that the raw bit stream that is conveyed over a S/PDIF interface is being also conveyed over HDMI, i.e. S/PDIF over HDMI as it is sometimes described, minus the synchronisation and flag bits which are zeroed out as the comments suggest.

If so, you will get no audio if you bypass this, with or without samples containing IEC 61937 frames (which are at a higher level). So the comments you mentioned would not be connected to the frame header words (the first word, by the way, is the generic "here comes encoded audio" one, the second being specific to AC3, 01 being the enum value for that codec and the other 16 bits specifying the number of bits per frame).

When sending "raw", in this context it seems to me that is to mean "without IEC 61937 frames" rather than without the base S/PDIF means of transmitting samples.

[popcornmix]

So, given a ffmpeg-encapsulated AC3 wav file, whose contents looks like:
72f81f4e 01000038 770b97e4 401effe0
what are the bytes that should be sent over hdmi?

[tghewett]

So, given a ffmpeg-encapsulated AC3 wav file what are the bytes that should be sent over hdmi?

All of them, as LPCM encapsulated in IEC 60958 frames but not within IEC 61937 frames. I wonder whether this is what is happening already, which then would take us back to the start and the question of why it isn't working on the Pi. It would be interesting to know what header words you find when playing ordinary PCM, e.g. playing an audio WAV file with omxplayer not created from AC3 with ffmpeg, remembering from earlier that you found them in the HDMI output stream even for PCM.

[popcornmix]

Yes, I'll try and produce a wav file with, say incrementing numbers in, and see what gets sent over hdmi.
I'll also try to create a path so incrementing numbers sent from host and sent to HDMI with no modification.
That should make it theoretically possible to output any type of passthrough (including PCM) from the ARM.(*)

(*) Although not quite. There's an "n" and "cts" value the hardware needs to know that relates to the samplerate (see hdmi 1.3 spec pp.7.2). It also needs to know the number of channels.

However I think passthrough formats use the same samplerate/channels settings as stereo PCM would, so you can just setup openmax for stereo PCM at right samplerate and set the magic "raw" flag, and it could work.

[tghewett]

However I think passthrough formats use the same samplerate/channels settings as stereo PCM would, so you can just setup openmax for stereo PCM at right samplerate and set the magic "raw" flag, and it could work.

That's right, e.g. if you have a USB S/PDIF device you can play the WAV files created by ffmpeg directly using aplay, the only requirement being that the end-to-end gain is set to x 1.0, which is what I originally thought was the problem in relation to this situation. Some amps are tolerant to the wrong sample rate being used, mine uses the incoming sample rate to clock the DAC and will play 48k AC3 slower when sent at 44.1k. Others read the sample rate in the AC3 frames and reject the stream if incoming rate is wrong. The sample size doesn't seem to make much difference, 16/20/24 bits works fine at least with the two amps I have here, including over HDMI from the Mac. Setting the Mac's HDMI output to 8 channels causes just AC3 noise to be played.

[popcornmix]

Why does
ffmpeg -i file.ac3 -acodec copy -f spdif - | ffmpeg -f s16le -ac 2 -ar 48000 -i - -acodec copy -f wav file.wav
produce a file 3 times the size?

[tghewett]

Even at the highest encoding bit rate of 640kbps AC3 has a much lower rate than 16 bit 2-channel PCM at 48kHz, so when it is framed up for S/PDIF there is a lot of empty space/padding. Both should be the same duration when played in omxplayer.

[popcornmix]

So if input ac3 file were decoded to stereo PCM, and spdif encoded, would that be the same size as my file.wav file?

i.e. any compression achieved through encoded audio is lost through padding (but you get the multi-channel)

[tghewett]

Yes though there would be no need to S/PDIF encode it in PCM form. You'll need to tell ffmpeg to downmix to 2 channels or else it will create a 5.1 channel WAV file, which will be... let's see... 3 times larger than the S/PDIF WAV file.

You can get the raw AC3 back out of a ffmpeg S/PDIF-encoded WAV file with:

ffmpeg -i file.wav -f s16le -ac 2 -ar 48000 - | ffmpeg -f spdif -i - -acodec copy -f ac3 file.ac3

I think it is even possible with just one use of ffmpeg but don't remember the options right now.

[popcornmix]

Sorry about the delay. I got the hdmi spec and a hdmi analyser and found what looks like the problem.
We were setting the audio info frame "audio channel count" to 2 for stereo. There's actually an option 0 (refer to stream header) which makes it work.

You need to ensure resampling doesn't occur. I've added a config.txt parameter no_hdmi_resample=1 that you will need to set. (I'd like to make that a default, but I believe it caused some issue in the past, so it's optional for now).

Now I can "aplay file.wav" (after creating it with commands in first post) and receiver is showing dolby, and playing audio.

I'll push updated firmware later tonight....

And it's pushed:
sudo rpi-update
no_hdmi_resample=1 in config.txt
try your tests using aplay.

[tghewett]

I set the no-resample flag as above with the new firmware. Tried aplay to the BCM device, now works fine, the amp now decodes the DD 5.1 or DTS audio when in a wav file. However the same wav file played by omxplayer still comes out as noise. Is this an OMX library setup issue that needs to be updated in omxplayer to work with the changes?

When I get the chance I'll change the BCM ALSA driver to support 8 channels to see if these changes also fix the 8 channel issue (#178).

[popcornmix]

I had a look at #178 when I had the analyser plugged in, and it complained about the speaker mapping for some channel numbers. I've fixed those, and hello_audio was detected correctly with channels 2,3,4,5,6,7,8.

I've got an update to hello_audio itself to make it handle non-power of two number of channels.

[popcornmix]

BTW, I assume 8 channel DTS-HD, TrueHD encapsulated as stereo PCM should now be possible?

[tghewett]

BTW, I assume 8 channel DTS-HD, TrueHD encapsulated as stereo PCM should now be possible?

8-channel lossless compression wouldn't fit in 1536kbps IMHO, presumably it is sent encapsulated in 8-channel PCM instead. Dolby True-HD is just MLP, the same as used for DVD-Audio. Maybe the ffmpeg developers found some unencrypted source material to test their code with, code is in place for both with specific options for setting the encoding rate etc..

I managed to get the amp to recognise 5.1 audio from hello_audio but not the 7.1 audio which it sees from the Mac, so the changes evidently are in effect in the same path as that used by the ALSA driver, but my modified hello_audio which can play from a PCM file from stdin still produces noise from an AC3 wav file.

[popcornmix]

8-channel lossless compression wouldn't fit in 1536kbps IMHO, presumably it is sent encapsulated in 8-channel PCM instead. D

Can you play 8 channel on your Mac encapsulated in PCM? (with however many channels are required).
I'm wondering if we can add generic support for passthrough that will allow all the possible formats just by arm side code packaging them correctly.

I found that the speaker mapping of 0x1f (FRC FLC RR RL FC LFE FR FL) was fine on analyser, but my Onkyo didn't like it. I've changed that to 0x13 (RRC RLC RR RL FC LFE FR FL) and it detects 7.1.

I'll push the updated firmware and hello audio shortly.

Can you post the PCM version of ac3 in wav file (suitable for playing with your modified hello_audio) and I'll check why that isn't working (I'm assuming something like resampling or bit depth conversion is occurring).

Okay, disabled resampling through openmax and hello_audio passes through the AC3, like aplay.
Just rebuiding new firmware.

[popcornmix]

Firmware pushed.

[tghewett]

Firmware pushed.

Thank you. Everything now works, i.e. 7.1 output, hello_audio and omxplayer playing AC3/DTS wav files etc.. Playing those wav files as 8-channel PCM (with 6 channels of silence) instead of stereo results in noise but the same happens with the Mac. Maybe they need to be re-encapsulated for 8 channels, the libavformat source code is possibly the best documentation for that side of things, e.g.

/*

• It seems Dolby TrueHD frames have to be encapsulated in MAT frames before
• they can be encapsulated in IEC 61937.
• Here we encapsulate 24 TrueHD frames in a single MAT frame, padding them
• to achieve constant rate.
• The actual format of a MAT frame is unknown, but the below seems to work.
• However, it seems it is not actually necessary for the 24 TrueHD frames to
• be in an exact alignment with the MAT frame.
  */
  #define MAT_FRAME_SIZE 61424
  #define TRUEHD_FRAME_OFFSET 2560
  #define MAT_MIDDLE_CODE_OFFSET -4

static int spdif_header_truehd(AVFormatContext *s, AVPacket *pkt) {

[...]

Can you post the PCM version of ac3 in wav file (suitable for playing with your modified hello_audio) and I'll check why that isn't working (I'm assuming something like resampling or bit depth conversion is occurring).

From http://www.diatonis.com/surround_sound_music.html (with plenty of useful and interesting formats) get http://www.diatonis.com/downloads/diatonis_ac3_wav_anfos.zip. This is a wav which can be converted to a raw PCM file which hello_audio can play directly, do "ffmpeg -i diatonis_invisible_order_anfos_ac3.wav -acodec pcm_s16le -f s16le diatonis_invisible_order_anfos_ac3.pcm".

Can you play 8 channel on your Mac encapsulated in PCM?

I don't have any Dolby HD or DTS HD audio files to try sending either format in a form encapsulated by ffmpeg but it may be worth seeing if ffmpeg can create either format from ordinary AC3 and DTS and then test with that. Most Dolby/DTS HD audio is encrypted at this point...

[popcornmix]

There are samples with DTS-HD here:
http://www.demo-world.eu/trailers/high-definition-trailers.php

I'd be very interested if you can get that to play with 8 channels.

[tghewett]

I've found that 48kHz Dolby TrueHD is S/PDIF encapsulated by ffmpeg into a file 16 times larger than for the same (non-HD) AC3 audio. There is no way that can play over an 8-channel PCM stream, perhaps it needs to be at 192kHz to provide the 16x bandwidth. Neither the Mac nor OMX provides that over HDMI.

ffmpeg doesn't appear to extract DTS-MA in full, just the built-in non-HD compatibility DTS stream.

It may be the case that these formats just don't use PCM/encapsulated S/PDIF as the transport on HDMI and are sent differently. OMX doesn't appear to support either HD format specifically.

[popcornmix]

OMX has support for two passthrough formats (the 6 channel DTS and AC3 formats).
This requires quite a lot of GPU code.
It would obviously be better to support a more generic interface, where the ARM could pass through what it likes, using ffmpeg for encapsulation.

OMX has a limit of 96kHz on sampling rate, but I don't imagine there's a big problem in raising that to 192kHz.
8 channels at 192kHz is starting to get to a bitrate that may become expensive for the ARM to deal with.

I know xbmc supports DTS-HD and TrueHD passthough on other platforms:
http://wiki.xbmc.org/index.php?title=AudioEngine

so it may be worth looking at how it extracts and encapsulated the bitstream.

[tghewett]

OMX has a limit of 96kHz on sampling rate, but I don't imagine there's a big problem in raising that to 192kHz.
8 channels at 192kHz is starting to get to a bitrate that may become expensive for the ARM to deal with.

I had 192kHz and "8" channels working by modifying the ALSA driver (which seems to bypass the OMX limit), I don't recall it being too much for the ARM but was heavily loaded. OMX seems to use more CPU time for the same audio so may be more of a problem.

For me this issue is now resolved but if you wish then contact me privately then I'm happy to explore this some more.

[popcornmix]

Sure, contact me at popcornmix at gmail dot com.

[tghewett]

I'm not sure it is 100% resolved, in that yesterday the HDMI output stopped completely, it wasn't possible to output audio on it and the amp detected no HDMI input. The only fix was a reboot. So far it has only happened once but it never happened before these changes.

[popcornmix]

See if you can find a sequence of operations that results in the problem.

[tghewett]

It hasn't happened since in spite of being used in the same way, so will treat as unrelated and raise a new issue if necessary.