-
-
Notifications
You must be signed in to change notification settings - Fork 33
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Surround sound #49
Comments
I could test it, have a receiver connected with hdmi arc |
@mariotaku Could you elaborate on what makes surround sound hard to test? Like @roykaandorp, I (and surely many other people) have the equipment necessary to test this and would be happy to help test if that really is the only thing getting in your way of testing surround sound. |
@ryang3d I don't have surround sound device for verification, which is the reason. |
Understood. We're definitely happy to help test surround as needed. Thanks again for all the effort! |
I could help test surround sound too if needed with a full 7.2 setup. |
I've tested surround with version 0.7.5 and I can say it's working! |
Nice! @roykaandorp, can you share any details about your home theatre configuration? What receiver are you using, what TV are you using, what cable (optical or hdmi) are you using between your TV and receiver, etc. UPDATE: I just tested surround sound using 0.7.5 (again using my same setup: 65SM8600PUA hooked up via optical cable to a Denon AVR-S510BT), but all the surround test tones are coming out of my front left and right channel speakers. No sound comes out of my center, rear left, rear right, or sub woofer. Definitely curious to hear about your working setup. |
Not sure if @roykaandorp is using local builds. This feature is work in progress and only available in debug builds (not in releases). But if it can play surround successfully, would be a great news indeed! |
I'd tested with the release 0.7.5 |
Correct me if i'm wrong, moonlight uses multichannel pcm for surround, optical out doesn't support that (not enough bandwidth). So unless the tv first transcodes it to a dts/dolby digital stream (that I assume will give noticeable latency) it won't work. I think it will only work with ARC. |
@roykaandorp Just a bit curious, because I don't remember I have provided enabled surround sound yet. I'll do in next release though. |
Yes I was surprised too, because you didn't mentioned it in that release, but i'd seen you was busy with that so I tried and it worked. With the older release I was using (0.6.8) my amp recognized the audio as stereo, now with 0.7.5 as multi in, the test video confirmed every channel works. |
@roykaandorp I can confirm that none of previous version has correct setup parameter for surround sound.
|
To use surround sound,
And enjoy! PulseAudio comes with acceptable audio latency, and I'll be trying to improve it in future releases. |
You might be onto something there... I enjoy surround sound, but I'm not much of an audiophile and my knowledge of requirements and compatibility are limited. Out of curiosity, I did try loading up the fraunhofer.de video in my TV's built-in web browser, and the surround sound worked perfectly (but when run through moonlight-tv it still comes out stereo only, even with v0.7.7 with PulseAudio and 5.1 enabled). But from what it sounds like you are saying, is that moonlight's potential use of multichannel PCM may be why it is not working in my case (makes sense!). Looks like it may be time for some system upgrades here! Definitely curious about the experiments in that raspi link, but not gonna get my hopes up there. Stoked to see this all come together. Keep up the amazing work! EDIT: Just thought I'd add that there is a good amount of discussion on this topic now that I've looked around more myself. Everything looks to confirm @roykaandorp's suggestion. One good example is here: moonlight-stream/moonlight-android#482 (comment) |
As seen here https://fwd.nl/app/uploads/HDMI21-earc-1024x459.jpg it only seems to be possible to get multichannel pcm with eARC. Read somewhere that LG tv models of 2020 or newer support that. Maybe take a look at transcoding it to ac3 isn't a bad idea? |
Please keep in mind that encoding audio via software is much slower especially for multiple channels. Too much processing will make latency unbearable, especially for gaming purposes. Currently there are only 2 ways reasonable, one is to play 5.1ch PCM, another is feed opus audio directly, although I'm not sure how well webOS can handle. |
For the second way I've been investigating for quite a long time, but seems the channels are always incorrect for webOS. |
I'm afraid you're right, however I found a post of someone who said he's using it succesful (didn't mention about latency), but for that the alsa plugin a52 is needed, I don't know if it's possible to use that with the read only file system? Another option I was thinking about, encoding it as ac3 on the host. Did a request for that as option at the Sunshine github, but probably then the client needs an adjustment too? |
@roykaandorp Alsa is not working by any attempt on webOS. So this couldn't be an option. Also transmitting other audio codec then opus is not possible, as this whole protocol should be exact same to NVIDIA's official implementation. |
@mariotaku There is also a dts encoder that should work with pulseaudio and could also be integrated. But I can imagine that it would take too much effort, as there is no guarantee the latency will be acceptable. |
I was thinking about doing some experiments about performing opus -> pcm -> ac3/aac conversion, but this will bring a whole lot tasks. |
A little bit update: opus -> pcm -> fdk-aac can have sounds, but have some problems. I'll fix this then you can try on your TV. |
So I have implemented AAC encoded playback on webOS 4 (NDL Audio). But there are few problems:
If AAC playback doesn't have surround sound, then I can say it's nearly not possible for Moonlight to have surround sound on webOS. |
Would like to try out, but have some difficulties with building the NDK, hopefully fixed now. Ubuntu 16.04 had some files it couldn't download, 18.04 errors on building qemu. Found a fix for that so hopefully it will build now, then I shall test the surround tomorrow |
You can grab prebuilt NDK to save some time, plus it can run on almost any distro |
Ok, I did a test, and things are a little weird... First thing, my LG's dev mode had reset, so I had to redo that process and re-install moonlight. I had been busy and hadn't played a game for 6+ weeks. Things aren't working like they used to, but the way this is manifesting has me questioning if an update (whether moonlight, webos, sunshine, etc) has broken my workaround or if the old receiver was upmixing the whole time. I tried two games, one of them just plays stereo, and the other isn't playing sound at all (the one with advertised atmos support). It's possible my old receiver was in an upmixing mode, but I don't think upmixing is capable of knowing where an object is in 3d game space and transitioning it from in front of me, to behind my right shoulder, left behind my left as I rotate the camera. That's something that used to happen, where the sound was coming out of that speaker exclusively. But the real reason I'm wondering if an update broke it is because the 2nd game i tested was plaque tale requiem, which worked perfectly fine in the past. Now, it only played sound (stereo) during the pre- menu intro video, and now it's silent in the menu and game. When I played through the game late last year, it was entirely through moonlight. Nothing has changed between now and then except software updates for moonlight, the TV, sunshine, GPU driver, and the new receiver. You can see in the picture that it's in home theater mode with 3d audio on. The key fact for me is: its silence tells me the spatial audio is blocking the sunshine from capturing audio. This didn't used to be the case. I'm fairly confident of this, but only way I could prove myself wrong is to roll back the software versions of everything which is too impractical. So maybe we're back to square one, so I apologize if I got anyone's hopes up. But @mariotaku , what do you think about the fact that I'm not getting audio at all from plaque tale? It seems to indicate that the normal methods of capturing audio are unavailable in this mode. If I quit game, turn off spatial audio, and open it I hear the game again. Rather than building an AC-3 encoder, is it possible to configure sunshine to capture the dolby encoded audio stream and pass it to moonlight, which can just pass it through eARC to the receiver just like Netflix and other apps do? It sounds like such an elegant solution to the problem and I think it will avoid the additional lag you mentioned, as we'd be hooking into a native solution that works lag-free already. It's like Microsoft handed us the solution on a silver platter, and the only thing that might get in our way is LG. |
@astarktc @pankompot I'll try to explain how Sunshine on Windows transfers audio to Moonlight.
So I doubt Dolby audio can be transferred at all. As for capturing Dolby audio stream - I'm afraid even we can do that, webOS will not be able to handle that either. It only supports PCM, AC3 on webOS 3 and 4. And PCM, OPUS on webOS 5+. Guess this can't be verified without me having a real surround sound setup... |
I have purchased a small gear to decode 5.1ch AC-3 from HDMI ARC. After it's arrival I'll do more research to see if it's possible to play surround sound. |
Awesome, hopefully your research and testing pans out. I really appreciate your effort on this (and I'm sure many others do too!). BTW, I figured out that the DD+ codec is EAC3, which is the most likely format that this windows atmos for home theater is using for output. Hopefully, the device you bought supports that too, but if it doesn't work for what you're trying to test, if it's AC-3 only that could be a reason why. Don't know if you found these already, but I found a few developer focused resources in case they're helpful: https://professionalsupport.dolby.com/s/article/Dolby-Atmos-ADM-Profile-specification?language=en_US https://developer.dolby.com/platforms/windows/dolby-atmos/implementation/ https://learn.microsoft.com/en-us/windows/win32/coreaudio/spatial-sound?redirectedfrom=MSDN Now, in regards to the formats you mentioned that webos can handle, that's for decoding right? From what I'm reading on LG's dev site, those are the formats supported for standalone audio. If sunshine is streaming the video and audio separately, that makes sense. The key to supporting AC3, EAC3, etc is to mux the audio and video together into a single stream via the mkv, mp4, or ts formats: https://webostv.developer.lge.com/develop/specifications/video-audio-230 So, I think what you're looking to implement is what software like Plex, Kodi etc do: stream a muxed audio/video format to the TV over the network. If in your research you find folks complaining about plex not streaming atmos on the webos app, I now realized based on that info above that it's because those folks are usually trying to play truehd audio from blurays, and that documentation makes it plain that truehd isn't a supported format, but DD+ (EAC3) is supported, so that won't prevent you from accomplishing the goal here. Technically, I don't even want moonlightv to decode the audio at all, I want it to pass through to the receiver. However, I think once the stream is streaming through mkv, mp4, or ts, that the webos decoder is automatically going to follow the tv's settings and pass the EAC3 through to the receiver. Just thinking outside the box, one other area we can also use to look for this information might be their DRM documentation. We all know the content creators/studios pushed hard for content protection for media played through streaming apps like Netflix, so maybe we'll find what you need here? https://webostv.developer.lge.com/develop/specifications/streaming-protocol-drm https://webostv.developer.lge.com/develop/guides/drm-content-playback |
@astarktc There are multiple principles for surround sound codec:
That's why AC-3 is the only option left. It can be encoded by FFMPEG, seems to be supported by webOS when streaming audio. |
I understand. We have to work within the limits we have, unfortunately. It would be nice if LG didn't make this difficult, especially in 2024. I did some more research, and copyright wouldn't be an issue for atmos as that is there is no license cost for atmos for games through windows. Truly, our only roadblock is the inability to mux without introducing latency 😞 if standalone e-ac3 decoding was supported in webos, it would be easy to capture the encoded audio from windows spatial sound and pass it through, even if we were only targeting 5.1/7.1 and discarding the atmos metadata. What I need to find is the least expensive client device that can support HDR, 7.1, and 120fps at the same time. Probably a mini or sff pc. That seems to be the only way to avoid any compromises. |
OK so here comes my first update of surround sound feature take 2: Proof of concept works! OPUS audio got decoded to PCM, re-encoded to AAC-LC on TV and passthrough to the receiver. I can hear sound coming from the correct speaker. However there are more problems that need to be solved:
|
@mariotaku While I've moved onto streaming via NVIDIA Shield, I've still got my modifications to Sunshine for experimentally switching the audio codec to AAC-LC, encoded using the Fraunhofer FDK AAC. I'll see how easy that is to update to the latest Sunshine version and then maybe push the fork so you can try it out for yourself to check if moving the encoding step right to Sunshine helps or if the encoder simply isn't fast enough. |
Good news for everyone - 5.1 surround is really working & with pretty good performance! There are a few limitations:
|
Great to hear. Very exciting. Is a new release on your side required as well? A backport to webOS 4/4.5 would be appreciated. |
@GingerAdonis Hi, I also tried to work on webOS 4, but unfortunately, I didn't find a way to play surround audio with low latency yet (it has more than 1000ms of latency). So I'm unable to support webOS 4 and below at this moment. |
Thank you for giving a shot. :) |
This feature has been released in 1.6.13 and above. Finally it's implemented and thanks everyone for testing & feedback & providing information! |
Great work! One thing that is not entirely clear yet, does this also work with GFE or only with Sunshine? |
It should work with GFE and the current version of Sunshine, with a minor performance degrade. Once my patch for Sunshine is released, the performance will be even better (5ms less latency) |
Should this work out of the box ? I installed 1.6.15, and sunshine's current version (I wanted to test it even though the latency is not fixed yet in it). But, it only shows up as STEREO PCM on my sound bar, even though I set the audio to passthrough on my TV. I even tried to install the steam speaker, and set it to 7.1 or 5.1 without success so far. Thanks a lot for the continued work. I am happy to see such feature arriving to this beautiful port 🙂 |
@Maltique It supposed to work on webOS 5+ if you select "5.1 surround". Can you provide more info? |
When using GFE, I actually get Multichannel PCM (Yay :D). Somehow, when using sunshine it does not work. I may try to reinstall my sunshine or look at tips on the internet. |
Got it. For future discussions about this issue you see, please create a new one, so we can drop more info there. |
My bad here, GFE is always in surround mode whereas Sunshine switches to it only in games. After some more digging I have been able to make it work. Thanks @mariotaku |
So it doesnt work on WebOS 4? I have a LG C9. I can select the option in settings but had no chance to try yet |
@gotwig No, it doesn't work on webOS 4 and I'm unable to find out any method to achieve that. |
@mariotaku I wanted to take a moment to give you my thanks, gratitude, and kudos for fixing surround sound on WebOS 😊 I tested it, and it works perfectly. Latency is imperceptible. With a true 5.1 signal, I feel better about using the upmixing mode on my receiver for the 7.1 effect 😉 |
I would also like to say a big thank you to @mariotaku for this really excellent work!! 😊👍 I've been waiting for this for a long time and now I can finally play my games on the LG in 4K, HDR, 120fps and 5.1 surround sound. |
@astarktc I would like to ask if you are using a soundbar? Because I am currently encountering the same delay problem that I have encountered before. When the TV is set to pass the signal to the soundbar, the surround sound works normally but there is a delay issue |
Half the time I am, half the time I'm not. My LG in the living room has a denon receiver, and the one in my bedroom has a Samsung soundbar. No delay on either 🙂 Is this only happening for you on moonlight, or other content too? One thing you could try is under LG's sound settings -> advanced -> match screen and sound, and try turning on 'Bypass' if it isn't turned on already. |
Moonlight is almost able to handle it, but hard to test yet.
The text was updated successfully, but these errors were encountered: