[Main UI] Add support for webaudio

[wborn]

Fixes #743

[relativeci]

Job #459: Bundle Size — 10.78MB (+0.01%).

981dc61 vs a70b965

Changed metrics (2/10)

Metric	Current	Baseline
Initial JS	1.68MB(+0.05%)	1.68MB
Cache Invalidation	18.82%	26.53%

Changed assets by type (1/7)

	Current	Baseline
JS	8.72MB (+0.02%)	8.72MB

---

View Job #459 report on app.relative-ci.com

[ghys]

Thanks for adding this feature.
I'm not particularly happy that it requires the maintenance of a permanent dedicated EventSource connection (something other UIs like HABPanel or Paper UI do/did anyway so it was less of a overhead) provided if it doesn't end up having adverse implications, like running out of the 6 max HTTP/1.1 connections per server - not per tab!

But I'm open to have it out of the interest of having this feature working again for those who need it.

[wborn]

like running out of the 6 max HTTP/1.1 connections per server - not per tab

Yes I also don't like the creation of an additional event source but there currently does not seem to be an easy way around it. Some infrastructure that allows for reusing / updating SSE subscription might help. Switching to websockets might also be an option (openhab/openhab-core#2891). It would also be nice if we can get HTTP/2 working with Jetty. 🙂

[ghys]

Some infrastructure that allows for reusing / updating SSE subscription might help. Switching to websockets might also be an option (openhab/openhab-core#2891). It would also be nice if we can get HTTP/2 working with Jetty. 🙂

I'm all for switching to websockets as a mid/long-term goal since EventSource seems all but abandoned as a technology (too bad because they were a good fit for pushing events with little overhead), see the difficulties in passing authorization headers, we have to rely on a polyfill even when there's a native implementation.

I tried to limit the number of SSE connections given this limitation, usually you would only have one for events (in settings pages) and one for item state updates. It means you can open 3 openHAB tabs at a maximum (never mind other UIs) before running into trouble.

[ghys]

@wborn just had an idea.
There are a number of "device/browser-local" options on the Help & About screen in the main UI:

Those normally are focused only on appearance (hence the name of the Vue component: https://github.com/openhab/openhab-webui/blob/main/bundles/org.openhab.ui/web/src/components/theme-switcher.vue) and they're backed by local storage.

We could expand these local options to more than appearance, including maybe a "Enable Web Audio sink support" option (which I'd like to remain disabled by default to save a precious SSE connection), so users could eventually enable it on select devices, that are perhaps dedicated to displaying the UI, and thus won't be at risk of running out of these 6 connections.

Then I'd have no more objections to adding this feature because it would be something that despite its cost is something the user can control and opt-in to.

[wborn]

Having an option to disable it is always useful, as you may not want to use webaudio on each and every device running an OH UI.
🔈 🙂

[wborn]

Config option added. ✔️

[ghys]

Having an option to disable it is always useful, as you may not want to use webaudio on each and every device running an OH UI.
🔈 🙂

Yes, exactly!

And that way this has no impact on the number of working simultaneous openHAB tabs on a given browser (the main UI uses up to 2 SSE connections per tab, one /rest/events/states and one /rest/events in settings pages) in the default scenario, when the option is left alone.

Perfect, thanks, glad to have this feature back and running.

[stefan-hoehn]

@wborn Would you mind providing a bit of description how to use the feature here as I am eager to try it out? I'd be happy to add this to the official documentation then.

[wborn]

Hi @stefan-hoehn!

This PR makes it possible to play any audio sent to the webaudio sink in the UI.

To enable webaudio in the UI first switch it on using the new config option:

Then you can quickly test if it works by playing some static content that comes with openHAB in the /sounds dir using a console command:

openhab> audio play webaudio barking.mp3

If you have a dog that gets triggered by barking you may opt for playing doorbell.mp3 instead. If you play doorbell.mp3 your dog will then probably play its own barking audio sound. 😉

Note that most browsers block web pages from playing audio/video automatically nowadays. You first need to interact with the webpage before it will allow audio playback. But browsers allow for overriding this per website using some settings:

These are some useful actions available to rules in which you can specify webaudio as sink, see:

• https://www.openhab.org/docs/configuration/multimedia.html#actions-2
  playSound("webaudio", "barking.mp3")

• https://www.openhab.org/docs/configuration/multimedia.html#actions-3
  say("Hello world!", "voicerss:enUS", "webaudio")

You can also configure webaudio as sink when creating rules in the UI:

If you really like webaudio, you can also make it the default sink in the "Audio" settings. Then you can omit the webaudio sink ID from commands/actions. 🙂

[stefan-hoehn]

I will try it out as soon as it has become part of the first milestone. I am not brave enough to install the snapshot atm ;-)

[stefan-hoehn]

I can finally confirm that webaudio works fine. Very nice.

One minor thing to mention is that it needs to be switched on like described above in each browser that runs the UI (which of course totally makes sense). It is therefore a client-side UI setting.

And it even works nicely with blocklies

[stefan-hoehn]

However, @wborn , it seems that streaming doesn't work. Did you try this out? Is it supposed to work?

var audio = Java.type('org.openhab.core.model.script.actions.Audio');
audio.playStream('webaudio', 'https://liveradio.swr.de/sw282p3/swr3/play.mp3');`

[wborn]

I didn't test streaming but it seems to cause CORS errors. It also doesn't work for the same reason when using HABPanel.

[stefan-hoehn]

Yes, you are right, I checked it.

As the streaming happens from the browser directly (and not via openHAB-server) the browser throws a CORRS exception. The only complex way would be if there would be a proxy sitting at the openHAB server that the UI would talk to (same server as the UI was loaded from) and then forward that to the real streaming url I am sure we will do that as part of openHAB. Or in other words: streaming should work if the streaming url has the same server like the openHAB server.

[stefan-hoehn]

@wborn I was actually just thinking to document that functionality but it actually already starts that not even the about page is currently documented. Would that makes sense first and if, where would that page documenation go?

e.g. here: https://www.openhab.org/docs/tutorial/first_steps.html ?
or rather an "About"- subpage under https://www.openhab.org/docs/configuration/

I would start writing/documenting it but I have a few questions

@ghys

Can you help with these settings? I tried to research it in the forum or the docs or by looking into the could but could find a description that would make me understand it.

btw, what URL is called the "homepage"? / or something else?

Even though I tried to find out by looking into the code I am not sure about the behaviour.

a) Simple navigation bar on home page
I don't see anything appearing when I switch on. I am sure I just overlook it.

b) Standard home page background color

and regarding the other info I even don't find a good place where to dokument that because the whole settings part is kind of not documented in the way it is structured in the UI. Seems I am opening pandoras box here... :)

What color is applied? It is not clear to me why this is on / off. I can see in the code that it chooses "default" or "standard" what what is the difference? (I don't see any)

cc: @Confectrician

[wborn]

I was actually just thinking to document that functionality but it actually already starts that not even the about page is currently documented. Would that makes sense first and if, where would that page documenation go?

I don't maintain the docs, so I think it is best for the @openhab/documentation-maintainers to answer where they'd like to see docs for main UI specific configuration.

[ghys]

a) Simple navigation bar on home page
I don't see anything appearing when I switch on. I am sure I just overlook it.

It means the navigation bar will look the same even whether you're at the top of the page or not - you won't have the alternate look with the date.

b) Standard home page background color
and regarding the other info I even don't find a good place where to dokument that because the whole settings part is kind of not documented in the way it is structured in the UI. Seems I am opening pandoras box here... :)

This is only for the light color scheme. The background color will be the same as the regular pages you can show from e.g. the sidebar, which is a kind of gray instead of white.

[stefan-hoehn]

Thanks, @ghys

FYI: for that reason I have started a new contribution for all the openhab settings here that we can jointly fill over the period of time partially with new content that hasn't been written yet and partially by linking to documentation that is already available in the documentation:

[stefan-hoehn]

@wborn I finally found the time to document it at openhab/openhab-docs#1892

[dilyanpalauzov]

My understanding is that to enable WebAudioSink for BasicUI on all devices, one has to go to http://…/settings/services/org.openhab.basicui and enable it. But for MainUI the WebAudioSink has to be enabled per device by visiting Help → About → “Enable Web Audio sink support”.

This makes no sense. Either the option shall be enabled for all devices, or per device, but not for Main UI — per device, while for Basic UI — for all devices from one place.

Moreover I would like to be able to set the per-device defaults on one place. That is configure once that webaudiosink in ON, and then from all places (browsers, devices) where I load the openHAB webpages, the webaudio will work.