Phone calls via HomeAssistant

[sorgfresser]

With the rise of LLMs, automating phone calls is a more prominent topic.
There are many different use cases for this. For one, HA could inform the user of any critical errors with the house. In addition to this, HA might be envisioned to start calls autonomously (call doctors, family members in case of emergencies). HA could also take calls in absence, notifying the user of what happened.
I'd love to see this become reality and I am happy to contribute code for this.
I know that the latter two are a long shot, but the first is already possible.

More precisely, existing add-ons and integrations allow calling arbitrary SIP devices with a pre-defined text (for example https://tech7fox.github.io/sip-hass-docs/docs/showcase/call_if_left_open).

While this works, phone calls normally require interaction and short response times. For this, a few architectural changes are certainly needed.
First of all, TTS would require streaming support.
Secondly, Speech to Text does also require some kind of streaming (though since Whisper does not support streaming for example, we could use some voice activity detection based solution similar to whisper.cpp).
And thirdly, I am not aware of any component that could moderate between the two, i.e. receive transcribed speech, generate some text in response, and then pass this text to a streaming TTS (which would have a media player entity = the phone call).

[balloob]

We already have the VoIP integration that does this. I guess it could be split up the sip layer from audio in/out so other integrations can use it.

[sorgfresser]

If I'm not mistaken, the VoIP integration is not meant to actively start calls to a separate device.
So the above use case of calling a family member would not be supported
(Please correct me if this is not the case)

[balloob]

#1143

#1150

[sorgfresser]

I see, thanks! That would be very neat. Sorry, apparently my search was not thorough enough.

The only remaining thing then is the streaming support for TTS, to reduce latency in these conversations.

[sorgfresser]

Closing this as answered - sorry for bothering!
I'd still be happy to discuss the streaming TTS thing though, since this might improve assist latency in general.