Incorrect OPUS negotiation

[admin-toneca]

Describe the bug
FreeSWITCH does not negotiate opus with different rate

To Reproduce
Steps to reproduce the behavior:

1. Make call with OPUS/48000 on A leg and OPUS/16000 on B leg.
2. FreeSWITCH will select PCMU codec

Package version or git hash

• It has happened since version 1.10.10 and has not reproduced on FreeSWITCH 1.10.9.

2023-09-01 17:47:56.424983 98.23% [DEBUG] switch_core_media.c:5526 Audio Codec Compare [opus:102:48000:20:0:1]/[opus:116:48000:20:0:2]
2023-09-01 17:47:56.424983 98.23% [DEBUG] switch_core_media.c:5569 Audio Codec Compare [opus:116:48000:20:0:2] is saved as a near-match
2023-09-01 17:47:56.424983 98.23% [DEBUG] switch_core_media.c:5526 Audio Codec Compare [opus:102:16000:20:0:1]/[PCMU:0:8000:20:64000:1]
2023-09-01 17:47:56.424983 98.23% [DEBUG] switch_core_media.c:5526 Audio Codec Compare [opus:102:16000:20:0:1]/[PCMA:8:8000:20:64000:1]
2023-09-01 17:47:56.424983 98.23% [DEBUG] switch_core_media.c:5526 Audio Codec Compare [PCMU:0:8000:20:64000:1]/[opus:116:48000:20:0:2]
2023-09-01 17:47:56.424983 98.23% [DEBUG] switch_core_media.c:5526 Audio Codec Compare [PCMU:0:8000:20:64000:1]/[PCMU:0:8000:20:64000:1]
2023-09-01 17:47:56.424983 98.23% [DEBUG] switch_core_media.c:5588 Audio Codec Compare [PCMU:0:8000:20:64000:1] ++++ is saved as a match
2023-09-01 17:47:56.424983 98.23% [DEBUG] switch_core_media.c:5526 Audio Codec Compare [PCMU:0:8000:20:64000:1]/[PCMA:8:8000:20:64000:1]
2023-09-01 17:47:56.424983 98.23% [DEBUG] switch_core_media.c:5526 Audio Codec Compare [PCMA:8:8000:20:64000:1]/[opus:116:48000:20:0:2]
2023-09-01 17:47:56.424983 98.23% [DEBUG] switch_core_media.c:5526 Audio Codec Compare [PCMA:8:8000:20:64000:1]/[PCMU:0:8000:20:64000:1]
2023-09-01 17:47:56.424983 98.23% [DEBUG] switch_core_media.c:5526 Audio Codec Compare [PCMA:8:8000:20:64000:1]/[PCMA:8:8000:20:64000:1]
2023-09-01 17:47:56.424983 98.23% [DEBUG] switch_core_media.c:5588 Audio Codec Compare [PCMA:8:8000:20:64000:1] ++++ is saved as a match
2023-09-01 17:47:56.424983 98.23% [DEBUG] switch_core_media.c:5438 Set telephone-event payload to 103@48000
2023-09-01 17:47:56.424983 98.23% [DEBUG] switch_core_media.c:3734 Set Codec sofia/sbc/1867@voip.com PCMU/8000 20 ms 160 samples 64000 bits 1 channels

log with trace.txt
opus.conf.xml.txt

[dragos-oancea]

You need to add enable all used incarnations of Opus codec, eg opus@48000h@1c - for the call on leg A and opus@16000h@1c for the call on leg B. it's choosing PCMA because that's an exact match, not a near-match.
eg, before bridge:

<action application="set" data="absolute_codec_string=opus@48000h@1c,OPUS@16000h@1c,opus,PCMA"/>

"opus" defaults to 48000 khz / 2 channels.

[admin-toneca]

I wouldn't want to use transcoding because FreeSWITCH can negotiate the lower sample rate of both legs and it is worked successfully in version 1.10.9.

It is log of version 1.10.9:

2023-09-04 13:43:56.121525 97.50% [DEBUG] switch_core_media.c:5527 Audio Codec Compare [opus:102:48000:20:0:1]/[opus:116:48000:20:0:2]
2023-09-04 13:43:56.121525 97.50% [DEBUG] switch_core_media.c:5582 Audio Codec Compare [opus:116:48000:20:0:2] ++++ is saved as a match
2023-09-04 13:43:56.121525 97.50% [DEBUG] switch_core_media.c:5527 Audio Codec Compare [opus:102:48000:20:0:1]/[PCMU:0:8000:20:64000:1]
2023-09-04 13:43:56.121525 97.50% [DEBUG] switch_core_media.c:5527 Audio Codec Compare [opus:102:48000:20:0:1]/[PCMA:8:8000:20:64000:1]
2023-09-04 13:43:56.121525 97.50% [DEBUG] switch_core_media.c:5527 Audio Codec Compare [PCMU:0:8000:20:64000:1]/[opus:116:48000:20:0:2]
2023-09-04 13:43:56.121525 97.50% [DEBUG] switch_core_media.c:5527 Audio Codec Compare [PCMU:0:8000:20:64000:1]/[PCMU:0:8000:20:64000:1]
2023-09-04 13:43:56.121525 97.50% [DEBUG] switch_core_media.c:5582 Audio Codec Compare [PCMU:0:8000:20:64000:1] ++++ is saved as a match
2023-09-04 13:43:56.121525 97.50% [DEBUG] switch_core_media.c:5527 Audio Codec Compare [PCMU:0:8000:20:64000:1]/[PCMA:8:8000:20:64000:1]
2023-09-04 13:43:56.121525 97.50% [DEBUG] switch_core_media.c:5527 Audio Codec Compare [PCMA:8:8000:20:64000:1]/[opus:116:48000:20:0:2]
2023-09-04 13:43:56.121525 97.50% [DEBUG] switch_core_media.c:5527 Audio Codec Compare [PCMA:8:8000:20:64000:1]/[PCMU:0:8000:20:64000:1]
2023-09-04 13:43:56.121525 97.50% [DEBUG] switch_core_media.c:5527 Audio Codec Compare [PCMA:8:8000:20:64000:1]/[PCMA:8:8000:20:64000:1]
2023-09-04 13:43:56.121525 97.50% [DEBUG] switch_core_media.c:5582 Audio Codec Compare [PCMA:8:8000:20:64000:1] ++++ is saved as a match
2023-09-04 13:43:56.121525 97.50% [DEBUG] switch_core_media.c:5443 Set telephone-event payload to 103@48000
2023-09-04 13:43:56.121525 97.50% [DEBUG] mod_opus.c:619 Opus encoder: set bitrate to local settings [72000bps]
2023-09-04 13:43:56.121525 97.50% [DEBUG] mod_opus.c:619 Opus encoder: set bitrate to local settings [72000bps]
2023-09-04 13:43:56.121525 97.50% [DEBUG] switch_core_media.c:3750 Set Codec sofia/sbc/1867@voip.com opus/48000 20 ms 960 samples 0 bits 1 channels
2023-09-04 13:43:56.121525 97.50% [DEBUG] switch_core_codec.c:111 sofia/sbc/1867@voip.com Original read codec set to opus:116

[dragos-oancea]

does adding the extra incarnations to the codec string work for you ? In 1.10.9 it was displaying as a match something that was not a real match.

[admin-toneca]

I use the following codec settings:

<param name="codec-prefs" value="OPUS,PCMU,PCMA,G722,GSM"/>
<param name="inbound-codec-negotiation" value="generous"/>
<param name="inbound-codec-prefs" value="OPUS,PCMU,PCMA,G722,GSM,VP8,H264"/>
<param name="inbound-late-negotiation" value="true"/>
<param name="inherit_codec" value="true"/>
<param name="outbound-codec-prefs" value="OPUS,PCMU,PCMA,G722,GSM"/>

[dragos-oancea]

so does this work for you , is Opus being selected ?

<param name="codec-prefs" value="OPUS,OPUS@48000h@1c,OPUS@16000h@1c, PCMU,PCMA,G722,GSM"/>
<param name="inbound-codec-negotiation" value="generous"/>
<param name="inbound-codec-prefs" value="OPUS,OPUS@48000h@1c,OPUS@16000h@1c,PCMU,PCMA,G722,GSM,VP8,H264"/>
<param name="inbound-late-negotiation" value="true"/>
<param name="inherit_codec" value="true"/>
<param name="outbound-codec-prefs" value="OPUS,OPUS@48000h@1c,OPUS@16000h@1c,PCMU,PCMA,G722,GSM"/>

[admin-toneca]

so does this work for you , is Opus being selected ?

<param name="codec-prefs" value="OPUS,OPUS@48000h@1c,OPUS@16000h@1c, PCMU,PCMA,G722,GSM"/>
<param name="inbound-codec-negotiation" value="generous"/>
<param name="inbound-codec-prefs" value="OPUS,OPUS@48000h@1c,OPUS@16000h@1c,PCMU,PCMA,G722,GSM,VP8,H264"/>
<param name="inbound-late-negotiation" value="true"/>
<param name="inherit_codec" value="true"/>
<param name="outbound-codec-prefs" value="OPUS,OPUS@48000h@1c,OPUS@16000h@1c,PCMU,PCMA,G722,GSM"/>

It doesn't help, OPUS is not being selected.

[VideoFX]

I am having this happen after updating 1.10.11 (from 1.10.9). Opus is no longer working with MicroSIP client due to a "near-match". Now, it will only choose PCMU. This has only changed with MicroSIP as the caller. Grandstreams seem to be OK matching with opus.

Some points:

• It worked fine in 1.10.9, like @admin-toneca mentioned.
• I tried the incarnations and settings you mentioned @dragos-oancea , but it didn't help.

2023-12-24 01:58:10.608325 100.00% [DEBUG] switch_core_media.c:5526 Audio Codec Compare [opus:96:48000:20:0:2]/[opus:116:48000:20:0:1]
2023-12-24 01:58:10.608325 100.00% [DEBUG] switch_core_media.c:5569 Audio Codec Compare [opus:116:48000:20:0:1] is saved as a near-match

[dragos-oancea]

There are upcoming patches upon the topic. If we don't take this route with the exact match, power features like eavesdrop won't work with the rate mismatch (eg: in an eavesdrop scenario with 3 calls, each with different sample rate).
Microsip uses opus@24khz.

[denyspozniak]

Hey!
I have a similar problem, if the FS receives 200 OK/SDP with OPUS fmtp line like below from Yealink, then it sends a BYE immediately.

---
SIP/SDP INVITE (FreeSWITCH -> Yealink):

m=audio 24244 RTP/AVP 9 0 8 102 101 103
a=rtpmap:9 G722/8000
a=rtpmap:0 PCMU/8000
a=rtpmap:8 PCMA/8000
a=rtpmap:102 opus/48000/2
a=fmtp:102 useinbandfec=1; maxaveragebitrate=30000; maxplaybackrate=48000; ptime=20; minptime=10; maxptime=40
a=rtpmap:101 telephone-event/8000
a=fmtp:101 0-15
a=rtpmap:103 telephone-event/48000
a=fmtp:103 0-15
a=silenceSupp:off - - - -
a=ptime:20

---
200 OK/SDP (Yealink -> FreeSWITCH):

m=audio 43272 RTP/AVP 102 101
a=rtpmap:102 opus/48000/2
a=fmtp:102 sprop-maxcapturerate=16000; maxaveragebitrate=20000; maxplaybackrate=48000; useylrtx=1; useinbandfec=1
a=rtpmap:101 telephone-event/8000
a=fmtp:101 0-15
a=sendrecv
a=rtcp:43273
a=ptime:20
a=direction:both

---
SIP BYE (FreeSWITCH -> Yealink):

Reason: Q.850;cause=88;text="INCOMPATIBLE_DESTINATION"

---
fs_cli>
2024-04-17 09:18:23.592898 96.00% [DEBUG] sofia.c:7503 Remote SDP:
v=0
o=- 20000 20000 IN IP4 127.0.0.1
s=SDP data
c=IN IP4 127.0.0.1
t=0 0
m=audio 43272 RTP/AVP 102 101
a=rtpmap:102 opus/48000/2
a=fmtp:102 sprop-maxcapturerate=16000; maxaveragebitrate=20000; maxplaybackrate=48000; useylrtx=1; useinbandfec=1
a=rtpmap:101 telephone-event/8000
a=fmtp:101 0-15
a=rtcp:43273
a=ptime:20
a=direction:both

2024-04-17 09:18:23.592898 96.00% [DEBUG] sofia.c:7493 Channel sofia/internal/100@127.0.0.1 entering state [ready][200]
2024-04-17 09:18:23.592898 96.00% [DEBUG] switch_core_media.c:5526 Audio Codec Compare [opus:102:48000:20:0:1]/[G722:9:8000:20:64000:1]
2024-04-17 09:18:23.592898 96.00% [DEBUG] switch_core_media.c:5526 Audio Codec Compare [opus:102:16000:20:0:1]/[PCMU:0:8000:20:64000:1]
2024-04-17 09:18:23.592898 96.00% [DEBUG] switch_core_media.c:5526 Audio Codec Compare [opus:102:16000:20:0:1]/[PCMA:8:8000:20:64000:1]
2024-04-17 09:18:23.592898 96.00% [DEBUG] switch_core_media.c:5526 Audio Codec Compare [opus:102:16000:20:0:1]/[opus:116:48000:20:0:1]
2024-04-17 09:18:23.592898 96.00% [DEBUG] switch_core_media.c:5438 Set telephone-event payload to 101@8000
2024-04-17 09:18:23.592898 96.00% [DEBUG] switch_core_media.c:5856 sofia/internal/100@127.0.0.1 Set 2833 dtmf send payload to 101 recv payload to 101
2024-04-17 09:18:23.592898 96.00% [NOTICE] sofia.c:8702 Hangup sofia/internal/100@127.0.0.1 [CS_CONSUME_MEDIA] [INCOMPATIBLE_DESTINATION]
2024-04-17 09:18:23.592898 96.00% [DEBUG] sofia.c:1527 Channel is already hungup.
2024-04-17 09:18:23.592898 96.00% [DEBUG] switch_core_state_machine.c:581 (sofia/internal/100@127.0.0.1) Running State Change CS_HANGUP (Cur 2 Tot 649)
2024-04-17 09:18:23.592898 96.00% [DEBUG] switch_core_state_machine.c:844 (sofia/internal/100@127.0.0.1) Callstate Change RINGING -> HANGUP

FreeSWITCH Version 1.10.11-release+git-20240227T064603Z~d766b7da95~64bit

[ruddj]

I am experiencing the same Near-Match issue as well. Works on 1.10.9 but not on 1.10.11
The OPUS codecs match exactly apart from dynamic port but still marked as near-match. Both are Opus 48khz using 20 ms ptime with 1 channel

FreeSWITCH Version 1.10.11-release-25-f24064f7c9~64bit

13:27:54.104775 99.70% [DEBUG] switch_core_media.c:5526 Audio Codec Compare [opus:112:48000:20:0:1]/[opus:116:48000:20:0:1]
13:27:54.104775 99.70% [DEBUG] switch_core_media.c:5569 Audio Codec Compare [opus:116:48000:20:0:1] is saved as a near-match
13:27:54.104775 99.70% [DEBUG] switch_core_media.c:5526 Audio Codec Compare [opus:112:16000:20:0:1]/[opus:116:48000:20:0:1]
13:27:54.104775 99.70% [DEBUG] switch_core_media.c:5569 Audio Codec Compare [opus:116:48000:20:0:1] is saved as a near-match
13:27:54.104775 99.70% [DEBUG] switch_core_media.c:5526 Audio Codec Compare [opus:112:16000:20:0:1]/[G722:9:8000:20:64000:1]
13:27:54.104775 99.70% [DEBUG] switch_core_media.c:5526 Audio Codec Compare [opus:112:16000:20:0:1]/[PCMA:8:8000:20:64000:1]
13:27:54.104775 99.70% [DEBUG] switch_core_media.c:5526 Audio Codec Compare [opus:112:16000:20:0:1]/[PCMU:0:8000:20:64000:1]
13:27:54.104775 99.70% [DEBUG] switch_core_media.c:5526 Audio Codec Compare [G722:9:8000:20:64000:1]/[opus:116:48000:20:0:1]
13:27:54.104775 99.70% [DEBUG] switch_core_media.c:5526 Audio Codec Compare [G722:9:8000:20:64000:1]/[opus:116:48000:20:0:1]
13:27:54.104775 99.70% [DEBUG] switch_core_media.c:5526 Audio Codec Compare [G722:9:8000:20:64000:1]/[G722:9:8000:20:64000:1]
13:27:54.104775 99.70% [DEBUG] switch_core_media.c:5588 Audio Codec Compare [G722:9:8000:20:64000:1] ++++ is saved as a match
13:27:54.104775 99.70% [DEBUG] switch_core_media.c:5526 Audio Codec Compare [G722:9:8000:20:64000:1]/[PCMA:8:8000:20:64000:1]
13:27:54.104775 99.70% [DEBUG] switch_core_media.c:5526 Audio Codec Compare [G722:9:8000:20:64000:1]/[PCMU:0:8000:20:64000:1]

Here is the same connection on FreeSWITCH Version 1.10.9-release~64bit (-release 64bit). This marks it as a match.

14:06:30.532685 96.77% [DEBUG] switch_core_media.c:5527 Audio Codec Compare [opus:112:48000:20:0:1]/[opus:116:48000:20:0:1]
14:06:30.532685 96.77% [DEBUG] switch_core_media.c:5582 Audio Codec Compare [opus:116:48000:20:0:1] ++++ is saved as a match
14:06:30.532685 96.77% [DEBUG] switch_core_media.c:5527 Audio Codec Compare [opus:112:48000:20:0:1]/[G722:9:8000:20:64000:1]
14:06:30.532685 96.77% [DEBUG] switch_core_media.c:5527 Audio Codec Compare [opus:112:48000:20:0:1]/[PCMA:8:8000:20:64000:1]
14:06:30.532685 96.77% [DEBUG] switch_core_media.c:5527 Audio Codec Compare [opus:112:48000:20:0:1]/[PCMU:0:8000:20:64000:1]
14:06:30.532685 96.77% [DEBUG] switch_core_media.c:5527 Audio Codec Compare [G722:9:8000:20:64000:1]/[opus:116:48000:20:0:1]
14:06:30.532685 96.77% [DEBUG] switch_core_media.c:5527 Audio Codec Compare [G722:9:8000:20:64000:1]/[G722:9:8000:20:64000:1]
14:06:30.532685 96.77% [DEBUG] switch_core_media.c:5582 Audio Codec Compare [G722:9:8000:20:64000:1] ++++ is saved as a match
14:06:30.532685 96.77% [DEBUG] switch_core_media.c:5527 Audio Codec Compare [G722:9:8000:20:64000:1]/[PCMA:8:8000:20:64000:1]
14:06:30.532685 96.77% [DEBUG] switch_core_media.c:5527 Audio Codec Compare [G722:9:8000:20:64000:1]/[PCMU:0:8000:20:64000:1]

[shaunjstokes]

We have the same issue, Opus was working with no issue on 10.10.9, we're now getting near-match even after adding the extra incarnations to the codec string. The issue is resolved after applying patch #2582

[shaunjstokes]

After further testing we've found that the patch #2582 has the side effect of causing calls that fail to negotiate the same codec as Leg A for Leg B to fail with SIP 488 INCOMPATIBLE_DESTINATION. That's not an option for us, we need backwards compatibility with other codecs, some calls must be transcoded.

[shaunjstokes]

I've found a solution, rather than apply this for all codecs it should only apply for Opus.

Calls using Opus and other codecs establish correctly on both legs, and there are no issues with codec negotiation when the codecs on Leg A and Leg B don't match.

Patch attached. I'll create a PR when I have more time.
2226.patch