Call fail using latest sip.js in ionic app

[MarwaAbuEssa]

YOU MUST read first!

Please use Community Forum for general technical discussions and questions.

• I have used Google with the error message or bug in association with the library and Cordova words to make sure the issue I'm reporting is only related to iOSRTC.
• I have provided steps to reproduce (e.g. using sample app code https://github.com/cordova-rtc/cordova-plugin-iosrtc-sample or updated extra/renderer-and-libwebrtc-tests.js file).
• I have provided versions of third party library name, ios, Xcode and plugin version and adapter.js version if used.

Versions affected

• Capacitor 3 iOS Platform version (15):
• Plugin version (latest):
• iOS version (e.g 15.0.1):
• Xcode version (e.g13):
• WebRTC Framework version (e.g. SIPJS 20):

Description

HI ,

our VOIP app (ionic 5 and capacitor) using sipjs v20 and there is many issues using webrtc with callkit (specially with audio stream locked by ios callkit)

I thought to use your plugin to produce my ios app version and adopt callkit with it .

I installed the plugin successfully , did global registration , enumerate devices and get media stream.
my issues are :
1- since upgrade to ios 15 here is an issue with audio tracks (although i can get the input device) and i do think this is related to upgrade to v15
2- how can integrate your plugin with sipjs , I means should should i do the invite from spjs but get the stream and session description from your plugin ?? I need some clear examples about appropriate way to use the plugin with sipjs and also if the plugin is working with ios callkit

Thanks in advance , your help is really appreciated as I have been long time investigating with ios and webrtc

Marwa

[MarwaAbuEssa]

HI ,
Thanks a lot for your support , I added the Plugin and use registerGlobal but when I do an outgoing call (ios15) , it dropps immediately with the following error

at the invite of sipjs I'm using the following constraint

let inviteOptions:SessionInviteOptions = {
sessionDescriptionHandlerOptions: {
constraints: { audio: true, video: false} }
};

await inviter.invite(inviteOptions);

the same happens when answering incoming call

the app is not prompt for mic permission although it prompt for camera permission .I have a way to give the app permission to access the microphone but not prompt for using it .
any help ?
Marwa

[hthetiot]

HI , Thanks a lot for your support , I added the Plugin and use registerGlobal but when I do an outgoing call (ios15) , it dropps immediately with the following error

at the invite of sipjs I'm using the following constraint

let inviteOptions:SessionInviteOptions = { sessionDescriptionHandlerOptions: { constraints: { audio: true, video: false} } };

await inviter.invite(inviteOptions);

the same happens when answering incoming call

the app is not prompt for mic permission although it prompt for camera permission .I have a way to give the app permission to access the microphone but not prompt for using it . any help ? Marwa

This error has nothing to do with GetUserMedia, you SDP is rejected most likely because of a configuration issue in you SIP server.

[hthetiot]

I cannot help you more sorry this getOffer rejected come from your usage of the library or configuration of your SIP server

[MarwaAbuEssa]

Thanks a lot appreciate your help

[RaddishIoW]

@MarwaAbuEssa I am having exactly the same problem you are in the screenshot above... did you find out what was causing it, or have any pointers for a solution?

[MarwaAbuEssa]

Hi @RaddishIoW

I figured out the cause of the issue that sipjs uses MediaStreamTrackEvent datatype to dispatch event on addtrack/removetrack function which is different type of event returned from iosrtc . I solved it by doing 2 line modification in sip

1- go to file node_modules\sip.js\lib\platform\web\session-description-handler.js and replace the following

static dispatchAddTrackEvent(stream, track) { stream.dispatchEvent(new MediaStreamTrackEvent("addtrack", { track })); } to static dispatchAddTrackEvent(stream, track) { stream.dispatchEvent(new Event("addtrack", { track })); }
and

static dispatchRemoveTrackEvent(stream, track) { stream.dispatchEvent(new MediaStreamTrackEvent("removetrack", { track })); } to static dispatchRemoveTrackEvent(stream, track) { stream.dispatchEvent(new Event("removetrack", { track })); }

I know that this solution is improper but I had been so busy lately to change it. hopefully, this can help you. Thanks

[hthetiot]

Thank you @MarwaAbuEssa,

May be we can mitigate that on cordova side somehow by updating iosrtc.

PR is welcome 🙏 alternatively I might look at it next time I allocate time to iosrtc.

I will leave it open for now so it can be found more easily. I will hide non relevant comments and update titles to help issue context.

[RaddishIoW]

@MarwaAbuEssa Thanks so much for that!

I know that sip.js has a mechanism for creating your own SessionDescriptionHandlers, so I could probably create a SessionDescriptionHandler specifically for Cordova with this change in it, using the standard web one as a starting point.

[hthetiot]

@RaddishIoW thx

Would be great if we can update the jssip/sip.js iosrtc example app (https://github.com/cordova-rtc/cordova-plugin-iosrtc-sample) with this custom session Description handler thing.

See jssip example that you may duplicate for sip.js only that require to update www/index.html to use index-jssip.js instead of index-local.js

Index-jssip file for reference:
https://github.com/cordova-rtc/cordova-plugin-iosrtc-sample/blob/master/www/js/index-jssip.js

Like that I can test for regression on each release. In the past this sample worked for both depending what jsSipUrl value was (sip.js or jssip cdn url) because sip.js is a fork of the much more stable jssip, it had to broke when they updated, it was working for sip.js version 0.15.6 at the time.

Note: Edited for typos and better context and wording.

[RaddishIoW]

Doing some digging on this... The MediaStreamTrackEvent event datatype is defined in the WebRTC APIs.
https://developer.mozilla.org/en-US/docs/Web/API/MediaStreamTrackEvent

I'm probably missing something, but shouldn't iosrtc support this event datatype?

It would be possible to define a SessionDescriptionHandler with the changes above, but unfortunately the methods are marked private, so subclassing the platform/web/session-description-handler class won't work, as we can't override the methods... so we would have to duplicate the whole class' code in order to change those dispatch methods - which isn't ideal.

Thoughts?

[hthetiot]

Doing some digging on this... The MediaStreamTrackEvent event datatype is defined in the WebRTC APIs.
https://developer.mozilla.org/en-US/docs/Web/API/MediaStreamTrackEvent

I'm probably missing something, but shouldn't iosrtc support this event datatype?

Iosrtc is a SHIM, we can try define this event "type" or even try to use it if webrtc is available as it currently use native Event I'm not sure why it does not work. As any event can be dispatch like that in browser (you dont need MouseEvent to trigger click for example https://developer.mozilla.org/en-US/docs/Web/Events/Creating_and_triggering_events).

They may be checking somewhere the type some how using constructor.name. None other client webrtc library does that to my knowledge, might be more easy to fix on their side.

On iosrtc side change would need to be made there:

• cordova-plugin-iosrtc/js/MediaStream.js

  Line 448 in a502fcc

  function onEvent(data) {

And we would need something like (not tested) to declare this CustomEvent.

const MediaStreamTrackEvent = window.MediaStreamTrackEvent || function(type, eventInitDict) {
    this.type = type.toString();
    this.track = eventInitDict.track;
    return Event.prototype.constructor.call(this, type, eventInitDict);
}

Object.defineProperties(
	MediaStreamTrackEvent,
	Object.getOwnPropertyDescriptors(Event.prototype)
);

MediaStreamTrackEvent.prototype.constructor = MediaStreamTrackEvent;

I'm curious doh, can we confirm you load iosrtc registerGlobal BEFORE sip.js load, because that prevent bad reference to be made in sip.js library (see index-jssip.js referenced earlier) this as solve such problems in the past for twilio library.

[MarwaAbuEssa]

Thank you @MarwaAbuEssa,

May be we can mitigate that on cordova side somehow by updating iosrtc.

PR is welcome 🙏 alternatively I might look at it next time I allocate time to iosrtc.

I will leave it open for now so it can be found more easily. I will hide non relevant comments and update titles to help issue context.

Thank @hthetiot for reopening it . yes update iosrtc Is more suitable .I will try to do my best

[MarwaAbuEssa]

Hi @RaddishIoW

sipjs allows to create a sessiondescriptionhandler but with your custom mediastreamfactory .so mysolution was to use iosrtc mediastream and provide it to sipjs . you can find override how local media section at the following link

https://github.com/onsip/SIP.js/blob/main/docs/session-description-handler.md

of-course you need to define iosrtc globally in your project first and do the mediastreamevent modification too.

Regards

[hthetiot]

exposed MediaStreamTrackEvent via
via b5a3bb0

will be release on 8.0.4

[hthetiot]

Released into cordova-plugin-iosrtc@8.0.4 , can you test to confirm your fix please, thx.

https://github.com/cordova-rtc/cordova-plugin-iosrtc/blob/master/CHANGELOG.md#version-804

@MarwaAbuEssa @RaddishIoW

[hthetiot]

Ping @MarwaAbuEssa @RaddishIoW pls test 8.0.4