You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
On mobile devices with multiple IP addresses STUN connectivity checks are generated with wrong source addresses, causing network drops due to ISP anti-spoofing
#769
Open
juhaszp-uhu opened this issue
Nov 29, 2022
· 2 comments
Describe the bug
On some Android phones, and on certain 4G networks the network connection itself becomes unstable and keeps dropping while an Openvidu call is in progress.
Expected behavior
No network issues and interruption-free videocall
Wrong current behavior
Unusable video call, the ISP drops the network connection every few seconds.
That is,
on certain devices (Samsung S9, Samsung Galaxy A7, xiaomi mi note 10, but not on an OnePlus)
in certain browsers (Chrome)
on certain 4G networks (Vodafone HU), with certain SIMs
with a specific WebRTC video application (OpenVidu)
the connection keeps dropping to the point of unusability. And it's not just the WebRTC connection: the entire network interface goes down for a few seconds. It reconnects after a brief pause, then the application recovers, there is a burst of video for a few seconds, and the connection dies again.
We've got a hint from the ISP that it is their IP anti-spoofing rule that causes the disconnections. Apparently the affected devices have two IP addresses, one in the 100.x.x.x range and the other in the 10.x.x.x range. One of them is the "regular" data connection and the other is to the IMS APN. We can see from a packet dump produced during an affected WebRTC call that even though the connection has been established through the 100.x.x.x address, Chrome keeps sending STUN connectivity check packets (binding requests) from both that address and from the other 10.x.x.x address. And, apparently, with this ISP, this is not allowed, because the source address of these packet doesn't match the subscriber's IP address, so after the number of these (according to them) mis-addressed packets reaches a threshold, they kill the entire connection.
OpenVidu tutorial where to replicate the error
This was observed on https://demos.openvidu.io/openvidu-call/#/ too. Just create a room there and join with two clients (from which at least one is affected).
OpenVidu deployment info
Not relevant, no local servers were used.
Client device info (if applicable)
Chrome 103.0.5060.71 on xiaomi mi note 10 - android 11
I've even tried to patch the amalgamated js file in node_modules to filter the offending ICE candidates in RTCPeerConnection's icecandidate event handler, but it didn't help: from the server logs it seems that those candidates with the 10.x addresses were not sent, but the client still kept disconnecting.
It is important to note that other WebRTC applications are NOT affected. E.g. https://janus.conf.meetecho.com/echotest.html works without network interruptions. So even if it is a browser or OS bug, there is something that Openvidu does differently, and it would be great to find out what exactly, and if possible, fix it.
The text was updated successfully, but these errors were encountered:
Followup: it turns out that it is a general property of Linux (hence, all Android phones) that it sends UDP packets on the primary network interface (in case there are more than one active interfaces), regardless of the source address. More precisely, if an UDP socket is bound to a local source address, the packets sent on that socket will have that source address, but the kernel will still route the packets on the primary interface, even when the source address doesn't match.
And the ICE layer in browsers generates STUN binding requests (= UDP packets) from all active local addresses, in order to find the best candidate. It appears that all browsers on all operating systems do this with all webrtc applications, only the number and rate of the packets differ somewhat, and there is no clean way to influence those either.
The only workaround we've found is to use a TURN server, AND use the iceTransportPolicy: 'relay' option in the RTCPeerConnection constructor. Unfortunately, there is currently no option in Openvidu to enable this, so we had to resort to patching the openvidu-browser component.
So this is now also a feature request to add this option to Openvidu, to be able to set iceTransportPolicy. Should I open a new issue for that?
Describe the bug
On some Android phones, and on certain 4G networks the network connection itself becomes unstable and keeps dropping while an Openvidu call is in progress.
Expected behavior
No network issues and interruption-free videocall
Wrong current behavior
Unusable video call, the ISP drops the network connection every few seconds.
That is,
the connection keeps dropping to the point of unusability. And it's not just the WebRTC connection: the entire network interface goes down for a few seconds. It reconnects after a brief pause, then the application recovers, there is a burst of video for a few seconds, and the connection dies again.
We've got a hint from the ISP that it is their IP anti-spoofing rule that causes the disconnections. Apparently the affected devices have two IP addresses, one in the 100.x.x.x range and the other in the 10.x.x.x range. One of them is the "regular" data connection and the other is to the IMS APN. We can see from a packet dump produced during an affected WebRTC call that even though the connection has been established through the 100.x.x.x address, Chrome keeps sending STUN connectivity check packets (binding requests) from both that address and from the other 10.x.x.x address. And, apparently, with this ISP, this is not allowed, because the source address of these packet doesn't match the subscriber's IP address, so after the number of these (according to them) mis-addressed packets reaches a threshold, they kill the entire connection.
OpenVidu tutorial where to replicate the error
This was observed on https://demos.openvidu.io/openvidu-call/#/ too. Just create a room there and join with two clients (from which at least one is affected).
OpenVidu deployment info
Not relevant, no local servers were used.
Client device info (if applicable)
Screenshots
N/A
Additional context
This is a followup to #741, but some additional understanding of the problem was gained since then.
Also reported to Chromium at https://bugs.chromium.org/p/chromium/issues/detail?id=1393306 , because it might very well be a Chrome (or even Android kernel) bug.
I've even tried to patch the amalgamated js file in node_modules to filter the offending ICE candidates in RTCPeerConnection's icecandidate event handler, but it didn't help: from the server logs it seems that those candidates with the 10.x addresses were not sent, but the client still kept disconnecting.
It is important to note that other WebRTC applications are NOT affected. E.g. https://janus.conf.meetecho.com/echotest.html works without network interruptions. So even if it is a browser or OS bug, there is something that Openvidu does differently, and it would be great to find out what exactly, and if possible, fix it.
The text was updated successfully, but these errors were encountered: