Native SDK for Android / iOS

[ibc]

Given that mediasoup v2 uses a mediasoup protocol the native SDK for Android and iOS should implement that protocol.

Help, ideas and proposal are welcome.

[eirslett]

I started some work on a C++ based native client: https://github.com/eirslett/mediasoup-client-native-cpp

[ibc]

This is great!

[ibc]

NOTE: I've written libsdptransform (C++ version of the sdp-transform JavaScript library exposing the same API).

Since mediasoup-client uses sdp-transform, the corresponding libmediasoupclient (the C++ version) could then use libsdptransform.

[laowangnj]

Mediasoup protocol has been simple enough. But it would be better if the raw sdp from browser can be accepted as data.

[ibc]

No. Transmitting SDP is never "better" since different browsers use different SDP semantics (PlanB, UnifiedPlan) and ORTC browsers (Edge) do not use SDP at all. So no SDP in the wire. Never again.

[laowangnj]

Why not adapt sdp or ortc arguments in the server or room?

[ibc]

Because I prefer to do that in each client, which is the proper place to discover the device/browser vendor, version and capabilities. This is working and this is not gonna change.

For further discussions about this, please use the mailing list. This is a project task about the mediasoup-client native SDK.

[eirslett]

I had an idea to make a "translation server" in node.js that would speak SDP over the wire and translate to mediasoup's ORTC stuff before interacting with the mediasoup server. The main benefit would be that native clients could speak SDP to the server, so one wouldn't have to reimplement the SDP translation logic in another language (C/C++). Another option is to bundle a JavaScript engine with the native client, and run the sdp/ortc conversion inside that. The third option is to port everything from JS to C/C++, but that's a lot of work, and quite error-prone. @ibc I tried the last option, rewriting everything (I didn't finish, but got quite far), one of the main problems was lack of a decent unit test suite - if there were unit tests, it would be much more straight-forward to port the tests, then port the implementation, which would verify that there are no regressions introduced. (C++ is much more picky about null/undefined etc. than JS, so there were a couple of issues regarding edge cases.)

[ibc]

The main benefit would be that native clients could speak SDP to the server, so one wouldn't have to reimplement the SDP translation logic in another language (C/C++).

Sending and receiving a SDP is not enough to express all the possible actions. For example, you cannot "send a SDP to the other party" to tell him which video simulcast stream ("profile" in mediasoup API) you wish to receive.

Another option is to bundle a JavaScript engine with the native client, and run the sdp/ortc conversion inside that.

I don't think that Java (in Android) or Swift (in iOS) can call JS code. Anyway, this approach is already working when using React Native with mediasoup-client (JS lib) and react-native-webrtc (although there are many limitations in the latter).

The third option is to port everything from JS to C/C++, but that's a lot of work, and quite error-prone.

Agreed, but it must be done just once. That's the purpose of the mediasoup-client SDK project :)

[tkmn0]

Hi, thank you let me know this thread.
In my opinion, simple way of the signaling between client(include non browser clients like native mobile apps) and mediasoup is easy to use.
It is good that clients don't have to think about mediasoup too much.
Clients just send/receive SDP like original signaling way.
Then, mediasoup just parse the SDP and send back answer SDP.
I think the way which is not too difference from original signaling way because with this way, some non browser clients can connect mediasoup with a little changes.
Regards.

[ibc]

@tkmn0: mediasoup v1 was already "SDP based". Never again. I hate SDP, it's the worst media description format in the world. SDP O/A requires sending information about all the streams every time and it requires a useless offer/answer mechanism. We won't go that way, never.

Said that, I agree that making it easier for non mediasoup-client peers to connect to mediasoup is a MUST, and we are working on it. But we are not gonna revert the "RTP parameters" based design of mediasoup.

[ibc]

@tkmn0: If you want to be how mediasoup v3 will allow sending media from ffmpeg (a non mediasoup-client example), check this.

[tkmn0]

@ibc: I see that SDP based signaling is not suitable for mediasoup. thanks.
Is the way to connect a native mobile client to a mediasoup server is like below?
client => (create SDP) => (parse SDP with libsdptransform) => (send result) => mediasoup

Then, I read ffmpeg.md, thanks.
mediasoup can handle not only output rtp stream but also input , It is great!!:)

[ibc]

Is the way to connect a native mobile client to a mediasoup server is like below?
client => (create SDP) => (parse SDP with libsdptransform) => (send result) => mediasoup

No, that's just an example usage with ffmpeg sending an audio file to mediasoup v3. Regarding the mediasoup native SDK, there will be news in the future.

Then, I read ffmpeg.md, thanks.
mediasoup can handle not only output rtp stream but also input , It is great!!:)

That's something we are making possible in mediasoup v3, yes :)

[ibc]

Great! FYI we are also developing a C++ mediasoup-client-native that will implement the new v3 API. Still in a very early stage. So in the future we may want to join efforts.

…

On Thu, 20 Dec 2018 at 13:40, csb ***@***.***> wrote: hi ,guys: We have communicated in Google group , Does Mediasoup supports mobile platforms Last week，we have started implement the native mediasoup-client SDK for Android Based you libsdptransform lib, we have completed Android lib [sdptransformAndroid] and EventEmitterKotlin Next, we’re going to read mediasoup-client code and edit code about sdp like commonUtils,unifiedPlanUtils, RemoteUnifiedPlanSdp — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

-- Iñaki Baz Castillo <ibc@aliax.net>

[ibc]

That's so nice, @ckcsb. Thanks for the update.

[ibc]

Hi @ckcsb. Regarding react-native-webrtc and mediasoup-client, a complete status of art is described here. And yes, react-native-webrtc is just valid for 4 years old WebRTC usages.

[cloudwebrtc]

Hi @ibc
Maybe you can try to develop a cross-platform app with flutter .

This is a new solution. The UI framework developed by Google supports Android/iOS. It also supports desktop (win/linux/mac), Raspberry Pi and future Fuchsia OS, maybe in the future, we can run on all platforms except the Web through a set of UI code.

I have ported the flutter-webrtc plugin to the flutter and is currently running on iOS/Android.

We also did some work for mediasoup adaptation, ported dart-sdp-transform, dart-event-emitter and protoo-client For dart.

[ibc]

It seems that Flutter is "something" to build complete mobile apps (including UI and so on), am I wrong? Isn't just better to have native (Java and ObjC) SDKs?

[cloudwebrtc]

Yes, it is used to build mobile apps,
The native SDK needs to maintain multiple languages for developers, and UI development is a relatively large part of APP workload. Maintaining java/oc I think it will be more difficult than maintaining js, dart, all will be RN, flutter, etc. And other cross-platform frameworks. Flutter draws the UI in GPU + 2D, without the use of OEM Widgets, which renders faster.
Maybe my reply may have been wrong. This is an issue about the Native SDK.

[ibc]

Anyway it's good to know that it exists :)

[ibc]

That seems a bug in the libwebrtc Java bindings, am I wrong?

[ibc]

Given that mediasoup v3 comes with libmediasoupclient, the way to go should be creating Java and ObjC wrappers over it. This is, a Java (Android) and a ObjC (iOS) library/SDKs that:

• Implement the libmediasoupclient C++ API in Java and ObjC.

[ibc]

Closing this issue since there is already libmediasoupclient C++ library and what is missing are bindings for it on iOS/Android, which is not related to the mediasoup-client JS library anymore.