Audio / Video Devices Sinks and Sources - Bridge with libVLC

[ChristopheI]

Hi!

I just found out about SipSorcery and I have a few questions.

I am working on C # SDK (Cross-platform: UWP, WPF, MacOS, Android, iOS) using Rainbow platform bringing one-to-one and group discussion: https://www.nuget.org/packages/Rainbow.CSharp.SDK/

This Rainbow platform also supports WebRTC (P2P and audio / video / screen sharing) and SIP devices.

I would like to add support for WebRTC (SIP first) in my SDK using SIPSorcery.

I notice that the management of Audio / Video inputs / nettles is the sticking point to target all platforms.

It seems possible to use libVLC (according to this exchange sipsorcery-org/SIPSorceryMedia.FFmpeg#1). Do you think this is still correct? I would like to start creating an open source project to create this bridge between SIPSorcery and libVLC to allow the use of audio / video devices as receivers and sources.

Thanks in advance

[sipsorcery]

It seems possible to use libVLC (according to this exchange sipsorcery-org/SIPSorceryMedia.FFmpeg#1). Do you think this is still correct? I would like to start creating an open source project to create this bridge between SIPSorcery and libVLC to allow the use of audio / video devices as receivers and sources.

I think LibVLCSharp is the most likely option for cross platofrm audio/video support with .NET and by extension for use with the sipsorcery library.

One of the LibVLCSharp maintainers did email me a while ago mentioning that they were working on an example integration with sipsorcery. I haven't heard how it went. If you were feeling cheeky you could post an issue on the libvlcsharp repo asking if anyone knows of any demos using sipsorcery. You might get lucky ;).

[ChristopheI]

Hello,

Thank you for your reply. I will contact the LibVLCSharp team about this. Crossed fingers... :)

In the meantime, not being familiar with SIPSorcery and stuff about encoders / codecs, I took a closer look at the FFmpeg library and the examples you provided.

I noticed that there was no example of use using FFmpeg allowing:

• to list the audio outputs
• to list the audio or video entries
• to specifically capture an audio input or output
• to specifically capture a video input

Could this be a limitation of the FFmpeg.AutoGen library? It should be possible... Have you tried this track before?

Extract from APIs:
av_input_audio_device_next
av_input_video_device_next
av_output_audio_device_next
av_output_video_device_next
=> https://ffmpeg.org/doxygen/3.4/group__lavd.html#details

Thank you

[sipsorcery]

Could this be a limitation of the FFmpeg.AutoGen library? It should be possible... Have you tried this track before?

FFmpeg & FFmpeg.AutoGen do support audio I just haven't used it. I used NAudio instead. Wiring FFmpeg up is very time consuming and even more so when there's a native-to-managed interop layer to contend with. I've only used FFmpeg where there is no better .NET option.

[ChristopheI]

I see.

I have some availabilities on my side so I'll take a look.

My goal is to extend your library SIPSorceryMedia.FFmpeg step by step with this new features to allow WebRTC communications using Audio/Video devices of the end-user.

I will provide step by step my results

[mail2mhossain]

I have found an example "WebRTCAndLibVLC" (https://code.videolan.org/mfkl/libvlcsharp-samples/-/tree/master/WebRTCAndLibVLC) and it is working fine. But could not figure out how to use LibVLCSharp as audio/video source. Any help in this regard.

[ChristopheI]

@mail2mhossain ,
Sorry I can't help you. I give up using libVLC. It doesn't mean that it's not possible but I found a full solution using
SIPSorceryMedia.FFmpeg and SIPSorceryMedia.SDL2 to have Audio/Video Input and Audio/Video Source.

[mail2mhossain]

@ChristopheI ,
Thanks to share your experience. BTW: is this a cross platform solution? Means, Does it support Windows, Linux, MacOS, Android, iOS, with Uno Platform.

[ChristopheI]

I have already tested successfully on Windows, Linux and MacOS.

It should also worked on Android and iOS since both FFmpeg and SDL2 are available on these platforms too but I didn't have time to test on this platform

I never used Uno Platform but since is a UI library it should worked too

[mail2mhossain]

Thank you very much for prompt reply. I'll start to work on this soon.