This SDK is not polished enough to be production ready for a video calls app

[EvertonMJunior]

Hey LK Team,
We were working on an integration with you on our Flutter app to power our video calls, we were migrating from another provider. On a first impression, everything looked good and as it would work out, but we are having to migrate to another provider now due to some bad issues that hinder us of using LiveKit. I am giving this feedback here as I am a fan of the project and of open source, and would like to work with you in the future when this is solved:

• Screen sharing is buggy (On Android, we had issues after upgrading the flutter_webrtc package, with the local screen share not showing up to the user themselves; On iOS, same thing happened, although even on older versions of both flutter_webrtc and livekit_client). Our usecase relies a lot on screen sharing on mobile, and it needs to be rock solid. Another thing is the experience of making it work on iOS. It's pretty bad, to be honest, as there are lots of manual steps and places that things go wrong with setting up the XCode project. The provider we are using right now, also a maintainer of WebRTC on Flutter, have a way better experience in this aspect and things just worked out of the box.
• PIP is not implemented neither on Android nor iOS. We could figure out an implementation on Android with a package, but it's way harder on iOS, as it involves native Swift code. On our current provider, there's an implementation for both out of the box, as well.
• Video filters, especially blur, are not implemented. Today, that's not a shiny feature that might be present, that's a must in any video calling software. It's become a commodity, users are so used to it now, that it's really hard to ship without it.
• We had issues with delays on webhook delivery, and we need that working well. Everything worked out perfectly on our last provider, and seems to be working too on our current
• Lastly, we noticed a lot of UI jank on the implementations, Android, iOS and Web. We want to ship software that delights customers, and that doesn't.

All in all, this is feedback of our experience right now. I hope you can improve and deploy more effort on the Flutter implementation, as it looks like the Android, iOS and JS native implementations are way ahead. The amount of issues on this project shows this, and nothing I am detailing above is a new thing, there are issues talking about each and every one. Flutter is growing a lot, and especially due to your AI related capabilities, it seems a waste to have a swiss cheese kind of implementation on Flutter.I am open to discussing all of the above. Thank you!

[bcherry]

@EvertonMJunior Thanks for the thoughtful post and for trying LiveKit, we really appreciate the feedback!

We're definitely aware of all the limitations you raise here and all are on our roadmap for improvement / implementation (with the exception of the slow webhooks - if there's more details you could share about that we'd love them, feel free to DM in our Slack if you'd like). This SDK remains under active development so I hope you'll see these issues (and ohers) get resolved relatively soon.

[cloudwebrtc]

Screen sharing is buggy (On Android, we had issues after upgrading the flutter_webrtc package, with the local screen share not showing up to the user themselves; On iOS, same thing happened, although even on older versions of both flutter_webrtc and livekit_client). Our usecase relies a lot on screen sharing on mobile, and it needs to be rock solid. Another thing is the experience of making it work on iOS. It's pretty bad, to be honest, as there are lots of manual steps and places that things go wrong with setting up the XCode project. The provider we are using right now, also a maintainer of WebRTC on Flutter, have a way better experience in this aspect and things just worked out of the box.

Just checked, this bug is indeed introduced by the recent PR
fixed for android: https://github.com/flutter-webrtc/flutter-webrtc/pull/1729/files

For iOS, it seems to be caused by PlatformView, because PlatformView cannot handle the display ratio of the rotated frame well, because of the coordination between the flutter widget and native. So I suggest that when using VideoTrackRenderer, change the renderMode to VideoRenderMode.texture to fix this issue
https://github.com/livekit/client-sdk-flutter/blob/main/example/lib/widgets/participant.dart#L146

will release a new version next week

[EvertonMJunior]

@EvertonMJunior Thanks for the thoughtful post and for trying LiveKit, we really appreciate the feedback!

We're definitely aware of all the limitations you raise here and all are on our roadmap for improvement / implementation (with the exception of the slow webhooks - if there's more details you could share about that we'd love them, feel free to DM in our Slack if you'd like). This SDK remains under active development so I hope you'll see these issues (and ohers) get resolved relatively soon.

Of course, @bcherry. I'll test more about the webhooks and will bring them to Slack. Here to help!

[EvertonMJunior]

Screen sharing is buggy (On Android, we had issues after upgrading the flutter_webrtc package, with the local screen share not showing up to the user themselves; On iOS, same thing happened, although even on older versions of both flutter_webrtc and livekit_client). Our usecase relies a lot on screen sharing on mobile, and it needs to be rock solid. Another thing is the experience of making it work on iOS. It's pretty bad, to be honest, as there are lots of manual steps and places that things go wrong with setting up the XCode project. The provider we are using right now, also a maintainer of WebRTC on Flutter, have a way better experience in this aspect and things just worked out of the box.

Just checked, this bug is indeed introduced by the recent PR
fixed for android: https://github.com/flutter-webrtc/flutter-webrtc/pull/1729/files

For iOS, it seems to be caused by PlatformView, because PlatformView cannot handle the display ratio of the rotated frame well, because of the coordination between the flutter widget and native. So I suggest that when using VideoTrackRenderer, change the renderMode to VideoRenderMode.texture to fix this issue
https://github.com/livekit/client-sdk-flutter/blob/main/example/lib/widgets/participant.dart#L146

will release a new version next week

Thanks @cloudwebrtc, the fix for Android did work! About iOS, it still didn't, the local screen share still doesn't appear even using texture renderMode on iOS screen shares. Other users can see the transmitted screen well

[cloudwebrtc]

hey @EvertonMJunior, Please try this PR again. I just tested and confirmed that iOS has also been fixed and there is no need to modify renderMode.

flutter-webrtc/flutter-webrtc#1729

[EvertonMJunior]

hey @EvertonMJunior, Please try this PR again. I just tested and confirmed that iOS has also been fixed and there is no need to modify renderMode.

flutter-webrtc/flutter-webrtc#1729

Hi @cloudwebrtc , wow, it worked! Thanks a lot. Those small details matter a lot, thanks for solving both of these issues briefly. That already makes the package well polished, all features are working. Now the two things that, for my usecase, are the most important, are the blur effects, and solving the UI jank that has also been reported in other issues. All in all, thanks, and we'll get make more tests after the holidays. Happy holidays!