[bug] Agent interrupts itself when on speaker phone

[torphix]

Describe the bug
When using the flutter sdk with agent sdk if the agent is on speaker phone it interrupts itself whilst its talking. ie: it triggers the VAD system from its own voice. Note this only happens whilst the agent is on speaker phone.

To Reproduce
git clone the client-sdk-flutter and run the example
deploy a livekit agent in the cli
Connect to the agent room and switch to speaker phone.
Have the agent talk and observe the interruption detection being triggered by its own voice

Expected behavior
I would assume that this is a bug and echo cancellation would remove the VAD trigger by the agents own voice

Platform information
Android

• Flutter version:
• [✓] Flutter (Channel stable, 3.24.4, on macOS 14.5 23F79
  darwin-arm64, locale en-GB)
  • Flutter version 3.24.4 on channel stable at
  /Users/jessedeng/development/flutter
  • Upstream repository
  https://github.com/flutter/flutter.git
  • Framework revision 603104015d (2 weeks ago),
  2024-10-24 08:01:25 -0700
  • Engine revision db49896cf2
  • Dart version 3.5.4
  • DevTools version 2.37.3

[✓] Android toolchain - develop for Android devices
(Android SDK version 35.0.0)
• Android SDK at /Users/jessedeng/Library/Android/sdk
• Platform android-35, build-tools 35.0.0
• Java binary at: /Applications/Android
Studio.app/Contents/jbr/Contents/Home/bin/java
• Java version OpenJDK Runtime Environment (build
17.0.11+0-17.0.11b1207.24-11852314)
• All Android licenses accepted.

• Plugin version:
  livekit_client: ^2.3.0
  livekit==0.18.0
  livekit-agents==0.11.0
  livekit-api==0.7.1
  livekit-plugins-openai==0.10.5
  livekit-protocol==0.6.0

• Flutter target OS:
  Android

• Flutter target OS version:
  Android

Thanks

[cloudwebrtc]

hi, What model of mobile phone are you using? Does this device properly cancel the echo when the speakerphone is on for two-way voice calls?

[schedawg74]

Same for me on iOS 18.2.

I've setup the example app with a VoiceAgentPipeline like in the livekit docs. However the agent interrupts itself when I'm not muting myself.

Does this device properly cancel the echo when the speakerphone is on for two-way voice calls?

How?

    AudioCaptureOptions(
      deviceId: _selectedAudioDevice!.deviceId,
      echoCancellation: true,
      noiseSuppression: true,
      autoGainControl: true,
    ),

This appears to have no effect, those are set to true by default anyways.

[torphix]

I was using S24 Ultra.
It is fixed when I tried recently with latest build.

Not sure what was changed but closing for now as original issue is gone

[APPXOTICA]

Same problem here.
Samsung Galaxy S22, Redmi Note 12.
But it works ok on iOS devices.
I am not using LiveKit Agent. I implemented Gemini 2.0 on my own. Please refer below link. The demo is running on Galaxy S22.

Echo cancellation failing demo

  Future<void> _prepareRoom() async {
    final devices = await rtc.navigator.mediaDevices.enumerateDevices();
    final mic = devices.firstWhere((device) => device.kind == _audioInput);

    _room = Room(
      roomOptions: RoomOptions(
        defaultAudioCaptureOptions: AudioCaptureOptions(
          deviceId: mic.deviceId,
          noiseSuppression: true,
          echoCancellation: true,
          autoGainControl: true,
          highPassFilter: true,
          typingNoiseDetection: false,
        ),
      ),
    );
  }

  Future<void> connect() async {
    ...
    await _room?.connect(
      livekitUrl,
      _token!,
    );
    _room?.localParticipant?.setMicrophoneEnabled(true);
    ....
  }