Skip to content

Conversation

@Nash0x7E2
Copy link
Member

In this PR, we introduce the class TurnDetection, which allows the implementation of various turn detection models. By default, we also include a small example using fal/smart-turn, which can be found in examples/example_turn_detection.py.

Params:

  • mini_pause_duration
  • max_pause_duration
  • is_detecting
  • on
  • process_audio
  • start
  • stop
  • emit

When using the TurnDetection class, the following events can be emitted and observed:

class TurnEvent(Enum):
    """Events that can occur during turn detection."""

    SPEECH_STARTED = "speech_started"
    SPEECH_ENDED = "speech_ended"
    TURN_STARTED = "turn_started"
    TURN_ENDED = "turn_ended"
    MINI_PAUSE_DETECTED = "mini_pause_detected"
    MAX_PAUSE_REACHED = "max_pause_reached"

The base agents/agent.py class has also been modified to support the turn_detection parameter in __init__.

Nash0x7E2 and others added 7 commits August 14, 2025 23:15
- Add FalTurnDetection class implementing the TurnDetection protocol
- Integrate with FAL AI's smart-turn model for audio-based turn detection
- Buffer audio data, upload to FAL API, and process predictions
- Wire into existing Agent class with event-driven architecture
- Add fal-client dependency and comprehensive example
- Create example_turn_detection.py demonstrating usage
- Include documentation and configuration options

Features:
- Real-time audio buffering and processing
- Configurable prediction thresholds and buffer durations
- Event emission for turn start/end detection
- Temporary file management and cleanup
- Integration with Stream's WebRTC audio pipeline
@Nash0x7E2 Nash0x7E2 self-assigned this Aug 15, 2025
@Nash0x7E2 Nash0x7E2 merged commit 2c54ccd into main Aug 15, 2025
@Nash0x7E2 Nash0x7E2 deleted the nash/turn-detection branch August 17, 2025 01:02
dangusev added a commit to tjirab/Vision-Agents that referenced this pull request Oct 30, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants