fix: remove duplicate track publishing code

[d3xvn]

• Remove duplicate track publishing and audio/video listening code in join() method
• Initialize timeout_errors and consecutive_errors before video processing loop
• Increment timeout_errors in TimeoutError exception handler
• Fixes potential crash when error counters are referenced but not initialized

Summary by CodeRabbit

• Bug Fixes
  • Prevented duplicate media publishing during session join, reducing setup glitches and unexpected behavior.
• Refactor
  • Improved video processing resilience with adaptive timeout handling and exponential backoff, leading to smoother recovery from network hiccups and fewer interruptions.
  • Enhanced internal error tracking to reset after successful processing, improving overall stream stability.

[coderabbitai]

Walkthrough

Consolidated duplicate track publishing in join to a single invocation, then proceeds to listening. In _process_track, added timeout and consecutive error counters, integrated exponential backoff on TimeoutError, and reset counters upon successful frame processing. Control flow now tracks and logs timeouts within the processing loop.

Changes

Cohort / File(s)	Summary
Join flow and track processing agents-core/vision_agents/core/agents/agents.py	Removed duplicate publish in join; added timeout_errors and consecutive_errors; implemented exponential backoff on timeouts with counter-based delay; reset timeout counter on success; updated related debug logging and control flow within the processing loop.

Sequence Diagram(s)

sequenceDiagram
  autonumber
  actor Client
  participant Agent
  participant MediaTracks

  Client->>Agent: join()
  activate Agent
  Agent->>MediaTracks: publish_tracks()
  Note over Agent,MediaTracks: Single publish (duplicate removed)
  Agent-->>Client: start listening
  deactivate Agent

Loading

sequenceDiagram
  autonumber
  participant Processor as _process_track
  participant Source as VideoSource

  loop For each frame
    Processor->>Source: get_next_frame(timeout)
    alt Frame received
      Source-->>Processor: frame
      Note right of Processor: Process frame<br/>timeout_errors = 0
    else TimeoutError
      Source-->>Processor: timeout
      Note right of Processor: timeout_errors += 1<br/>delay = base * 2^(timeout_errors-1)
      Processor->>Processor: sleep(delay)
    end
  end

Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Poem

The room holds its breath: one publish, not two—
A single bell in the skull’s cathedral.
Frames arrive like moths, or don’t; the dark refuses.
I count the silences, doubling their hush,
nursing the clock’s milk till it forgives.
Then—the image blooms, and error forgets my name.

Pre-merge checks

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%.	You can run @coderabbitai generate docstrings to improve docstring coverage.

✅ Passed checks (2 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title Check	✅ Passed	The title explicitly identifies the primary change—removing duplicate track publishing code—from the join() method and is concise, clear, and directly tied to the summary of changes without extraneous information.

---

Note

Free review on us!

CodeRabbit is offering free reviews until Wed Oct 08 2025 to showcase some of the refinements we've made.

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

Actionable comments posted: 0

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)

agents-core/vision_agents/core/agents/agents.py (1)

669-669: Unreachable code at Line 669.

This log statement is unreachable. The while True loop at line 627 only exits via:

• Line 663: return (on CancelledError)
• Line 666: raise (on other exceptions)

Line 669 will never execute.

Apply this diff to remove the unreachable code:

             except Exception:
                 raise
-
-        # Cleanup and logging
-        self.logger.info(f"🎥VDP: Video processing loop ended for track {track_id} - timeouts: {timeout_errors}, consecutive_errors: {consecutive_errors}")

🧹 Nitpick comments (1)

agents-core/vision_agents/core/agents/agents.py (1)

624-625: Consider removing unused consecutive_errors counter.

consecutive_errors is incremented on line 652 but never used for control flow or backoff logic—unlike timeout_errors. It was only logged in the now-unreachable line 669.

If there's no planned logic for consecutive_errors, remove it:

-        # Initialize error tracking counters
         timeout_errors = 0
-        consecutive_errors = 0
         
         while True:
             try:
                 # Use the shared forwarder instead of competing for track.recv()
                 video_frame = await shared_forwarder.next_frame(timeout=2.0)
 
                 if video_frame:
                     # Reset error counts on successful frame processing
                     timeout_errors = 0
-                    consecutive_errors = 0
 
                     if hasImageProcessers:
 
                         img = video_frame.to_image()
 
                         for processor in self.image_processors:
                             try:
                                 await processor.process_image(img, participant.user_id)
                             except Exception as e:
                                 self.logger.error(
                                     f"Error in image processor {type(processor).__name__}: {e}"
                                 )
 
 
                 else:
                     self.logger.warning("🎥VDP: Received empty frame")
-                    consecutive_errors += 1

Alternatively, implement logic that uses consecutive_errors (e.g., exit the loop or skip processing after N consecutive empty frames).

Also applies to: 652-652

📜 Review details

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 2d08f1d and 4f60ab2.

📒 Files selected for processing (1)

• agents-core/vision_agents/core/agents/agents.py (2 hunks)

🧰 Additional context used

📓 Path-based instructions (1)

**/*.py

📄 CodeRabbit inference engine (.cursor/rules/python.mdc)

**/*.py: Never adjust sys.path (e.g., sys.path.append/insert/assignment)
Docstrings must follow the Google style guide

Files:

• agents-core/vision_agents/core/agents/agents.py

⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (2)

• GitHub Check: unit / Ruff & mypy
• GitHub Check: unit / Test "not integration"

🔇 Additional comments (3)

agents-core/vision_agents/core/agents/agents.py (3)

240-252: LGTM! Clean track publishing flow.

The track publishing and listening logic is now consolidated properly. Publishing occurs once before setting up listeners, eliminating the duplication issue described in the PR objectives.

---

623-626: Proper initialization prevents crash.

Initializing timeout_errors and consecutive_errors before the processing loop correctly addresses the potential crash when these counters are referenced in exception handlers.

---

656-661: Timeout handling and backoff logic is correct.

The timeout_errors increment and exponential backoff calculation work as intended, capping at 30 seconds to prevent excessive delays.