System audio capture issues: MicOnly heuristic + samples_dropped buffer overflow

[upcollective]

First, thanks for the quick attention to my previous issue - really appreciate the responsiveness! 🙏

---

Summary

Two related issues preventing reliable speaker differentiation on macOS:

1. Built-in mic forces MicOnly mode - System audio never captured without headphones
2. System audio buffer overflow - Even when MicAndSpeaker mode activates, samples_dropped warnings flood the log and speaker audio eventually fails

Environment

• macOS: Sequoia 15.2
• Hyprnote: 1.0.1-nightly (via brew install --cask fastrepl/hyprnote/hyprnote@nightly)
• Hardware: MacBook Air (M-series)
• Audio devices tested:
  • Internal mic + internal speakers → MicOnly (no system audio)
  • AirPods Pro → MicAndSpeaker (but samples dropped)
• Browsers tested: Chrome, Safari, Comet - same results
• Permission: Screen & System Audio Recording granted ✓

---

Issue 1: MicOnly Heuristic

Problem

The ChannelMode::determine() logic returns MicOnly when using the built-in microphone:

if has_builtin_mic() && !is_default_input_external() → MicOnly

This means MacBook users without external audio devices can never capture system audio (remote participants in video calls).

Log Evidence

mic_device=Some("MacBook Air Microphone")
start_source_loop new_mode=MicOnly
connect_async: ...channels=1...multichannel=false...

Expected Behavior

Video call use case needs both mic AND system audio regardless of whether mic is built-in or external.

Workaround Found

Connecting headphones triggers the headphone detection check (condition #2) before the built-in mic check (condition #4), enabling MicAndSpeaker mode.

---

Issue 2: System Audio Buffer Overflow

Problem

Even when MicAndSpeaker mode activates (with headphones), the system audio stream constantly drops samples until it fails completely.

Log Evidence

mic_device=Some("Jarrett's AirPods Pro #2")
new_mode=MicAndSpeaker

WARN audio::speaker::macos: samples_dropped dropped=512
WARN audio::speaker::macos: samples_dropped dropped=512
WARN audio::speaker::macos: samples_dropped dropped=512
[... hundreds of these warnings ...]

WARN failed_to_cast_speaker_chunk

Observed Behavior

1. Recording starts correctly with "You" and "Speaker 1" differentiation
2. After ~30-60 seconds, remote participant audio stops being transcribed
3. Only local mic audio continues to be captured
4. Logs show continuous samples_dropped warnings throughout

Possible Causes

• Buffer size too small for system audio capture
• Sample rate mismatch between mic and speaker streams
• Threading/timing issue in audio callback

---

Additional Note: AirPods Detection

AirPods don't trigger the auto-detection notification to start recording (mic detection doesn't fire for Bluetooth devices). Recording must be started manually. This is a minor issue compared to the above.

---

Reproduction Steps

For Issue 1 (MicOnly):

1. Use MacBook with internal mic/speakers (no headphones)
2. Grant Screen & System Audio Recording permission
3. Join a video call
4. Hyprnote auto-detects and records
5. → All speakers show as "Speaker 1", logs show MicOnly

For Issue 2 (samples_dropped):

1. Connect headphones (wired or AirPods)
2. Start recording a video call
3. Initially shows "You" vs "Speaker 1" correctly
4. → After ~30-60 seconds, remote audio disappears from transcript

---

Happy to provide additional logs or test builds if helpful!

[yujonglee]

hi @upcollective - thanks for the detailed report.

Issue 1: MicOnly Heuristic

This was actually intentional. We spent a fair amount of time trying to remove echo (speaker capture bleeding back into the microphone) using a combination of heuristics and AI models, but they are not reliable enough to ship yet. (We do plan to revisit this in the future.)

As a result, we decided to default to MicOnly under the condition if has_builtin_mic() && !is_default_input_external(), since speaker capture would inevitably echo back into the mic anyway.

Does this behavior not make sense from your perspective? Please let me know.

Issue 2: System Audio Buffer Overflow

Additional Note: AirPods Detection

This is very strange, because everyone on the team uses AirPods, and we have multiple AirPods models across different generations, all of which work well in our testing.

This is a critical issue that I want to fix at all costs, but it is difficult to take action without being able to reproduce it. If you can share any additional information, that would be extremely helpful.

[yujonglee]

Related: #2942

[yujonglee]

Also handled in #2974.