docs(plan): finalize Phase 3 design — mic=You, diarize loopback-masked (FR-SPK)

This commit is contained in:
iamdoubz
2026-07-13 21:15:48 -05:00
parent 6dd6c3c69f
commit 41c2c79797
@@ -1,9 +1,44 @@
# Phase 3 design sketch: per-stream speaker attribution (FR-SPK)
Status: **draft for sign-off**, 2026-07-13. Deepens Phase 3 of
Status: **signed off & in progress**, 2026-07-13. Deepens Phase 3 of
[`2026-07-13-diarization-speaker-accuracy.md`](./2026-07-13-diarization-speaker-accuracy.md).
Requires ADR-0005 review (done below) and a decision before any code lands — this changes
the diarization data flow, so it is gated per the parent plan.
The §§1–8 below are the original sketch (ADR-0005 review + alternatives); the **finalized
design** the user signed off on is at the top and supersedes the RMS-dominance heuristic.
---
## Finalized design (signed off 2026-07-13)
Simpler than the sketch's dominance heuristic, because the user records on **headphones** so the
mic never hears the far side (no bleed):
> **The microphone stream is "You" by definition. Diarization runs on the loopback (speaker)
> audio only, never the mic. Any loopback speaker is "Speaker N".**
Mechanism, all in **`audio.wav` frame time** (so You-spans and sherpa-spans share one clock):
1. **Mic-activity timeline** — captured in the loopback writer, the one place mic and loopback
exist separately in `audio.wav`'s timebase (`capture_loop`, where `bridge.pull()` is folded
in). Per 100 ms, flag whether the pulled mic chunk was speech-level (reuse `VOICE_ENERGY_FLOOR`).
Held in a shared `MicActivity` (sibling to `VoiceSample`); persisted for reprocess.
2. **At stop / reprocess:**
- Collapse active windows → **"You" spans** (dropping sub-`MIN_SPAN_MS` blips).
- **Mask** the You ranges out of `audio.wav` (zero them) and run sherpa on the remainder → far
side splits into **"Speaker N"** spans. Clustering never sees the mic, so the mic can't
pollute or merge with a far-side cluster.
- Merge span lists; `assign_by_overlap` unchanged.
3. **Voiceprint (Phase 1/2) becomes the fallback** for meetings with no timeline (imports, older
recordings). Live-tick voiceprint "You" is retired once the timeline drives labels.
Timebase note: with the capture-silence fix (FR-CAP-7) `audio.wav` is now continuous, so the
frame-indexed timeline aligns cleanly with what sherpa reads back via `read_wav_mono_16k`.
Implementation order: (a) `MicActivity` struct + capture wiring; (b) stop-path attribution +
masked diarize; (c) persist timeline + reprocess/import; (d) amend ADR-0005 + `docs/02`/`03`.
---
_Original sketch (ADR-0005 review, risks, alternatives) follows._
Phases 0–2 shipped: cluster explosion tamed (0.7 + 700 ms floor), reprocess re-diarizes with a
persistent `voiceprint.wav`, and live "You" reaches the UI. Those all still treat diarization as