docs(plan): finalize Phase 3 design — mic=You, diarize loopback-masked (FR-SPK)
This commit is contained in:
@@ -1,9 +1,44 @@
|
||||
# Phase 3 design sketch: per-stream speaker attribution (FR-SPK)
|
||||
|
||||
Status: **draft for sign-off**, 2026-07-13. Deepens Phase 3 of
|
||||
Status: **signed off & in progress**, 2026-07-13. Deepens Phase 3 of
|
||||
[`2026-07-13-diarization-speaker-accuracy.md`](./2026-07-13-diarization-speaker-accuracy.md).
|
||||
Requires ADR-0005 review (done below) and a decision before any code lands — this changes
|
||||
the diarization data flow, so it is gated per the parent plan.
|
||||
The §§1–8 below are the original sketch (ADR-0005 review + alternatives); the **finalized
|
||||
design** the user signed off on is at the top and supersedes the RMS-dominance heuristic.
|
||||
|
||||
---
|
||||
|
||||
## Finalized design (signed off 2026-07-13)
|
||||
|
||||
Simpler than the sketch's dominance heuristic, because the user records on **headphones** so the
|
||||
mic never hears the far side (no bleed):
|
||||
|
||||
> **The microphone stream is "You" by definition. Diarization runs on the loopback (speaker)
|
||||
> audio only, never the mic. Any loopback speaker is "Speaker N".**
|
||||
|
||||
Mechanism, all in **`audio.wav` frame time** (so You-spans and sherpa-spans share one clock):
|
||||
|
||||
1. **Mic-activity timeline** — captured in the loopback writer, the one place mic and loopback
|
||||
exist separately in `audio.wav`'s timebase (`capture_loop`, where `bridge.pull()` is folded
|
||||
in). Per 100 ms, flag whether the pulled mic chunk was speech-level (reuse `VOICE_ENERGY_FLOOR`).
|
||||
Held in a shared `MicActivity` (sibling to `VoiceSample`); persisted for reprocess.
|
||||
2. **At stop / reprocess:**
|
||||
- Collapse active windows → **"You" spans** (dropping sub-`MIN_SPAN_MS` blips).
|
||||
- **Mask** the You ranges out of `audio.wav` (zero them) and run sherpa on the remainder → far
|
||||
side splits into **"Speaker N"** spans. Clustering never sees the mic, so the mic can't
|
||||
pollute or merge with a far-side cluster.
|
||||
- Merge span lists; `assign_by_overlap` unchanged.
|
||||
3. **Voiceprint (Phase 1/2) becomes the fallback** for meetings with no timeline (imports, older
|
||||
recordings). Live-tick voiceprint "You" is retired once the timeline drives labels.
|
||||
|
||||
Timebase note: with the capture-silence fix (FR-CAP-7) `audio.wav` is now continuous, so the
|
||||
frame-indexed timeline aligns cleanly with what sherpa reads back via `read_wav_mono_16k`.
|
||||
|
||||
Implementation order: (a) `MicActivity` struct + capture wiring; (b) stop-path attribution +
|
||||
masked diarize; (c) persist timeline + reprocess/import; (d) amend ADR-0005 + `docs/02`/`03`.
|
||||
|
||||
---
|
||||
|
||||
_Original sketch (ADR-0005 review, risks, alternatives) follows._
|
||||
|
||||
Phases 0–2 shipped: cluster explosion tamed (0.7 + 700 ms floor), reprocess re-diarizes with a
|
||||
persistent `voiceprint.wav`, and live "You" reaches the UI. Those all still treat diarization as
|
||||
|
||||
Reference in New Issue
Block a user