docs(adr-0005): amend with Phase 3 per-stream 'You' attribution (FR-SPK)

This commit is contained in:
iamdoubz committed 2026-07-14 00:53:04 -05:00
1 parent 0b5a85f461
commit 6385e38b33
1 file changed
+24
+24
View File
@@ -32,6 +32,30 @@ labels in the post-meeting pass; this keeps latency low while improving final ac
- Speaker IDs (`S1`, `S2`, …) are internal and stable per meeting; name mappings live in the DB
and are applied at render/export time, never destructively rewritten onto segments.
### Phase 3 refinement — per-stream "You" attribution (FR-SPK, 2026-07-14)
The single-pass-over-the-whole-recording model above blind-clusters a **summed mono** signal
(mic + loopback), then guesses which cluster is the user via a voiceprint match. In practice that
clustering is unreliable in both directions on the summed signal — it over-split a 2-speaker call
into 83 clusters at one threshold and merged two clearly distinct voices into one at another — and
the voiceprint can only *label* a cluster, never *create* the separation.
So when the **microphone is enabled**, WA no longer relies on clustering to find the user:
- A live **mic-activity timeline** (`audio::MicActivity`) records, per 100 ms of `audio.wav` frame
time, when the mic was speech-level — captured in the loopback writer, the one place the mic and
loopback exist separately in the recording's own timebase. Those ranges become **"You"** spans
directly (no clustering, no embedding).
- sherpa clustering is then run over the recording with the "You" ranges **masked out** (zeroed),
so it only ever splits the **far side** into `Speaker N`. The mic can never pollute or merge into
a far-side cluster.
- The timeline is persisted as `mic_activity.json` (retained meetings) so `reprocess_transcript`
re-attributes identically without the live capture.
The original blind-clustering + voiceprint pass **remains the fallback** for mic-off recordings and
imports (no timeline). Naming stays uniform (`build_name_map`: "You", then `Speaker 2…`). Segment
IDs and the names-in-DB rule are unchanged; only the *source* of the spans changes.
## Revisit if
A single model gives joint ASR + diarization with better accuracy, or whisper.cpp gains
production diarization.