# Plan: dual-channel capture (mic = left, loopback = right) — FR-SPK / FR-CAP

Status: **implemented** 2026-07-14 (all 3 sign-off answers: DB column; retire MicActivity with a
code note; export → dual-mono). Migration 0009 + `write_wav_split` + `attribute_split` +
`fold_wav` + ADR-0005 Phase 3.5. Supersedes the summed-mono + masked-diarize approach
in [`2026-07-13-diarization-phase3-per-stream-attribution.md`](./2026-07-13-diarization-phase3-per-stream-attribution.md).
The proposed **ADR-0005 amendment** is at the bottom — apply it on approval.

## Motivation

Every diarization problem this month traces to one root: **`audio.wav` is a summed mono mix** of
mic + loopback, so the two sources are irrecoverable after the fact. Phase 3 works around that by
(a) recording a separate `mic_activity.json` "You" timeline and (b) diarizing the mix with the You
ranges masked out. It works at stop but is fragile on **reprocess** — the persisted timeline has to
re-align against freshly-transcribed segments over a mono mix, and it currently collapses to a
single "Speaker 2". The timelines are being written correctly (verified: real `you_spans`), so the
failure is the masked-mono re-alignment itself, not the capture.

**Fix the root, not the instance:** keep both sources *physically separate in the recording* by
recording **stereo — left channel = microphone ("You"), right channel = system/loopback
("Speaker")**. The separation then lives in the file forever; reprocess and diarization always have
both streams; no sidecar timeline, no masking, no summed-mono clustering.

## Key facts that make this cheap

- `audio.wav` is **already 2-channel** on a normal (stereo) render endpoint — today we *sum the mic
  into both channels*. Writing mic→L and loopback(downmixed)→R is the **same channel count, same
  bytes** — no size increase in the common case.
- Transcription already downmixes to mono: `read_wav_mono_16k` averaging L+R of a split file yields
  `(mic + loopback)/2` = the same summed signal it gets today, so **transcription is unchanged**.
- The mic↔loopback alignment machinery (`MicBridge`, the drain fix, the capture-silence fix) carries
  over unchanged — we just route the aligned samples to two channels instead of summing them.

## Design

### 1. Capture write path (`audio`)
- Force the recording spec to **2ch, 16-bit, native rate** when the mic is enabled ("split"
  layout). Loopback-only (mic off) keeps today's behavior.
- New writer (replacing `write_wav_bytes`'s summing for split recordings): per frame,
  **L = mic sample** (0 when the mic underran), **R = downmix(loopback channels)**.
- Idle path (loopback silent): **L = drained mic, R = 0** — same silence-synthesis + device-paced
  drain we just fixed, now writing to L instead of summing.

### 2. Layout versioning (distinguish old "summed" from new "split")
Content alone can't tell a summed-stereo file from a split-stereo one, so mark it explicitly:
- Add `audio_layout TEXT` to the `meetings` row: `'summed'` (default / NULL for all existing rows
  and imports) or `'split'` (new mic-enabled recordings).
- Include it in the **`MeetingBundle` manifest** so exported recordings stay interpretable on
  another machine.
- Playback and diarization branch on it; old recordings keep every current code path untouched.

### 3. Read helpers (`audio`)
- `read_wav_mono_16k` — **unchanged** (downmix all channels → mono 16k). Used for transcription of
  both layouts; on a split file it returns the summed signal, exactly as wanted.
- New `read_wav_channel_16k(path, channel)` — extract one channel → mono 16k. Diarization reads the
  **right** channel (far side); "You" detection reads the **left** (mic).

### 4. Diarization / attribution (`commands`) — *simpler* than Phase 3
For a **split** recording, one path replaces the timeline + masking:
```
far_spans = diarizer.diarize_samples( read_wav_channel_16k(wav, RIGHT) )   // clean far side
you_spans = vad_spans( read_wav_channel_16k(wav, LEFT) )                    // energy-windowed mic
merge + assign_by_overlap + build_name_map("You")                           // You + Speaker N
```
- The mic is **never** in the clustering input — structurally, not procedurally.
- `vad_spans` is the `MicActivity` windowing logic (energy floor + 100 ms windows) applied to the L
  channel from the file — so **stop and reprocess share one path** and produce identical results.
- **This deletes `MicActivity`, `mic_activity.json` (persist + read + delete), and `mask_ranges`.**
  Net: less code than the current Phase 3.
- Mic-off (`'summed'`) recordings keep the whole-signal pass + voiceprint fallback.

### 5. Playback (`commands::serve_recording`)
- A split file plays mic-in-left / computer-in-right — wrong for a listener. On playback, **downmix
  L+R → mono** (dual-mono) so both sources come out both ears.
- `serve_recording` already reads+decrypts the whole file into memory; add a downmix-to-mono step
  for `'split'` layout and serve that mono WAV (range/seek computed over the transcoded bytes).
  `'summed'` files stream as today.

### 6. Migration & portability
- **Existing recordings** (`'summed'`): every current path unchanged. Their reprocess falls back to
  the voiceprint match (acceptable — they predate split). We may retire the `MicActivity`/mask code
  once split is default; the handful of recent test recordings lose Phase 3 on reprocess, which is
  fine.
- **Bundle export**: export **downmixes the canonical split `audio.wav` to mono** so a shared file
  plays normally in any player; the working file stays split for WA's own re-processing.

## What this removes / simplifies

| Removed | Replaced by |
|---|---|
| `MicActivity` struct + capture wiring | L-channel VAD read from the file |
| `mic_activity.json` (write/read/delete) | nothing — recompute from L channel |
| `mask_ranges` + masked diarize | diarize the R channel directly |
| stop vs reprocess divergence | one shared split-attribution path |

The capture-silence fix, the device-paced `drain`, `diarize_samples`, and the `build_name_map`
naming all **stay**.

## Costs / risks (none fatal)
- **New code:** split writer, `audio_layout` column + migration + manifest field, channel read
  helper, playback downmix, `vad_spans`, and the split-vs-summed branch. Bigger than a patch, but
  it *retires* comparable Phase 3 code.
- **Loopback stereo image lost** (R = loopback downmixed to mono) — irrelevant for speech.
- **Mono render endpoint** (rare): 1ch→2ch doubles that recording's size.
- **Raw file plays split** in a third-party player until exported (export downmixes) — minor.
- **Clock drift** between mic/loopback now shows as tiny L/R desync instead of dropped summed
  samples — harmless (diarization tolerates it; playback sums it away).

## Open questions for sign-off
1. Layout marker: **DB column** (proposed) vs. a self-describing WAV chunk? DB column is simpler and
   travels via the bundle manifest; recommend it.
2. Retire `MicActivity`/`mask_ranges` outright, or keep them for existing `'summed'` recordings'
   reprocess? Recommend **retire** (summed → voiceprint fallback) to avoid two live systems.
3. Export downmix to mono — agree that's the right default for shared/synced copies?

## Implementation order
1. `audio_layout` column + `MeetingBundle` field + read on `get_meeting` (no behavior yet).
2. Split writer + force 2ch when mic on; set `audio_layout='split'`. (`audio` + `start_recording`.)
3. `read_wav_channel_16k` + `vad_spans`; split-attribution path in stop + reprocess.
4. Playback downmix for split; bundle-export downmix.
5. Delete `MicActivity` / `mic_activity.json` / `mask_ranges`; update tests.
6. Apply the ADR-0005 amendment + update `docs/02`/`03`.

---

## Proposed ADR-0005 amendment (apply on approval)

> ### Phase 3.5 refinement — dual-channel capture (FR-SPK/FR-CAP, 2026-07-14)
>
> The Phase 3 per-stream attribution (above) reconstructs the mic/far-side split *after the fact*
> from a summed-mono `audio.wav` plus a `mic_activity.json` timeline, then masks the mic ranges
> before clustering. This is reliable at stop but fragile on reprocess (re-aligning a sidecar
> timeline against a mono mix), and it carries a parallel persistence path.
>
> **Decision:** when the microphone is enabled, record `audio.wav` as **stereo with the streams
> separated — left = microphone ("You"), right = system/loopback ("Speaker")** — instead of summing
> them. The separation is then intrinsic to the recording:
> - Diarization runs on the **right channel only** → `Speaker N`; "You" comes from **left-channel**
>   voice activity. The mic is never clustered, by construction.
> - Reprocess recomputes both from the file — no sidecar timeline, no masking.
> - Transcription still downmixes (L+R) to the same summed mono it uses today; playback and bundle
>   export downmix to mono so shared/played audio is normal.
> - A `meetings.audio_layout` flag (`summed` | `split`, in the bundle manifest for portability)
>   distinguishes recordings; existing `summed` recordings keep the original blind-clustering +
>   voiceprint path. On a stereo render endpoint this is **size-neutral** (the file was already 2ch
>   with the mic summed into both).
>
> This **supersedes** `MicActivity` / `mic_activity.json` / masked diarization from Phase 3; the
> capture-silence handling, `diarize_samples`, and the `build_name_map` naming are retained. Segment
> IDs and the names-in-DB rule are unchanged.