113 Commits
Author SHA1 Message Date
iamdoubz 21f3d76ed0 Merge pull request 'Feature chore bug 009' (#24) from feature_chore_bug_009 into main
Reviewed-on: #24
2026-07-15 18:13:11 -05:00
iamdoubz 19bd5815c0 chore: bump version 0.7.2 2026-07-15 18:09:07 -05:00
iamdoubz a552ddb1fa fix(ui): action items as two-row cards so the action text is always readable in the narrow pane
Text row (confirm + full-width text + delete) over a wrapping meta row
(owner, due date, reminder). Also void the seekNonce dependency read to
satisfy no-unused-expressions.
2026-07-15 17:41:38 -05:00
iamdoubz de25ac70e7 fix(notes): adopt server-side notes.md changes (speaker rename, reprocess) into the open editor buffer
Also untangles the a11y svelte-ignore comment from its prose so eslint's
no-unused-svelte-ignore stops parsing the explanation as ignore codes.
2026-07-15 17:41:37 -05:00
iamdoubz 28d9b2988a fix(spk): live split attribution + honest model reporting + notes rename rewrite (FR-SPK, T3.4)
- live diarization tick on split recordings now uses attribute_split (mic
  channel = 'You' from the first pass) instead of whole-mix clustering +
  voiceprint matching, which never reliably surfaced 'You' before stop
- attribute_split assigns via diarization::assign_split (channel-total
  overlap) so the user's speech can't be swallowed by a longer far-side span
- stop_recording overlays live user renames over the automatic name map
- rename_speaker/map_speaker_to_participant rewrite notes.md's baked-in
  **Name:** dialogue tags so the Notes pane follows the Speakers pane
- model_used records the ONNX model when the NPU/DirectML engine is routed
  (was reporting the unused ggml model id) + info log naming the encoder EP
2026-07-15 17:41:28 -05:00
iamdoubz f24adc3a59 fix(diarization): channel-total split attribution so mic speech can't be swallowed by a far span (FR-SPK) 2026-07-15 17:33:40 -05:00
iamdoubz 94425d456c docs(release): v0.7.1 release notes with SHA-256 checksums 2026-07-15 14:44:01 -05:00
iamdoubz 49cdbd6a54 chore: bump version 0.7.1 2026-07-15 14:38:31 -05:00
iamdoubz b4d72622da docs(data): close_to_tray setting in settings.json schema 2026-07-15 14:02:22 -05:00
iamdoubz c1ccdce497 docs(api): toggle_microphone_mute command + recording://mic event 2026-07-15 14:02:21 -05:00
iamdoubz a319dd9e97 feat(i18n): mute + close-to-tray strings 2026-07-15 14:01:43 -05:00
iamdoubz 6a01afbba6 feat(settings): default close_to_tray in settings store 2026-07-15 14:01:42 -05:00
iamdoubz 8c6a0ea25c feat(settings): close-to-tray toggle in Settings 2026-07-15 14:01:41 -05:00
iamdoubz 4a000f07a2 feat(app): M-key + mic mute button in recording controls 2026-07-15 14:01:40 -05:00
iamdoubz 0e2a3bb110 feat(recording): micMuted state + toggleMute synced to recording://mic 2026-07-15 13:59:04 -05:00
iamdoubz 95b3d34713 feat(api): toggleMicrophoneMute + recording://mic event + close_to_tray setting 2026-07-15 13:58:18 -05:00
iamdoubz 588354f488 fix(tray): remove duplicate trayIcon from config — one tray icon only 2026-07-15 13:57:38 -05:00
iamdoubz ee08f2bc88 feat(app): single tray with Show/Quit menu, close-to-tray, register mute cmd 2026-07-15 13:57:37 -05:00
iamdoubz a71c17965f feat(recording): toggle_microphone_mute command + recording://mic event 2026-07-15 13:57:36 -05:00
iamdoubz 3c73ec1cd4 feat(settings): close_to_tray setting (default on) 2026-07-15 13:57:35 -05:00
iamdoubz 8f805b8ef5 feat(audio): live mic mute — zero mic samples via CaptureHandle.set_muted (FR-CAP-7) 2026-07-15 13:57:33 -05:00
iamdoubz f83234bde1 docs(api): document import_media (model, background) + import://progress event 2026-07-15 13:46:50 -05:00
iamdoubz d4b1578478 feat(i18n): transcript.transcribed_with label 2026-07-15 13:46:01 -05:00
iamdoubz 185e4ec803 feat(import): show import tracker + 'transcribed with' model in meeting view 2026-07-15 13:46:00 -05:00
iamdoubz 30f4a4d3ec feat(import): init imports store for live progress tracking 2026-07-15 13:45:59 -05:00
iamdoubz 86c3587d19 feat(i18n): import model/tracker/phase strings; background import copy 2026-07-15 13:44:00 -05:00
iamdoubz d736b78ecb feat(import): model selector + ffmpeg/yt-dlp download links in add-meeting dialog 2026-07-15 13:43:26 -05:00
iamdoubz 98f6fcdc16 feat(import): ImportTracker Domino's-style 4-step progress component 2026-07-15 13:42:19 -05:00
iamdoubz 683838bef1 feat(import): imports store tracking per-meeting import phase progress 2026-07-15 13:41:08 -05:00
iamdoubz c63cabaf88 feat(import): api model param + import://progress event and types 2026-07-15 13:39:55 -05:00
iamdoubz fe355f034d feat(import): run import in background with per-phase progress + model override 2026-07-15 13:39:13 -05:00
iamdoubz e0e9377273 feat(storage): set_meeting_status for background import status transitions 2026-07-15 13:35:06 -05:00
iamdoubz a95eafe952 docs(api): enhance_notes, monitor_audio_level, stress_test_hardware + events 2026-07-15 09:50:17 -05:00
iamdoubz 5b9e0ed1f7 feat(app): register enhance_notes, monitor_audio_level, stress_test_hardware 2026-07-15 09:50:16 -05:00
iamdoubz 6c343c4204 feat(hardware): enhance_notes, monitor_audio_level, stress_test_hardware commands 2026-07-15 09:50:15 -05:00
iamdoubz 7d16c864e8 feat(ui): i18n for notes editor, vault, and hardware test/stress 2026-07-15 09:50:13 -05:00
iamdoubz e461d4eb22 feat(ui): enhanceNotes/monitorAudioLevel/stressTestHardware bindings + events + types 2026-07-15 09:50:12 -05:00
iamdoubz b14728e1c8 feat(ui): vault lock card, audio device test meter, quick hardware stress test 2026-07-15 09:50:00 -05:00
iamdoubz b7e6f54d91 feat(ui): header reshuffle — actions far-left, template beside them, drop wordmark 2026-07-15 09:49:58 -05:00
iamdoubz cf919f1499 fix(notes): refresh notes pane after re-transcribe; add AI-enhance + slash/toolbar
Re-sync the editor buffer after reprocess (was stale until restart). Adds a
Granola-style AI-enhance button (grounded in transcript, one-step undo), H3/
numbered/quote/divider toolbar buttons, and slash commands.
2026-07-15 09:49:57 -05:00
iamdoubz c616e0bfec Merge pull request 'Chores customize install' (#23) from chores_customize_install into main
Reviewed-on: #23
2026-07-14 21:40:02 -05:00
iamdoubz ecbe3e464c docs(release): v0.7.0 release notes with SHA-256 checksums 2026-07-14 20:01:04 -05:00
iamdoubz 7f6c0fac81 chore: bump version 0.7.0 2026-07-14 19:53:03 -05:00
iamdoubz 6fa9916f7e docs(readme): opt-in startup, install scope, enterprise deployment pointer 2026-07-14 16:27:02 -05:00
iamdoubz d78a426a7e docs(api): set_auto_start command 2026-07-14 16:27:01 -05:00
iamdoubz 1b36df020a docs(data-model): auto_start setting + wa-defaults.ini first-run seeding 2026-07-14 16:27:00 -05:00
iamdoubz 4af008da0f docs(deploy): enterprise deployment guide (wa-defaults.ini, install flags) 2026-07-14 16:26:59 -05:00
iamdoubz 223dc5f34b docs(adr): ADR-0012 launch-at-login & enterprise deployment defaults 2026-07-14 16:26:58 -05:00
iamdoubz 27773a03e5 feat(ui): i18n for launch-at-login and hardware refresh 2026-07-14 16:24:28 -05:00
iamdoubz 203e7b1bf6 feat(ui): launch-at-login toggle + refresh-hardware button 2026-07-14 16:24:27 -05:00
iamdoubz a6e1728346 feat(ui): setAutoStart store action + auto_start default 2026-07-14 16:24:26 -05:00
iamdoubz 26808970df feat(ui): setAutoStart binding + auto_start in AppSettings 2026-07-14 16:24:25 -05:00
iamdoubz 4a63b38ee0 chore(bundle): include wa-defaults.ini in the Vulkan release resources 2026-07-14 16:21:43 -05:00
iamdoubz 5d6563231c chore(bundle): bundle wa-defaults.ini + NSIS installMode both (per-user/all-users) 2026-07-14 16:21:42 -05:00
iamdoubz 2edd0382d3 feat(deploy): ship commented wa-defaults.ini template (no-op until edited) 2026-07-14 16:21:41 -05:00
iamdoubz 1530bca976 feat(app): register autostart plugin, first-run deploy seed, auto_start reconcile 2026-07-14 16:21:35 -05:00
iamdoubz 55c5143de0 feat(deploy): first-run wa-defaults.ini settings seeding
Reads an admin-supplied .ini once before settings.json exists; coerces
values onto typed Settings via serde; drops secret-looking keys.
2026-07-14 16:21:34 -05:00
iamdoubz 47071b5861 feat(app): set_auto_start command + default_settings/save_settings visibility 2026-07-14 16:21:33 -05:00
iamdoubz ded5e00750 feat(app): add auto_start setting (opt-in, default off, NFR-RES-4) 2026-07-14 16:21:26 -05:00
iamdoubz 2fa3ea6824 feat(app): add tauri-plugin-autostart dependency
Pulls in the official launch-at-login plugin (per-user HKCU\Run, no admin).
Regenerated ACL/schema manifests reflect the new plugin.
2026-07-14 16:21:25 -05:00
iamdoubz 14300fccee Remove temp changelog files 2026-07-14 15:54:41 -05:00
iamdoubz 2c8958da01 Bump version 0.6.0 2026-07-14 15:48:50 -05:00
iamdoubz f2c945838c docs(plan): mark dual-channel capture implemented (FR-SPK) 2026-07-14 12:22:59 -05:00
iamdoubz 0e0f3a09f1 docs(architecture): per-stream diarization via split channels in stop data flow (FR-SPK) 2026-07-14 12:22:57 -05:00
iamdoubz 3d183fdd7c docs(data-model): audio.wav split layout + audio_layout column; drop mic_activity.json (FR-SPK) 2026-07-14 12:22:56 -05:00
iamdoubz 32bb4057ec docs(adr-0005): Phase 3.5 dual-channel capture supersedes the masked-mono timeline (FR-SPK) 2026-07-14 12:22:55 -05:00
iamdoubz 90a9cbd476 feat(recording): fold split stereo to mono on playback, dual-mono on export (FR-SPK)
fold_wav folds a WAV's channels to their average across N output channels.
serve_recording plays it as mono so a split (mic-left/loopback-right) recording
plays both sources in both ears; bundle export decrypts + folds to dual-mono so
the shared file is a normal stereo file (manifest records it as summed). Test
covers mono + dual-mono.
2026-07-14 12:21:12 -05:00
iamdoubz 5bd76b80e8 feat(diarization): channel-based split attribution; retire masked-mono path (FR-SPK)
stop + reprocess now, for a split recording, diarize the right (loopback)
channel into Speaker N and take You from left-channel (mic) VAD — read back from
the file so both agree. Fixes reprocess collapsing to one speaker. Summed
recordings keep the whole-signal + voiceprint fallback. Removes MicTimelineFile,
phase3_attribute, mask_ranges, mic_activity.json (retirement note retained).
2026-07-14 12:16:03 -05:00
iamdoubz a1d71d1c73 refactor(recording): drop RecordingSession.mic_activity (retired) (FR-SPK) 2026-07-14 12:16:02 -05:00
iamdoubz e287fa5ebe feat(audio): read_wav_channel_16k + vad_spans; retire MicActivity (FR-SPK)
Add a channel-selective 16k reader (share the decode with read_wav_mono_16k via
read_wav_16k) and vad_spans (100ms energy-windowed You detection). Remove the
MicActivity live timeline + its capture wiring — the split layout makes the
mic/far-side separation intrinsic to the file, so You is recomputed from the
left channel instead.
2026-07-14 12:16:01 -05:00
iamdoubz 534f5ca350 feat(commands): mark recordings split/summed at stop from the session layout (FR-SPK) 2026-07-14 12:04:46 -05:00
iamdoubz 1e99c4c747 feat(recording): RecordingSession.audio_layout (split when mic on) (FR-SPK) 2026-07-14 12:04:45 -05:00
iamdoubz 5e5f61f3b7 feat(audio): write_wav_split — stereo mic-left/loopback-right capture (FR-SPK)
New split writer + a `split` flag through the capture path: when the mic is on,
audio.wav is recorded 2ch with the mic on the left channel and the loopback
(downmixed) on the right, keeping the two sources separate in the file. Same
byte size as today's summed stereo. Transcription still downmixes L+R (= the
summed signal). Unit test covers L=mic / R=loopback.
2026-07-14 12:04:44 -05:00
iamdoubz dec9251e7b feat(commands): set audio_layout at each finalize/export site (summed for now) (FR-SPK) 2026-07-14 11:58:25 -05:00
iamdoubz d4e2ab494c feat(models): MeetingBundle.audio_layout (serde default) for portable layout (FR-SPK) 2026-07-14 11:58:23 -05:00
iamdoubz 845903a772 feat(storage): thread audio_layout through FinalizeMeeting + Meeting + get_meeting (FR-SPK) 2026-07-14 11:58:22 -05:00
iamdoubz 998de62df2 feat(storage): migration 0009 — meetings.audio_layout column (FR-SPK/FR-CAP) 2026-07-14 11:58:21 -05:00
iamdoubz e449c194ef docs(plan): draft dual-channel (mic-L/loopback-R) capture plan + ADR-0005 amendment for review (FR-SPK) 2026-07-14 11:28:16 -05:00
iamdoubz 6cea767dec fix(audio): drain mic device-paced during silence to stop playback skips/ticks (FR-CAP-7)
The capture-silence fix metered the mic pull by wall-clock, but the mic fills
the bridge at its own device clock. Over a long silent stretch they drifted, the
bridge backed up to its 0.5s cap and dropped oldest samples -> the recording
skipped/'fast-forwarded' late in the gap. And a 10ms idle threshold punched
silence into the loopback on brief inter-packet stalls -> audible ticks. Now:
drain the whole bridge (device-paced, no overflow) and only synthesize after a
sustained 150ms gap.
2026-07-14 11:10:07 -05:00
iamdoubz 60ae97c771 docs(plan): mark Phase 3 slices a–d landed (FR-SPK) 2026-07-14 00:53:07 -05:00
iamdoubz d8587b8e93 docs(architecture): note per-stream diarization in the stop data flow (FR-SPK) 2026-07-14 00:53:06 -05:00
iamdoubz 695a12dc2d docs(data-model): add mic_activity.json to the meeting dir list (FR-SPK) 2026-07-14 00:53:05 -05:00
iamdoubz 6385e38b33 docs(adr-0005): amend with Phase 3 per-stream 'You' attribution (FR-SPK) 2026-07-14 00:53:04 -05:00
iamdoubz 0b5a85f461 feat(diarization): persist mic_activity.json; reprocess uses the Phase 3 timeline (FR-SPK)
Factor the Phase 3 attribution (you_spans -> mask -> diarize far side -> merge +
name) into a shared phase3_attribute() used by both stop and reprocess. stop now
writes mic_activity.json (the 'You' spans) next to a retained audio.wav
(plaintext timing metadata, local-only, lives/dies with audio.wav); reprocess
prefers that timeline over the voiceprint fallback so re-transcription keeps
correct per-stream 'You'/'Speaker' attribution.
2026-07-14 00:51:41 -05:00
iamdoubz fd9311c482 feat(diarization): Phase 3 stop attribution — mic timeline is 'You', diarize far side masked (FR-SPK)
stop_recording now, when the mic is on, takes 'You' spans straight from the
MicActivity timeline and diarizes the recording with those ranges zeroed, so
sherpa only ever clusters the far side into Speaker N. Merges the span lists and
names them via the existing voiceprint map (You + Speaker 2..). Falls back to
the whole-signal pass + voiceprint match when the mic is off. Adds mask_ranges
helper + tests.
2026-07-14 00:04:58 -05:00
iamdoubz 39294adcbc feat(audio): populate MicActivity timeline from the loopback writer (FR-SPK) 2026-07-14 00:04:57 -05:00
iamdoubz fee18ebc59 refactor(diarization): expose first_appearance_order + build_name_map to crate (FR-SPK) 2026-07-14 00:04:56 -05:00
iamdoubz 51f5ff085c feat(diarization): Diarizer::diarize_samples + short-input guard for masked far-side pass (FR-SPK) 2026-07-14 00:04:55 -05:00
iamdoubz 6d4204fb22 feat(recording): RecordingSession.mic_activity for Phase 3 attribution (FR-SPK) 2026-07-14 00:04:54 -05:00
iamdoubz 185489fa18 feat(audio): MicActivity timeline for Phase 3 per-stream 'You' attribution (FR-SPK)
Records, per 100ms of audio.wav frame time, when the mic (the user) was
speech-level — captured in the loopback writer where mic and loopback exist
separately in the recording's own timebase. you_spans() collapses active
windows into merged 'You' ranges. The far side will be diarized with these
ranges masked out, so clustering never sees the mic. Struct + span logic land
first (tested); capture wiring + stop attribution follow.
2026-07-13 21:15:49 -05:00
iamdoubz 41c2c79797 docs(plan): finalize Phase 3 design — mic=You, diarize loopback-masked (FR-SPK) 2026-07-13 21:15:48 -05:00
iamdoubz 6dd6c3c69f fix(audio): capture mic during system-audio silence (FR-CAP-7)
WASAPI loopback delivers no packets when the render endpoint is idle, so the
recording only advanced while system audio played. The bridged mic (the user
talking over silence) was lost to MicBridge overflow, and audio.wav lost those
stretches entirely — corrupting diarization and dropping the user's solo
remarks in real meetings. When loopback is idle, synthesize silence for the
elapsed gap and fold the mic in, keeping audio.wav continuous. Loopback +
mic-enabled only.
2026-07-13 20:58:23 -05:00
iamdoubz 9468781218 fix(audio): gate voiceprint capture on mic energy so it's real speech (FR-SPK)
VoiceSample kept the first N samples unconditionally, so a user silent for the
first 30s got a voiceprint of room noise/a cough. That noise embedding
spuriously matched a cluster and mislabeled every speaker 'You', even during
silence. Skip sub-threshold (VOICE_ENERGY_FLOOR) chunks so the window tracks
actual speech; until ~1s of real voice accumulates, the match returns empty
(no premature 'You').
2026-07-13 20:35:25 -05:00
iamdoubz b39352dbff fix(diarization): clear stale speaker rows on reprocess so Participants pane matches (FR-SPK)
Reprocess re-diarizes and upserts the new labels but never deleted the old
ones, so get_meeting (Participants pane) kept returning the discarded run's
labels (e.g. 83) while the transcript showed the new 5. Clear the meeting's
speaker rows before finalize re-inserts the fresh set.
2026-07-13 20:09:03 -05:00
iamdoubz 0d263637df feat(storage): Store::clear_speakers to drop a meeting's speaker rows (FR-SPK) 2026-07-13 20:09:01 -05:00
iamdoubz bb6e222101 docs(plan): Phase 3 per-stream attribution design sketch + ADR-0005 review (FR-SPK) 2026-07-13 19:32:15 -05:00
iamdoubz 85ed3b5587 docs(api-contracts): note transcript://segment re-emit + live diarization://updated (FR-SPK) 2026-07-13 19:25:31 -05:00
iamdoubz 68d692fbe8 fix(transcript): resolve live speaker names incl 'You' in the recording view (FR-SPK) 2026-07-13 19:25:30 -05:00
iamdoubz fa664e5147 feat(recording): live speakers state from diarization://updated (FR-SPK) 2026-07-13 19:25:29 -05:00
iamdoubz b5c99621b5 feat(api): onDiarizationUpdated event listener for live speaker updates (FR-SPK) 2026-07-13 19:25:28 -05:00
iamdoubz db52b9e812 feat(diarization): live tick re-emits relabeled segments + live 'You' (FR-SPK)
Phase 2 backend. The 15s provisional pass now (1) runs the mic voiceprint
match every tick and merges 'You'/'Speaker N' into the session name map without
overwriting user-set names, and (2) re-emits every committed segment whose
speaker changed via transcript://segment (replace-by-id) so the live transcript
reflects the refined labels instead of a frozen 'S1'.
2026-07-13 19:25:27 -05:00
iamdoubz 319e7a31cc docs(data-model): add voiceprint.wav to the meeting dir file list (FR-SPK) 2026-07-13 19:19:06 -05:00
iamdoubz 25011d0f27 fix(diarization): reprocess re-diarizes + persist mic voiceprint.wav (FR-SPK)
Phase 1. stop_recording writes voiceprint.wav next to a retained audio.wav
(ADR-0009 gated, sealed at rest with the vault); reprocess_transcript now
re-diarizes the fresh transcript and rebuilds the speaker name map from the
voiceprint instead of reusing the stale original-run names, fixing the
collapse to a single 'You' speaker on re-transcription.
2026-07-13 19:19:06 -05:00
iamdoubz 7733e86cfd feat(audio): write_wav_mono_16k helper for persisting the mic voiceprint (FR-SPK) 2026-07-13 19:19:05 -05:00
iamdoubz a481f5dcbf fix(diarization): tame cluster explosion with threshold 0.7 + 700ms min span (FR-SPK)
Phase 0 of the speaker-accuracy plan. sherpa fast-clustering threshold 0.5
over summed mic+loopback mono exploded a 2-speaker call into 83 clusters;
raise to 0.7 (larger threshold -> fewer clusters) and drop sub-700ms spans
whose embeddings are unstable and only churn labels in assign_by_overlap.
2026-07-13 19:12:04 -05:00
iamdoubz bd6519b65e docs(plan): phased fix plan for diarization speaker accuracy (FR-SPK) 2026-07-13 18:40:44 -05:00
iamdoubz 4f3af98bfa Add Clippy marketing image 2026-07-13 16:15:18 -05:00
iamdoubz 77f9a8b518 docs(readme): bump status to v0.5.2 and add UI-localization comparison row 2026-07-13 16:07:12 -05:00
iamdoubz 2a7e6229f4 docs(readme): list the 20 available interface languages and RTL support 2026-07-13 16:05:12 -05:00
iamdoubz 64e7baad66 docs(release): v0.5.2 release notes with SHA-256 checksums 2026-07-13 16:00:00 -05:00
iamdoubz 804d451bfa chore(release): bump version to 0.5.2 2026-07-13 15:59:58 -05:00
iamdoubz a29e60b47d fix(storage): show native error dialog on startup DB failure instead of panic
A migration-checksum mismatch (an edited-after-shipping migration) made
SqliteStore::connect() return Err, which .expect() turned into a panic=abort
fast-fail: the window never opened and the OS reported only a 0xc0000409 crash.
Replace the expect with a blocking MessageBoxW that names the failure and points
at wa.db, then exit(1) cleanly.
2026-07-13 15:52:55 -05:00
iamdoubz 969ed6022e build(deps): enable Win32_UI_WindowsAndMessaging for startup error dialog 2026-07-13 15:52:54 -05:00
44 changed files with 4519 additions and 527 deletions
+25 -7
View File
@@ -1,3 +1,7 @@
<p align="center">
<img src="bonus/clippy-says-hej.png" alt="WhispAssist logo with friendly Clippy saying hi there showing open source, private, note taker, and meeting advisor">
</p>
# WhispAssist (WA)
**A privacy-first, Windows-native meeting assistant that runs entirely on-device.**
@@ -7,7 +11,7 @@ on-device acceleration (**NPU → GPU → CPU**), labels speakers, structures th
Markdown notes, and optionally augments them with a locally hosted LLM (Ollama). Audio and
transcripts **never leave the machine** unless you explicitly configure a destination.
> **Status: working application (v0.4.0).** Capture (system audio **+ your microphone**, with a
> **Status: working application (v0.5.2).** Capture (system audio **+ your microphone**, with a
> live **dual level meter**), transcription (CPU / Intel NPU / Vulkan GPU) with a **fluid live
> transcript**, speaker diarization, storage + crash recovery, local-LLM summaries, AI tags,
> opt-in recording with **in-app playback**, at-rest encryption, self-hosted sync, **Outlook
@@ -27,6 +31,7 @@ transcripts **never leave the machine** unless you explicitly configure a destin
| **NPU / GPU auto-acceleration** | n/a (cloud) | ⚠️ CPU/GPU, manual | ✅ **NPU→NVIDIA→AMD→Intel→CPU** ladder, one binary |
| **Speaker diarization** | ✅ cloud | ⚠️ limited | ✅ offline (sherpa-onnx) |
| **Summaries / notes AI** | ☁️ cloud LLM | ✅ local (Ollama) / BYO | ✅ local (Ollama, localhost **or LAN**) + optional hosted |
| **Interface localization (UI language)** | ⚠️ English-only UI (multilingual transcription) | ⚠️ English-only UI (more planned) | ✅ **20 languages** + RTL (Arabic/Urdu) |
| **Default data egress** | ☁️ audio + notes to cloud | 🔒 local (cloud optional) | 🔒 **none** — everything off by default, allowlist-enforced |
| **Calendar / Outlook `.pst` context** | ✅ cloud calendar | ❌ | ✅ local `.pst` import + **auto-record on events** |
| **Import an existing recording (file/URL)** | ❌ live capture only | ❌ | ✅ any file or URL (ffmpeg + yt-dlp) |
@@ -51,7 +56,7 @@ accelerated, zero-egress-by-default** option: it exploits the NPU/GPU in modern
everything on the device unless you opt in, and adds Windows-specific context (Outlook) and a
coding-agent handoff.
## What's built (v0.4.0)
## What's built (v0.5.2)
- **Bot-free capture — now both sides, with a live dual meter** — WASAPI loopback records the
system mix (all participants), and an optional **microphone** path captures your own voice,
@@ -103,6 +108,14 @@ coding-agent handoff.
(Claude, Codex, Copilot, OpenCode) over a **loopback-only, token-gated** MCP server that is off
by default, scope-limited, audited, and **adds no egress** — data leaves only via the agent's
own provider, which WA discloses.
- **20 interface languages, with RTL** — the whole UI (shell, meetings list, transcript/notes,
summary, and Settings) is available in **20 languages**, chosen under **Settings ▸ Language**:
Arabic (العربية), Bengali (বাংলা), Simplified Chinese (中文简体), English, Finnish (Suomi),
French — France & Canada (Français), German (Deutsch), Hindi (हिन्दी), Korean (한국어),
Norwegian Bokmål (Norsk bokmål), Portuguese — Brazil & Portugal (Português), Russian (Русский),
Sinhala (සිංහල), Spanish — Spain & Mexico (Español), Swedish (Svenska), Tamil (தமிழ்), and
Urdu (اردو). Selecting Arabic or Urdu flips the interface to **right-to-left**; any untranslated
string falls back to English rather than showing a raw key. Adding a language is a one-file job.
- **One universal installer** — a single signed MSI and NSIS `-setup.exe` that covers every
machine: Vulkan for all GPUs, the Intel NPU path (+ DirectML fallback), and CPU. The Vulkan
loader is bundled so it launches even on machines without a GPU driver.
@@ -125,10 +138,15 @@ during setup).
On first run WA detects your hardware and picks the best transcription backend
(**NPU → NVIDIA → AMD → Intel → CPU**). It works on your **CPU or GPU** (GPU via Vulkan) out of the
box; to use an Intel **NPU**, open **Settings ▸ Hardware** and download the one-time NPU
acceleration package. The app runs without admin rights, never adds itself to startup, and keeps all
data under `%LOCALAPPDATA%\WhispAssist`.
acceleration package. The app runs without admin rights, does not add itself to startup unless you
opt in (**Settings ▸ Recording ▸ Launch at login**), and keeps all data under
`%LOCALAPPDATA%\WhispAssist`.
Prefer the NSIS installer? Grab **`WhispAssist_<version>_x64-setup.exe`** from the same page.
Prefer the NSIS installer? Grab **`WhispAssist_<version>_x64-setup.exe`** from the same page — it
lets you choose a **current-user** (no admin) or **all-users** install.
**Deploying to many machines?** See [`docs/enterprise-deployment.md`](docs/enterprise-deployment.md)
for silent install, custom install location, and presetting defaults with a `wa-defaults.ini` file.
## Optional dependencies
@@ -142,7 +160,7 @@ If you do not have these installed, WhispAssist will still work, but some featur
WhispAssist is a **Tauri 2** application: a small Rust core with a compiled **Svelte + TypeScript**
frontend rendered through the OS WebView2 (no bundled browser → low idle memory). Every decision
and its alternatives are recorded in [`docs/adr/`](docs/adr/) (ADR-0001–0011).
and its alternatives are recorded in [`docs/adr/`](docs/adr/) (ADR-0001–0012).
- **Shell / IPC:** Tauri 2 (Rust ⇄ WebView2)
- **Audio capture:** WASAPI loopback
@@ -163,7 +181,7 @@ and its alternatives are recorded in [`docs/adr/`](docs/adr/) (ADR-0001–0011).
WhispAssist/
├── docs/ # The engineering plan (read this first)
│ ├── 00-overview.md … 07-research-findings.md
│ └── adr/ Architecture Decision Records (0001–0011)
│ └── adr/ Architecture Decision Records (0001–0012)
├── src-tauri/ # Rust core — implemented service modules:
│ └── src/{audio,transcription,diarization,storage,llm,calendar,
│ hardware,notes,sync,mcp,vault}
-96
View File
@@ -1,96 +0,0 @@
# WhispAssist v0.5.0
**Privacy-first, Windows-native meeting assistant — everything on-device, nothing leaves unless you say so.**
This release is about **living with your recordings**: play back a meeting while following along in
the transcript, manage action items by hand, move recordings between computers, drop a meeting into
your Obsidian vault, and — new in this release — **use WhispAssist in your own language**. Everything
stays off-by-default and local-first.
One universal installer (**MSI** and **NSIS**) covers every machine: **Vulkan** for all GPUs
(NVIDIA/AMD/Intel), the Intel **NPU** (OpenVINO), a **DirectML** fallback, and **CPU**.
---
## ✨ Highlights
### Click the transcript to play that moment
The recording player and the transcript now talk to each other. **Click any transcript line to jump
the audio to that moment** and start playing, and as playback runs the **current line highlights and
scrolls into view** so you never lose your place. If you scroll by hand, auto-scroll steps aside for
a few seconds so it doesn't fight you.
### Interface language selector (i18n)
WhispAssist can now be **fully translated**. A new **Settings ▸ Language** picker switches the
interface language, English ships as the baseline, and **every** user-facing string across the app —
the shell, meetings list, transcript/notes, summary panel, and all of Settings — now flows through a
single translation layer. Adding a new language is as simple as translating **one JSON file**; no
code changes. (This release ships English; the groundwork is in place for community translations.)
### Manage action items yourself
Action items are no longer just whatever the summary extracted. You can now **add, edit, and delete
them by hand** in the summary panel, set an owner and due date, and toggle a local reminder. Your
edits are the source of truth and are reconciled cleanly — deleting an item also cancels its reminder.
### Move recordings between computers (Export / Import)
A new **Export & import** section in **Settings ▸ Storage** writes each meeting as a portable
**bundle folder** (audio, transcript, notes, summary, and a `meeting.json` manifest) and imports them
back on another machine. Imported meetings get a fresh id, so re-importing never overwrites anything.
Export to any folder — a synced drive, a USB stick, or a sync target's local mount — and carry it across.
### Export a meeting to Obsidian
A new **Obsidian** export writes a single self-contained vault note — YAML frontmatter
(title, date, duration, participants, tags) plus notes, summary, action items, and a timestamped
transcript — **without the audio**. Drop it in your vault and the transcript is fully searchable.
---
## 🚀 Also new since v0.4.0
- **Per-segment transcript timestamps** — every line now shows a quiet `m:ss` (or `h:mm:ss`) time
prefix, in both the live and finalized views.
- **Auto-resync on edit** — when sync is enabled, editing a meeting's notes, summary, transcript,
tags, or action items **re-uploads just the changed artifacts** automatically (deduped by hash, so
an unchanged save uploads nothing). Off unless sync is configured.
## 🐛 Fixes & polish
- Transcript scroll-intent handling refined so playback auto-scroll never yanks you back while you're
reading.
- Import preserves each meeting's original date rather than stamping the import time.
---
## 📦 Install
**Requirements:** Windows 10 or 11 (x64). WhispAssist needs **WebView2** (preinstalled on Windows 11;
the installer fetches it on Windows 10). Importing from a file/URL additionally needs **`ffmpeg`**
(and **`yt-dlp`** for URLs) on your PATH — install those separately.
1. Download **`WhispAssist_0.5.0_x64_en-US.msi`** (or the NSIS **`WhispAssist_0.5.0_x64-setup.exe`**).
2. Run it and accept the UAC prompt. If SmartScreen appears, choose **More info → Run anyway**.
3. Launch **WhispAssist** from the Start menu.
On first run WA picks the best transcription backend (**NPU → NVIDIA → AMD → Intel → CPU**). It runs
without admin rights, never adds itself to startup, and keeps all data under
`%LOCALAPPDATA%\WhispAssist`.
## 🔐 Checksums (SHA-256)
```
b423feed1a1171384e46c5e0b5aa63cbf912bfe8f29cc89a5a027bb15358075c WhispAssist_0.5.0_x64_en-US.msi
2dce26b5603a5d02094533c594d814a3d72e05512e1d735ea69e1e309f15b710 WhispAssist_0.5.0_x64-setup.exe
```
Verify after download:
```powershell
Get-FileHash .\WhispAssist_0.5.0_x64_en-US.msi -Algorithm SHA256
```
---
## Privacy, unchanged
Everything optional is **off by default**. With nothing configured, WhispAssist makes **no content
egress at all**. Recording is opt-in; sync/AI credentials live only in the OS credential store; the
MCP server is loopback-only and adds no egress. The reachable-host allowlist is derived from your
settings and enforced in the core.
-73
View File
@@ -1,73 +0,0 @@
# WhispAssist v0.5.1
**Privacy-first, Windows-native meeting assistant — everything on-device, nothing leaves unless you say so.**
v0.5.0 put the **i18n groundwork** in place. v0.5.1 fills it in: WhispAssist now ships in
**20 languages**, so you can run the whole app — the shell, meetings list, transcript/notes, summary
panel, and every corner of Settings — in your own language. Everything stays off-by-default and
local-first; this is a UI-language release with no change to what leaves your device (nothing, by default).
One universal installer (**MSI** and **NSIS**) covers every machine: **Vulkan** for all GPUs
(NVIDIA/AMD/Intel), the Intel **NPU** (OpenVINO), a **DirectML** fallback, and **CPU**.
---
## ✨ Highlights
### 20 interface languages
Pick your language under **Settings ▸ Language**. Alongside English, this release adds full
translations for:
- **Arabic** (العربية) · **Bengali** (বাংলা) · **German** (Deutsch)
- **Spanish** — Spain (Español, España) and **Mexico** (Español, México)
- **Finnish** (Suomi) · **French** — France (Français, France) and **Canada** (Français, Canada)
- **Hindi** (हिन्दी) · **Korean** (한국어) · **Norwegian Bokmål** (Norsk bokmål)
- **Portuguese** — Brazil (Português, Brasil) and **Portugal** (Português, Portugal)
- **Russian** (Русский) · **Sinhala** (සිංහල) · **Swedish** (Svenska)
- **Tamil** (தமிழ்) · **Urdu** (اردو) · **Mandarin Chinese, Simplified** (中文简体)
Your choice persists across launches, and any untranslated string quietly falls back to English rather
than showing a raw key — so partial translations degrade gracefully.
### Right-to-left layout
Selecting **Arabic** or **Urdu** flips the whole interface to **right-to-left**, so those languages
read and lay out correctly rather than being crammed into an LTR shell.
Adding a further language remains a one-file job — drop in a single JSON translation, no code changes.
---
## 📦 Install
**Requirements:** Windows 10 or 11 (x64). WhispAssist needs **WebView2** (preinstalled on Windows 11;
the installer fetches it on Windows 10). Importing from a file/URL additionally needs **`ffmpeg`**
(and **`yt-dlp`** for URLs) on your PATH — install those separately.
1. Download **`WhispAssist_0.5.1_x64_en-US.msi`** (or the NSIS **`WhispAssist_0.5.1_x64-setup.exe`**).
2. Run it and accept the UAC prompt. If SmartScreen appears, choose **More info → Run anyway**.
3. Launch **WhispAssist** from the Start menu.
On first run WA picks the best transcription backend (**NPU → NVIDIA → AMD → Intel → CPU**). It runs
without admin rights, never adds itself to startup, and keeps all data under
`%LOCALAPPDATA%\WhispAssist`.
## 🔐 Checksums (SHA-256)
```
911618e6996a079bad1cbe46731c89ee247f3f5f17c09b60091d7e6a5e7cbef0 WhispAssist_0.5.1_x64_en-US.msi
ffabbd74def981da176931349b2813a3927d7376c0d18e9d191891abfc17c385 WhispAssist_0.5.1_x64-setup.exe
```
Verify after download:
```powershell
Get-FileHash .\WhispAssist_0.5.1_x64_en-US.msi -Algorithm SHA256
```
---
## Privacy, unchanged
Everything optional is **off by default**. With nothing configured, WhispAssist makes **no content
egress at all**. Recording is opt-in; sync/AI credentials live only in the OS credential store; the
MCP server is loopback-only and adds no egress. The reachable-host allowlist is derived from your
settings and enforced in the core.
+78
View File
@@ -0,0 +1,78 @@
# WhispAssist v0.7.0
**Privacy-first, Windows-native meeting assistant — everything on-device, nothing leaves unless you say so.**
This release is about **control over install and startup** — for a single user and for admins rolling
WhispAssist out across a fleet. You can now launch WhispAssist at login, choose a per-user (no-admin)
or all-users install, preset every default with a deployment file, and re-scan your hardware without
restarting. No feature here changes the privacy posture: everything optional stays off-by-default and
local-first.
One universal installer (**MSI** and **NSIS**) covers every machine: **Vulkan** for all GPUs
(NVIDIA/AMD/Intel), the Intel **NPU** (OpenVINO), a **DirectML** fallback, and **CPU**.
---
## ✨ New
### Launch at login (opt-in)
A new **Settings ▸ Recording ▸ Launch WhispAssist at login** toggle starts WhispAssist when you sign
in to Windows. It's **off by default**, needs **no admin** (a per-user startup entry), and does **not**
begin recording on its own.
### Choose a per-user or all-users install
The NSIS setup (`WhispAssist_0.7.0_x64-setup.exe`) now lets you install for **just yourself**
(no admin rights required) or for **all users**. The MSI remains the per-machine, all-users installer.
### Customize deployments with a file (`wa-defaults.ini`)
Admins can preset WhispAssist's defaults for every machine using native Windows tooling (Group Policy,
SCCM, Intune, `msiexec`) — no management console. Drop a `wa-defaults.ini` next to the installer or in
`%PROGRAMDATA%\WhispAssist\`, and each machine's **first launch** seeds its settings from it: record-
by-default, preferred backend, a model to auto-download, retention, storage location, AI provider, and
more. You can also set a **custom install location** with standard `msiexec INSTALLDIR=…` / NSIS `/D=`
flags. **Secrets are never read from this file** — API keys and tokens stay in the OS credential store.
Full key reference and silent-install examples in
[`docs/enterprise-deployment.md`](docs/enterprise-deployment.md).
### Refresh hardware
A **Refresh** button in **Settings ▸ Hardware** re-detects your GPU/NPU on the spot — handy after
installing a driver or plugging in an eGPU — no restart needed.
---
## 📦 Install
**Requirements:** Windows 10 or 11 (x64). WhispAssist needs **WebView2** (preinstalled on Windows 11;
the installer fetches it on Windows 10). Importing from a file/URL additionally needs **`ffmpeg`**
(and **`yt-dlp`** for URLs) on your PATH — install those separately.
1. Download **`WhispAssist_0.7.0_x64_en-US.msi`** (or the NSIS **`WhispAssist_0.7.0_x64-setup.exe`**).
2. Run it and accept the UAC prompt. If SmartScreen appears, choose **More info → Run anyway**.
3. Launch **WhispAssist** from the Start menu.
On first run WA picks the best transcription backend (**NPU → NVIDIA → AMD → Intel → CPU**). It runs
without admin rights, does **not** add itself to startup unless you opt in, and keeps all data under
`%LOCALAPPDATA%\WhispAssist`.
Deploying to many machines? See [`docs/enterprise-deployment.md`](docs/enterprise-deployment.md).
## 🔐 Checksums (SHA-256)
```
730676580dbef5c4bb2c155f46a73a5791722139ab144a3fc0286e3a4449c432 WhispAssist_0.7.0_x64_en-US.msi
6a34c16eb6f876fc92c7a79d37414b4fdd2c09f4f40873008149db36cd0a30aa WhispAssist_0.7.0_x64-setup.exe
```
Verify after download:
```powershell
Get-FileHash .\WhispAssist_0.7.0_x64_en-US.msi -Algorithm SHA256
```
---
## Privacy, unchanged
Everything optional is **off by default**. With nothing configured, WhispAssist makes **no content
egress at all**. Recording is opt-in; sync/AI credentials live only in the OS credential store; the
MCP server is loopback-only and adds no egress; the deployment file never carries secrets. The
reachable-host allowlist is derived from your settings and enforced in the core.
+94
View File
@@ -0,0 +1,94 @@
# WhispAssist v0.7.1
**Privacy-first, Windows-native meeting assistant — everything on-device, nothing leaves unless you say so.**
A quality-of-life release focused on **importing meetings, taking notes, and living in the
background**. Adding a meeting from a file or link now runs without freezing the app and shows you
exactly where it's up to; notes gained AI cleanup and quick formatting; and WhispAssist can now mute
your mic mid-meeting and tuck itself into the system tray. No feature here changes the privacy
posture: everything optional stays off-by-default and local-first.
One universal installer (**MSI** and **NSIS**) covers every machine: **Vulkan** for all GPUs
(NVIDIA/AMD/Intel), the Intel **NPU** (OpenVINO), a **DirectML** fallback, and **CPU**.
---
## ✨ New
### Add a meeting — now in the background, with a progress tracker
Importing a recording (a local audio/video file or a YouTube/streaming/direct URL) no longer blocks
the app while it works. Click **Import** and the meeting appears in your list immediately with a
**four-step progress tracker** — *Transcode → Transcribe → Identify speakers → Finalize* — where the
current step pulses and finished steps show how long they took. A 25-minute video that used to lock
the window for ~13 minutes now transcribes quietly in the background.
- **Pick the transcription model** right in the dialog, and see **"Transcribed with …"** on the
finished meeting so you always know how it was produced.
- **One-click links** to download `ffmpeg` and `yt-dlp` (still external, not bundled).
- A failed import stays in your list marked **error** instead of vanishing.
### Better notes
- **AI-enhance** (✨): turn rough notes into clean, structured notes using your local LLM, with a
one-step **Undo**. Off unless you have a local model configured.
- **Slash commands & a formatting toolbar**: type `/todo`, `/h1`, `/quote`, … or use the toolbar for
headings, lists, checkboxes, quotes, and dividers.
- **Fix:** notes no longer show stale text after re-transcribing a meeting — the pane updates in
place, no restart needed.
### Mute your microphone — press **M**
Mute/unmute the mic mid-meeting with the **M** key or the new mic button by the level meter. The mic
channel goes silent (recording, live transcript, and meter) while system/loopback audio keeps
capturing.
### Close to system tray
Closing the window now **keeps WhispAssist running in the background** instead of quitting. Reopen it
from the tray icon; the tray's **Quit** exits fully. On by default — toggle it in
**Settings ▸ Recording ▸ Close to system tray**. (This release also fixes a bug that showed **two**
WhispAssist icons in the tray — there's now just one.)
### Privacy & hardware odds and ends
- **Vault lock card** in **Settings ▸ Privacy**: lock/unlock the encrypted store and change its
password at a glance.
- **Test your audio devices**: a live level meter for your mic and system audio, plus a test tone.
- **Quick hardware stress test**: benchmark the available backends against your installed models and
apply the fastest real-time combination.
- Tidier recording header.
---
## 📦 Install
**Requirements:** Windows 10 or 11 (x64). WhispAssist needs **WebView2** (preinstalled on Windows 11;
the installer fetches it on Windows 10). Importing from a file/URL additionally needs **`ffmpeg`**
(and **`yt-dlp`** for URLs) on your PATH — the Import dialog now links to both.
1. Download **`WhispAssist_0.7.1_x64_en-US.msi`** (or the NSIS **`WhispAssist_0.7.1_x64-setup.exe`**).
2. Run it and accept the UAC prompt. If SmartScreen appears, choose **More info → Run anyway**.
3. Launch **WhispAssist** from the Start menu.
On first run WA picks the best transcription backend (**NPU → NVIDIA → AMD → Intel → CPU**). It runs
without admin rights, does **not** add itself to startup unless you opt in, and keeps all data under
`%LOCALAPPDATA%\WhispAssist`.
Deploying to many machines? See [`docs/enterprise-deployment.md`](docs/enterprise-deployment.md).
## 🔐 Checksums (SHA-256)
```
bd96a059db3658a9bee81161edc709cfc4741ed99d6c66a12194403b339f9369 WhispAssist_0.7.1_x64_en-US.msi
711fa7df618c8fc3f03c18d543abad2288d56f750b532288204e0ec0f9426595 WhispAssist_0.7.1_x64-setup.exe
```
Verify after download:
```powershell
Get-FileHash .\WhispAssist_0.7.1_x64_en-US.msi -Algorithm SHA256
```
---
## Privacy, unchanged
Everything optional is **off by default**. With nothing configured, WhispAssist makes **no content
egress at all**. Recording is opt-in; sync/AI credentials live only in the OS credential store; the
MCP server is loopback-only and adds no egress; the deployment file never carries secrets. The
reachable-host allowlist is derived from your settings and enforced in the core.
Binary file not shown.

After

Width:  |  Height:  |  Size: 728 KiB

+8 -4
View File
@@ -84,10 +84,14 @@ issues from feature briefs.
4. Provisional speaker turns (from cheap segmentation) tag segments live; the user may name a
speaker, applied immediately to past/future segments (FR-SPK-2).
5. The notes view renders Markdown that updates as segments arrive (FR-NOTE-1, FR-NOTE-6).
6. On `stop_recording`: finalize the WAV; run full `diarization`, align speaker IDs to segments,
apply name mappings; persist transcript JSON + metadata via `storage` (FR-STORE-1). If "Record
this meeting" is **off**, delete the working WAV **after** the transcript is finalized; if **on**,
keep it as `audio.wav` (ADR-0009, FR-REC-1/4).
6. On `stop_recording`: finalize the WAV; run `diarization`, align speaker IDs to segments,
apply name mappings; persist transcript JSON + metadata via `storage` (FR-STORE-1). When the mic
is on, `audio.wav` is a **dual-channel split** (left = mic, right = loopback; ADR-0005 Phase 3.5,
FR-SPK), so attribution is **per-stream**: sherpa clusters the **right channel only** →
`Speaker N`, and "You" comes from **left-channel** voice activity. Blind whole-signal clustering
+ voiceprint is the mic-off (`summed`) fallback. If "Record this meeting" is **off**, delete the
working WAV **after** the transcript is finalized; if **on**, keep it as `audio.wav` (ADR-0009,
FR-REC-1/4).
7. If an LLM provider is configured: `llm` builds a prompt (transcript + metadata + template),
streams a summary/decisions/action-items into the summary panel (FR-LLM-2/4).
8. Action items are parsed and presented for confirmation; confirmed ones persist and may raise
+24 -1
View File
@@ -14,7 +14,8 @@ Default root: `%LOCALAPPDATA%\WhispAssist\` (user-configurable, FR-STORE-2).
│ └── spk-eres2net.onnx
└── meetings\
└── <meeting_id>\ # one folder per meeting (uuid)
├── audio.wav # canonical recording — present ONLY if "Record" was on (ADR-0009)
├── audio.wav # canonical recording — present ONLY if "Record" was on (ADR-0009). Mic on → stereo split (L=mic/"You", R=loopback/"Speaker", `audio_layout=split`); mic off → summed. Playback folds to mono; export folds to dual-mono (ADR-0005 Phase 3.5)
├── voiceprint.wav # mic-only 16kHz sample for "You" re-matching on reprocess of a SUMMED recording (FR-SPK); lives/dies with audio.wav
├── transcript.json # canonical transcript (segments+speakers+timings)
├── manual_notes.json # raw user-authored notes captured live during recording
├── notes.md # the final notes document: manual notes + transcript, merged at finalize
@@ -86,6 +87,7 @@ CREATE TABLE meetings (
language TEXT, -- detected/selected language code
backend_used TEXT, -- npu|nvidia|amd|intel|cpu
model_used TEXT, -- e.g. whisper-base
audio_layout TEXT, -- split|summed|NULL; how audio.wav's channels are laid out (0009, ADR-0005 Phase 3.5)
calendar_event_id TEXT, -- FK -> calendar_events.id (nullable)
template_id TEXT, -- note-template id (T8.1, FR-NOTE-5); catalog is
-- a built-in Rust list (notes::templates), not a table
@@ -380,6 +382,13 @@ segment (the M1 grounding invariant, asserted by the golden-transcript test).
"expose_recordings": false, // never serve .wav unless explicitly true (FR-MCP-3)
},
"privacy": { "encrypt_at_rest": false },
// Launch WhispAssist at login (NFR-RES-4). Opt-in, OFF by default. Toggling this
// via `set_auto_start` also writes a per-user `HKCU\...\Run` entry (no admin);
// startup reconciles the OS entry to this flag (e.g. after a reinstall).
"auto_start": false,
// Closing the window hides WhispAssist to the system tray (keep running in background) instead of
// quitting; ON by default. Tray "Quit" is the real exit. Enforced in the Rust on_window_event handler.
"close_to_tray": true,
// Optional MS Graph calendar source (M4.4, T8.9, FR-CAL-6). Opt-in, explicit consent via OAuth
// PKCE — OFF by default. `credential_ref` points into the OS credential store; the token itself
// is never written here (same invariant as sync credentials, FR-SYNC-6).
@@ -387,6 +396,20 @@ segment (the M1 grounding invariant, asserted by the golden-transcript test).
}
```
## First-run deploy seeding (`wa-defaults.ini`)
For enterprise mass-deployment, the **first** launch on a machine (before `settings.json` exists)
optionally seeds its defaults from an admin-supplied INI. First file found wins:
1. `%PROGRAMDATA%\WhispAssist\wa-defaults.ini` — machine-wide (GPO / SCCM / Intune file copy).
2. `<install dir>\wa-defaults.ini` — the bundled template (shipped fully commented → no-op).
Keys are flat `key = value` matching `settings.json` field names (bools/ints coerced), plus the
special `auto_download_model = true` which fetches the configured `whisper_model` in the background.
**No secrets** — any key containing `key`/`token`/`secret`/`credential`/`password` is ignored; those
live only in the OS credential store. After first run the file is never read again. See
`docs/enterprise-deployment.md` and `src-tauri/src/deploy.rs`.
## Retention & recovery semantics
- **Retention** (FR-STORE-2): a background job deletes whole meeting folders + rows once a meeting
+25 -3
View File
@@ -23,6 +23,9 @@ start_recording(input: { meetingTitle?: string; calendarEventId?: string; record
stop_recording(input: { meetingId: MeetingId }): MeetingSummaryRef
pause_recording(input: { meetingId: MeetingId }): void
resume_recording(input: { meetingId: MeetingId }): void
// Mute/unmute the mic mid-meeting (FR-CAP-7): mic channel goes silent (recording + transcript + meter),
// loopback keeps capturing. Returns the new muted state; emits recording://mic. Errs if the mic is off.
toggle_microphone_mute(input: { meetingId: MeetingId }): boolean
set_recording_retention(input: { meetingId: MeetingId; record: boolean }): void // toggle mid-meeting (FR-REC-1)
acknowledge_recording_consent(): void // one-time (FR-REC-2)
@@ -36,8 +39,14 @@ update_live_notes(input: { meetingId: MeetingId; markdown: string }): void
set_segment_note(input: { meetingId: MeetingId; anchorMs: number; text: string }): void
// ---- Hardware ----
hardware_status(): { backends: BackendInfo[]; active: BackendId; modelSize: string; estRtf: number }
hardware_status(): { backends: BackendInfo[]; active: BackendId; modelSize: string; estRtf: number } // re-detects fresh each call — backs the Settings ▸ Hardware "Refresh" button
set_preferred_backend(input: { backend: BackendId | "auto" }): void
// Launch-at-login (NFR-RES-4). Writes/removes a per-user OS Run entry (no admin) and persists auto_start. Opt-in, off by default.
set_auto_start(input: { enabled: boolean }): void
// Device test: opens a mic ("input") or the render device in loopback ("loopback") for a few seconds and streams device://level (no recording, no retained audio). Refused while recording.
monitor_audio_level(input: { kind: "input" | "loopback"; deviceId?: string; durationMs?: number }): void
// Quick stress test: benchmarks each available backend × installed model (≤3 sizes) on a fixed sample, returns per-pair real-time factor + the most-accurate real-time-capable recommendation. Emits stress://progress. Refused while recording.
stress_test_hardware(): { results: { backend: string; model: string; rtf: number; realtime: boolean }[]; recommended: { backend: string; model: string } | null }
// ---- Transcription / models ----
// language (T8.7, M4.2): omitted reuses the meeting's current language rather than resetting it.
@@ -79,6 +88,8 @@ export_meeting(input: { meetingId: MeetingId; dest: string; format: "md" | "pdf"
// they enqueue for finalize-trigger targets and pump in the background. SHA-256 dedup means an
// edit that didn't alter a file uploads nothing.
update_notes(input: { meetingId: MeetingId; markdown: string }): void
// AI-enhance rough notes into structured Markdown grounded in the transcript (Granola-style), via the configured LlmProvider (no new egress). Takes the live buffer, returns the enhanced text WITHOUT persisting — the UI keeps or undoes it. Refused while recording; errors with no provider.
enhance_notes(input: { meetingId: MeetingId; notes: string }): string
// SearchHit = MeetingListItem fields (id, title, started_at, duration_secs, status, tags) + snippet: string
search(input: { query: string }): SearchHit[] // FTS (FR-SEARCH-1)
set_tags(input: { meetingId: MeetingId; tags: string[] }): void
@@ -93,6 +104,13 @@ bulk_export_meetings(input: { destDir: string; format: "md" | "pdf" | "docx" | "
// folder of them (from a bulk export). Each is reconstructed under a fresh meeting id (original
// title/date/duration/speakers/tags/action items preserved). Returns the count imported.
import_meeting_bundle(input: { dir: string }): number
// Add a meeting from an existing recording: a local audio/video file path or a URL (YouTube/
// streaming page or direct media URL). Needs ffmpeg (+ yt-dlp for URLs) on PATH; neither bundled.
// `model` overrides the Settings whisper model for this import (recorded as meeting.model_used).
// Returns the new meeting id IMMEDIATELY (status "transcribing"); transcode→transcribe→diarize→
// finalize run in the background, streaming import://progress and ending with transcript://finalized.
// A failed import is left in the list with status "error" (not deleted).
import_media(input: { source: string; title?: string; model?: string }): MeetingId
// ---- LLM / AI provider (ADR-0007/0011) ----
// provider ∈ ollama | custom | anthropic | openai | off. Hosted-provider API keys are passed to
@@ -183,9 +201,11 @@ privacy_self_check(): {
"recording://state" { meetingId, state: "recording"|"paused"|"stopped"|"cancelled", elapsedMs }
"recording://level" { meetingId, rms: number, peak: number } // waveform (FR-CAP-5)
"recording://device" { meetingId, recovered: boolean, message: string } // capture device change (FR-CAP-6)
"transcript://segment" { meetingId, segment: TranscriptSegment } // live segments (FR-TRX-2)
"recording://mic" { meetingId, muted: boolean } // mic mute toggled (FR-CAP-7)
"transcript://segment" { meetingId, segment: TranscriptSegment } // live segments (FR-TRX-2); may re-emit a committed segment with a refined `speaker` — replace by `segment.id`
"transcript://finalized" { meetingId, segmentCount }
"diarization://updated" { meetingId, speakers: SpeakerInfo[] } // after post-pass (FR-SPK)
"import://progress" { meetingId, phase: "prepare"|"transcribe"|"diarize"|"finalize", state: "active"|"done"|"error", elapsedMs: number|null, error: string|null } // background import_media tracker
"diarization://updated" { meetingId, speakers: SpeakerInfo[] } // post-pass AND live 15s provisional passes (FR-SPK); carries "You" once the mic voiceprint matches
"llm://token" { meetingId, text } // streamed summary (FR-LLM-4)
"llm://done" { meetingId, summary: SummaryFile } // full summary.json contents, not just a pointer
"model://progress" { id, receivedBytes, totalBytes }
@@ -193,6 +213,8 @@ privacy_self_check(): {
"calendar://linked" { ok: boolean, error?: string } // MS Graph OAuth handshake settled (M4.4)
"calendar://progress" { processed, total } // MS Graph import (M4.4)
"hardware://changed" { active: BackendId, reason: string } // fallback occurred (FR-HW-4)
"device://level" { kind: "input"|"loopback", rms?, peak?, done?: boolean } // Settings device test meter; done=window ended
"stress://progress" { backend: string, model: string } // quick stress test, per pairing benchmarked
"recording://retention" { meetingId, record: boolean } // retention toggled (FR-REC-1/3)
"sync://job" { jobId, meetingId, targetId, artifact, status, bytesSent, bytesTotal } // FR-SYNC-5
"sync://done" { meetingId, targetId, uploaded: number, failed: number }
+48
View File
@@ -32,6 +32,54 @@ labels in the post-meeting pass; this keeps latency low while improving final ac
- Speaker IDs (`S1`, `S2`, …) are internal and stable per meeting; name mappings live in the DB
and are applied at render/export time, never destructively rewritten onto segments.
### Phase 3 refinement — per-stream "You" attribution (FR-SPK, 2026-07-14)
The single-pass-over-the-whole-recording model above blind-clusters a **summed mono** signal
(mic + loopback), then guesses which cluster is the user via a voiceprint match. In practice that
clustering is unreliable in both directions on the summed signal — it over-split a 2-speaker call
into 83 clusters at one threshold and merged two clearly distinct voices into one at another — and
the voiceprint can only *label* a cluster, never *create* the separation.
So when the **microphone is enabled**, WA no longer relies on clustering to find the user:
- A live **mic-activity timeline** (`audio::MicActivity`) records, per 100 ms of `audio.wav` frame
time, when the mic was speech-level — captured in the loopback writer, the one place the mic and
loopback exist separately in the recording's own timebase. Those ranges become **"You"** spans
directly (no clustering, no embedding).
- sherpa clustering is then run over the recording with the "You" ranges **masked out** (zeroed),
so it only ever splits the **far side** into `Speaker N`. The mic can never pollute or merge into
a far-side cluster.
- The timeline is persisted as `mic_activity.json` (retained meetings) so `reprocess_transcript`
re-attributes identically without the live capture.
The original blind-clustering + voiceprint pass **remains the fallback** for mic-off recordings and
imports (no timeline). Naming stays uniform (`build_name_map`: "You", then `Speaker 2…`). Segment
IDs and the names-in-DB rule are unchanged; only the *source* of the spans changes.
### Phase 3.5 refinement — dual-channel capture (supersedes Phase 3's timeline, 2026-07-14)
The Phase 3 timeline above reconstructed the mic/far-side split *after the fact* from a summed-mono
`audio.wav` plus a `mic_activity.json` sidecar, masking the mic ranges before clustering. That was
reliable at stop but **fragile on reprocess** (re-aligning a sidecar against a mono mix — it
collapsed to a single speaker) and carried a parallel persistence path.
**Decision:** when the mic is enabled, record `audio.wav` as **stereo with the streams separated —
left = microphone ("You"), right = system/loopback ("Speaker")** instead of summing them. The
separation is then intrinsic to the recording:
- Diarization runs on the **right channel only** → `Speaker N`; "You" comes from **left-channel**
voice activity (`vad_spans`). The mic is never clustered, by construction.
- Reprocess recomputes both from the file — no sidecar, no masking; stop and reprocess agree.
- Transcription still downmixes (L+R) to the same summed mono; playback folds to mono and bundle
export folds to dual-mono so shared/played audio is normal.
- A `meetings.audio_layout` flag (`summed` | `split`, carried in the bundle manifest) distinguishes
recordings; existing `summed` recordings keep the blind-clustering + voiceprint path. On a stereo
render endpoint this is **size-neutral** (the file was already 2ch with the mic summed into both).
This **retires** `MicActivity` / `mic_activity.json` / masked diarization from Phase 3 (see the
retirement note at `attribute_split` in `commands.rs`); the capture-silence handling,
`diarize_samples`, and `build_name_map` naming are retained. Segment IDs and names-in-DB unchanged.
## Revisit if
A single model gives joint ASR + diarization with better accuracy, or whisper.cpp gains
production diarization.
@@ -0,0 +1,54 @@
# ADR-0012 — Launch-at-login & enterprise deployment defaults
- **Status:** Accepted
- **Date:** 2026-07-14
- **Context source:** User request (2026-07-14) — auto-start on boot; customize an installation
(install location, per-user/all-users, default settings) with native Windows tooling for
mass-deployment.
## Context
Two related needs. (1) Users want WhispAssist to **launch automatically at login**. NFR-RES-4
forbids adding WA to OS startup without explicit consent, so this must be opt-in. (2) An admin
mass-deploying WA to many machines wants to **customize the deployment** — install location, whether
it installs per-user (no admin) or all-users, and the app's default settings (record-by-default,
preferred backend, a model to pre-download, retention, AI provider) — using **native Windows tools**
(GPO / SCCM / Intune / `msiexec` / silent NSIS), not a bespoke management console.
## Decision
1. **Launch-at-login is opt-in, off by default.** A `set_auto_start` command uses
`tauri-plugin-autostart` to write a **per-user** `HKCU\...\Run` entry (no admin) and persists an
`auto_start` setting. Startup reconciles the OS entry to that flag (restores it after a reinstall).
Nothing runs on a timer — this is a registry entry, not a background process (NFR-RES-1).
2. **Install location & scope are native, no app code.**
- Location: `msiexec INSTALLDIR=…` (MSI) / NSIS `/D=…` (silent).
- Scope: NSIS `installMode: "both"` — the `.exe` setup lets the user choose **current-user
(no admin)** or **all-users (admin)**. The MSI stays per-machine as the enterprise all-users
artifact.
3. **Default settings via a first-run `wa-defaults.ini`.** On a machine's **first** launch (before
`settings.json` exists) WA reads an admin-supplied INI — `%PROGRAMDATA%\WhispAssist\wa-defaults.ini`
first, else the bundled `<install dir>\wa-defaults.ini` — and seeds `settings.json` from it, with an
optional `auto_download_model` to pre-fetch the model. The shipped template is fully commented, so
a normal install is unaffected. This is deployable purely by copying a file with existing Windows
management tooling; no WiX custom actions.
## Consequences
- **Positive:** opt-in startup honors NFR-RES-4 with zero idle cost; install location/scope reuse the
installers' native behavior (no custom code to maintain); one small INI + a first-run guard covers
the whole deployment-customization surface and works for MSI, NSIS, and portable copies alike.
- **Guardrail — no secrets in the deploy file (CLAUDE.md):** the INI must never carry credentials.
`deploy.rs` drops any key containing `key`/`token`/`secret`/`credential`/`password` as defense in
depth; API keys, OAuth tokens and sync passwords remain in the OS credential store only. Seeding a
provider (e.g. `llm_provider=anthropic`) still requires the admin/user to provision its key
separately — no new egress path is created by the file.
- **Negative / care:** the seed runs only when `settings.json` is absent (truly first run); it does
**not** re-apply on upgrade, matching "the user's own settings win thereafter." Array config merges
in Tauri **replace** rather than append, so `wa-defaults.ini` must be listed in both
`tauri.conf.json` and `tauri.vulkan.conf.json` bundle resources (the shipped build uses the latter).
## Revisit if
Admins need per-machine policy that **overrides** user settings on every launch (not just seeds
first-run), or a signed/locked-down enterprise policy channel beyond a plain INI.
+101
View File
@@ -0,0 +1,101 @@
# Enterprise deployment
How to mass-deploy WhispAssist and preset its defaults with native Windows tooling (Group Policy,
SCCM, Intune, `msiexec`, silent NSIS). No management console, no phone-home. See ADR-0012.
WhispAssist ships two bundles:
| Bundle | Scope | Admin? |
|---|---|---|
| `WhispAssist_<ver>_x64_en-US.msi` | Per-machine (all users) | Yes |
| `WhispAssist_<ver>_x64-setup.exe` (NSIS) | Current-user **or** all-users (prompts) | Only for all-users |
## Install location
- **MSI:** `msiexec /i WhispAssist_<ver>_x64_en-US.msi INSTALLDIR="D:\Apps\WhispAssist" /qn`
(`INSTALLDIR` is Tauri's WiX install-dir property; confirm against the generated `.wxs` if a build
changes it.)
- **NSIS:** `WhispAssist_<ver>_x64-setup.exe /S /D=D:\Apps\WhispAssist`
(`/S` = silent, `/D=` = install dir; `/D=` must be **last** and unquoted per NSIS.)
## Install scope (per-user vs all-users)
The NSIS `.exe` shows a "current user / all users" page. **Current user needs no admin** and installs
under the user profile; **all users** requires elevation. Silent all-users:
`WhispAssist_<ver>_x64-setup.exe /S`. The MSI is always per-machine (all-users) and requires admin.
## Auto-start at login
Off by default (NFR-RES-4). Turn it on for the user either in-app (Settings ▸ Recording ▸ *Launch
WhispAssist at login*) or by presetting `auto_start = true` in `wa-defaults.ini` (below). It installs
a **per-user** `HKCU\Software\Microsoft\Windows\CurrentVersion\Run` entry — no admin, and it does
**not** start recording on its own.
## Preset default settings — `wa-defaults.ini`
On a machine's **first** launch (before `settings.json` exists), WhispAssist reads an admin-supplied
INI and seeds that user's `settings.json`. After that the user's own settings win and the file is
ignored. First location found wins:
1. `%PROGRAMDATA%\WhispAssist\wa-defaults.ini` — machine-wide. Deploy with a GPO/SCCM/Intune file copy.
2. `<install dir>\wa-defaults.ini` — the template shipped next to the executable.
The shipped template is fully commented out, so a default install behaves as if it were absent.
Uncomment and edit the keys you want to preset.
### Format
Flat `key = value`, one per line. `;` and `#` comment lines and `[section]` headers are ignored.
`true`/`false` become switches, plain numbers become numbers, everything else is text. Unknown or
misspelled keys are ignored.
> **Never put secrets in this file.** API keys, OAuth tokens and sync passwords live only in the OS
> credential store. Any key containing `key`, `token`, `secret`, `credential` or `password` is
> dropped on read. Presetting `llm_provider = anthropic` still requires the key to be provisioned
> separately — the file adds no egress path.
### Keys
| Key | Values | Meaning |
|---|---|---|
| `default_record` | true/false | Record every meeting by default (consent notice still applies). |
| `preferred_backend` | auto\|npu\|nvidia\|amd\|intel\|cpu | Transcription backend. |
| `whisper_model` | catalog id (e.g. `base.en-q5_1`) | Default transcription model. |
| `auto_download_model` | true/false | Fetch `whisper_model` in the background on first launch. |
| `whisper_language` | auto\|ISO-639-1 | Default language (multilingual model only). |
| `low_overhead` | true/false | CPU + smallest model preset. |
| `storage_root` | path | Where meetings are stored. |
| `retention_max_age_days` | number | Delete meetings older than N days. |
| `retention_max_size_gb` | number | Cap total storage at N GB. |
| `llm_provider` | ollama\|custom\|anthropic\|off | Summary provider (key provisioned separately). |
| `llm_endpoint` | url | LLM endpoint. |
| `llm_model` | text | LLM model name. |
| `microphone_enabled` | true/false | Capture the user's mic into the transcript. |
| `auto_record_calendar` | true/false | Auto-start recording on calendar events (app open only). |
| `theme` | system\|light\|dark | UI theme. |
| `auto_start` | true/false | Launch WhispAssist at login (per-user Run entry). |
| `sync_enabled` | true/false | Sync master switch (targets/credentials configured in-app). |
### Example
```ini
default_record = true
preferred_backend = npu
whisper_model = small.en-q5_1
auto_download_model = true
retention_max_age_days = 90
auto_start = true
```
## Silent end-to-end example
```bat
:: 1. Push machine-wide defaults (as SYSTEM via GPO/SCCM)
mkdir "%ProgramData%\WhispAssist"
copy wa-defaults.ini "%ProgramData%\WhispAssist\wa-defaults.ini"
:: 2. Install per-machine, custom location, no UI
msiexec /i WhispAssist_<ver>_x64_en-US.msi INSTALLDIR="C:\Program Files\WhispAssist" /qn
```
Each user's first launch then seeds their `settings.json` from the machine-wide file.
@@ -0,0 +1,243 @@
# Phase 3 design sketch: per-stream speaker attribution (FR-SPK)
Status: **signed off & in progress**, 2026-07-13. Deepens Phase 3 of
[`2026-07-13-diarization-speaker-accuracy.md`](./2026-07-13-diarization-speaker-accuracy.md).
The §§1–8 below are the original sketch (ADR-0005 review + alternatives); the **finalized
design** the user signed off on is at the top and supersedes the RMS-dominance heuristic.
---
## Finalized design (signed off 2026-07-13)
Simpler than the sketch's dominance heuristic, because the user records on **headphones** so the
mic never hears the far side (no bleed):
> **The microphone stream is "You" by definition. Diarization runs on the loopback (speaker)
> audio only, never the mic. Any loopback speaker is "Speaker N".**
Mechanism, all in **`audio.wav` frame time** (so You-spans and sherpa-spans share one clock):
1. **Mic-activity timeline** — captured in the loopback writer, the one place mic and loopback
exist separately in `audio.wav`'s timebase (`capture_loop`, where `bridge.pull()` is folded
in). Per 100 ms, flag whether the pulled mic chunk was speech-level (reuse `VOICE_ENERGY_FLOOR`).
Held in a shared `MicActivity` (sibling to `VoiceSample`); persisted for reprocess.
2. **At stop / reprocess:**
- Collapse active windows → **"You" spans** (dropping sub-`MIN_SPAN_MS` blips).
- **Mask** the You ranges out of `audio.wav` (zero them) and run sherpa on the remainder → far
side splits into **"Speaker N"** spans. Clustering never sees the mic, so the mic can't
pollute or merge with a far-side cluster.
- Merge span lists; `assign_by_overlap` unchanged.
3. **Voiceprint (Phase 1/2) becomes the fallback** for meetings with no timeline (imports, older
recordings). Live-tick voiceprint "You" is retired once the timeline drives labels.
Timebase note: with the capture-silence fix (FR-CAP-7) `audio.wav` is now continuous, so the
frame-indexed timeline aligns cleanly with what sherpa reads back via `read_wav_mono_16k`.
Implementation status (all landed 2026-07-13/14): **(a)** `MicActivity` struct + capture wiring —
done; **(b)** stop-path attribution + masked diarize (`phase3_attribute`) — done; **(c)** persist
`mic_activity.json` + reprocess uses it (imports fall back, no timeline) — done; **(d)** ADR-0005 +
`docs/02`/`03` amended — done. **Not yet done:** empirical user test; and the *live* 15s tick still
uses the Phase 2 voiceprint, not the timeline (a later polish so live labels match the stop result).
---
_Original sketch (ADR-0005 review, risks, alternatives) follows._
Phases 0–2 shipped: cluster explosion tamed (0.7 + 700 ms floor), reprocess re-diarizes with a
persistent `voiceprint.wav`, and live "You" reaches the UI. Those all still treat diarization as
**blind clustering of a summed mono signal**, then guess which cluster is the mic. Phase 3
removes the guess for the mic side by using information WA throws away today.
---
## 1. ADR-0005 review — what it commits us to
ADR-0005 (Accepted, 2026-06-30) decides: sherpa-onnx offline diarization (pyannote segmentation
+ ERes2Net embedding + clustering), behind the `diarization::Diarizer` trait, run as a
**post-processing pass over recorded audio**, aligned to whisper segments by **timestamp
overlap**. Speaker IDs (`S1`…) are internal/stable; names map in the DB, applied at render time,
**never rewritten onto segments**.
What Phase 3 must respect vs. what it may change:
- **Keeps (contract-level):** the `Diarizer` trait, post-pass-over-audio model, `SpeakerSpan`
→ segment alignment by overlap (`assign_by_overlap`), stable internal labels, names-in-DB.
Phase 3 produces *more* `SpeakerSpan`s from a better source; it does not rewrite segments or
move naming.
- **Bends (needs the ADR noted/updated):** ADR-0005 assumes one clustering pass over "the
recorded audio." Phase 3 introduces a **second, non-clustered source of spans** (the mic
activity timeline) and restricts clustering to the **far-side** audio. That is new enough to
warrant an ADR amendment or a short ADR-0005a, because a future reader will otherwise expect
all spans to come from sherpa.
- **ADR's "Revisit if"** (joint ASR+diarization model, or whisper.cpp diarization) is unrelated
— Phase 3 is orthogonal and does not trigger it.
Conclusion: Phase 3 fits inside the trait and the overlap-alignment contract. The only doc debt
is recording that "mic-dominant spans bypass clustering," which is a genuine deviation from
ADR-0005's single-source assumption. **Recommend: amend ADR-0005** (Consequences section) rather
than a new ADR — same decision, refined.
---
## 2. The asset we currently discard
When the mic is enabled (FR-CAP-7), `start_recording` (`commands.rs:342`) wires two independent
16 kHz-mono streams into `spawn_mixer` (`audio/mod.rs:1020`). The `Mixer` (`audio/mod.rs:979`)
holds `loopback` and `mic` **time-aligned in the same buffer** and sums them sample-for-sample
into the transcript stream — then the per-stream identity is **gone**. Diarization later reads
`audio.wav` (the summed signal) and has to reverse-engineer which cluster was the mic. That
reversal is the entire reason `voiceprint.rs` exists.
Two facts make this cheap to exploit:
1. **Per-stream RMS already exists.** `audio_level(mono, mic)` (`audio/mod.rs:522`) runs on every
WASAPI chunk for *both* directions (`is_loopback` at :564, emitted at :678). We already know,
per ~10 ms chunk, how loud each side is. We just don't persist it against time.
2. **The two streams are already aligned at the mixer**, in the **same timebase the transcript
segments derive from** (the streaming worker consumes the mixer's summed output; segment
timestamps are cumulative-samples-fed / 16). So a timeline indexed by cumulative mixer-output
samples shares the segments' clock — tighter than sherpa spans, which come from `audio.wav`.
---
## 3. Core idea
> The mic stream is a **known** speaker. Don't cluster to find it — record when it's dominant,
> call those spans "You", and run sherpa only on the **far-side** audio to split the *other*
> participants.
Pipeline at stop (mic enabled + retained):
```
mixer ─┬─► loopback samples ──► [far-side WAV] ──► sherpa cluster ──► Speaker 2..N spans
│ │
└─► mic vs loopback RMS per 100 ms ──► mic-dominant ranges ──► "You" spans
│
merge span lists ──► assign_by_overlap ──► segments
```
The mic-dominant spans need no embedding, no clustering, no voiceprint — they are attribution by
construction. Voiceprint (Phases 1–2) stays as the **fallback** for meetings without a timeline
(imports, pre-Phase-3 recordings, mic-disabled meetings).
---
## 4. The two real risks (why this needs sign-off, not just a ticket)
### 4a. Acoustic bleed — the mic is not purely "You"
Without echo cancellation (WA has none — fully local, minimal), a user on **speakers** (not
headphones) has the far side playing into the room and back into the mic. Naive "mic has signal →
You" would attribute the far side to the user whenever the far side is loud.
Mitigation (this is why the parent plan says *mic-**dominant**,* not *mic-active*): a range is
"You" only when **mic RMS meaningfully exceeds loopback RMS** in that window (e.g.
`mic_rms > k · loopback_rms` with `k ≈ 2`, plus an absolute mic-VAD floor so silence isn't
"You"). When both are comparably loud → treat as far-side/ambiguous, let sherpa/overlap decide.
This degrades gracefully: headphone users get near-perfect mic isolation; speaker users get
"You" only on clear self-speech and fall back to the old behavior during overlap. **`k` is a
calibration knob, not a constant to bury** — real rooms differ.
### 4b. Timebase — three clocks, currently reconciled by tolerance
- **Segments** are in *mixer-output* time (streaming worker over the summed stream).
- **sherpa spans** are in *`audio.wav`* time (loopback thread's byte-accurate WAV, mic bridged in
at native rate then resampled to 16 kHz on read).
- The **new mic-timeline** would be in *mixer-output* time.
Today `assign_by_overlap` already bridges segments↔sherpa-spans across the first two clocks and
tolerates the small skew. Phase 3 merges a *third* source. The mic-timeline is actually the
**closest** to segment time (shared origin), so aligning You-spans to segments is *more* reliable
than sherpa's. The residual risk is only that You-spans (mixer time) and Speaker-2..N spans
(`audio.wav` time) coexist in one merged list with a small relative skew at the boundaries. In
practice that costs at most a fraction of a `MIN_SPAN_MS` at each turn boundary — below segment
granularity. **Verification item, not a blocker.** If it proves visible, the clean fix is to run
the far-side sherpa pass over the mixer's *loopback lane* buffered to a WAV rather than over
`audio.wav`, putting everything on one clock; heavier, so deferred until measured.
---
## 5. Concrete design
**Capture (live, near-zero cost).** Add a mic-activity accumulator the mixer writes to. In
`Mixer::drain_ready` (or the `mixer_loop`), for each emitted window compute `mic_rms` and
`loopback_rms` over the aligned overlap (samples are already in hand there) and append a coarse
flag per 100 ms:
```rust
// audio/mod.rs — new, alongside Mixer
struct MicActivity { flags: Vec<bool>, /* one per 100 ms of mixer output */ }
// ponytail: Vec<bool> at 10 Hz = 10 bytes/s ≈ 36 KB/hour. No ring buffer, no config.
```
Expose it via the same `Arc` sharing pattern `VoiceSample` uses so `stop_recording` can read it.
Reuse `audio_level`'s RMS math — do **not** add a second RMS implementation.
**Persist (retained meetings).** At stop, alongside `voiceprint.wav`, write the timeline as
`mic_activity.json` (or pack into an existing artifact) so **reprocess** can reuse it — same
ADR-0009 retention/consent gate as `voiceprint.wav`, same lives-and-dies-with-`audio.wav` rule
(delete it everywhere `audio.wav` is deleted). Tiny, plaintext-or-sealed to match.
**Attribute (stop + reprocess).**
1. Collapse the flag timeline into `You` `SpeakerSpan`s (merge adjacent flags, drop runs
`< MIN_SPAN_MS`, reuse Phase 0's floor).
2. Build a **far-side-only** 16 kHz signal: zero out (or excise) mic-dominant ranges from the
loopback lane, run the existing `SherpaDiarizer::diarize` over it → `Speaker 2..N` spans
(label numbering starts at 2, mirroring `voiceprint::build_name_map`).
3. Merge the two span lists; `assign_by_overlap` unchanged.
4. Fallback chain: no timeline → today's Phase-1/2 voiceprint path; no models → raw `S1`.
**Contract/doc changes.**
- Amend **ADR-0005** Consequences: "mic-dominant spans are attributed directly from a per-stream
activity timeline and bypass clustering; sherpa runs on the far-side residual only."
- `docs/02-architecture.md` data-flow: note the mixer emits a mic-activity side-channel.
- `docs/03-data-model.md`: add `mic_activity.json` to the meeting-dir list (like `voiceprint.wav`).
- No new egress; no new default-on setting; `k` threshold lives in code with a calibration comment.
**Tests.**
- Unit: flag-run → span collapse (boundaries, sub-`MIN_SPAN_MS` drop) — pure, table-driven.
- Unit: dominance classifier (`mic`, `loop` RMS pairs → You / far / ambiguous), incl. the bleed
case (both loud → not You).
- Manual: 2-person call on **headphones** → exactly "You" + 1 speaker, near-zero errors; repeat
on **speakers** → verify bleed doesn't mislabel far-side as You.
---
## 6. Alternatives considered (and why not)
- **Do nothing / stop at Phase 2.** Legitimate. Phases 0–2 already fix the reported bugs
(83-cluster explosion, reprocess collapse, live "You"). Phase 3 is an **accuracy ceiling
raise**, not a bug fix. *If 2-person calls are the dominant case and Phase 2's voiceprint "You"
tests well in the field, Phase 3 may not clear the cost/benefit bar — measure Phase 2 first.*
This is the ponytail-honest recommendation: **verify Phase 2 empirically before building 3.**
- **"Mic active → You" (no dominance test).** Simpler, but wrong under speaker bleed (4a). The
dominance test is the minimum that survives real rooms.
- **Full source separation / AEC.** Much larger, new deps, defeats "minimal + local." The
dominance heuristic gets ~90 % of the benefit for ~5 % of the effort.
- **Better voiceprint only** (bigger sample, re-match every tick — already done in Phase 2).
Still bottlenecked on sherpa's ability to *form* a clean mic cluster in the summed signal;
Phase 3's point is to stop relying on that.
---
## 7. Open questions for sign-off
1. **Build it now, or verify Phase 2 first?** Recommendation: run the empirical 2-person
re-diarization (outstanding for Phases 0–2) and a Phase-2 live "You" check **before**
committing to Phase 3. If Phase 2 accuracy is acceptable, Phase 3 becomes optional.
2. **Dominance threshold `k` and mic-VAD floor** — accept a code-constant default with a
calibration comment (proposed), or expose a hidden setting? Recommendation: constant first.
3. **Far-side signal construction** — zero-out mic-dominant ranges in the loopback lane (simple,
preserves timeline) vs. excise-and-concat (shorter audio, shifts timestamps → needs remap).
Recommendation: **zero-out**, keeps one timebase.
4. **ADR: amend 0005 vs. new ADR-0005a?** Recommendation: amend.
5. **Scope of `mic_activity.json`** — persist for reprocess (proposed) or compute live-only and
accept that reprocess falls back to voiceprint? Persisting is cheap and keeps reprocess at
parity.
## 8. Rough effort
Backend-only until the far-side pass; no frontend change (labels already flow via Phase 2). Est.
~1–1.5 days: mixer side-channel + accumulator (½ day), span collapse + far-side pass + merge
(½ day), persistence + reprocess wiring + docs/tests (½ day). Contained, reversible, behind the
existing mic-enabled + retention gates.
@@ -0,0 +1,137 @@
# Fix plan: speaker diarization accuracy ("You", live labels, 83-speaker explosion)
Status: planned 2026-07-13 (v0.5.2). Investigation confirmed in code; see memanto memories
`a9e5f29e` (diagnosis) and the entry referencing this file (fix order). Execute phases **in
order** — each phase is independently shippable and verifiable, and later phases assume
earlier ones landed.
## Symptoms (user report, all reproduced against code)
1. Live recording shows `S1` for every line; the user's own speech should show "You".
2. Post-stop diarization of a 2-person conversation produced **83** speakers.
3. Re-transcribing (English Medium) collapsed every segment to speaker "You".
## Root causes (verified anchors)
| # | Cause | Anchor |
|---|-------|--------|
| A | Live segments always carry the pre-diarization `"S1"` placeholder | `src-tauri/src/transcription/mod.rs:222`, emitted at `src-tauri/src/commands.rs:480` |
| B | The 15s provisional diarization tick relabels only the backend buffer and emits `diarization://updated`, which **no frontend code listens to** (`src/lib/api.ts` has no listener); live view renders raw labels with no name map (`src/lib/views/TranscriptNotes.svelte:527`) | `src-tauri/src/commands.rs:509-567` |
| C | "You" (mic voiceprint match) runs only in `stop_recording`, never live | `src-tauri/src/commands.rs:666-695`, `src-tauri/src/diarization/voiceprint.rs` |
| D | `reprocess_transcript` never diarizes: fresh segments all default to `"S1"`, then the meeting's **stale** name map (`S1 → "You"` from the original run) is reused, so the whole transcript renders "You" | `src-tauri/src/commands.rs:1646-1742` |
| E | Clustering over the **summed mic+loopback mono** WAV with `num_clusters: -1, threshold: 0.5` over-clusters badly (overlapped speech → mixed embeddings; short pyannote chunks → unstable ERes2Net embeddings) | `src-tauri/src/diarization/mod.rs:68-75` |
---
## Phase 0 — Tame the cluster explosion (do first; everything else is useless at 83 clusters)
Smallest possible change, lands before Phase 1 so re-diarization doesn't reproduce the
83-speaker mess.
1. In `SherpaDiarizer::new` (`src-tauri/src/diarization/mod.rs:73`) raise `threshold` from
`0.5` to `0.7` as a code constant with a comment naming the tuning evidence (83 clusters
for 2 speakers at 0.5 on mixed mono audio). sherpa fast-clustering semantics: larger
threshold → fewer clusters. Do **not** add a settings knob yet.
2. In `segment_to_span`'s caller (`SherpaDiarizer::diarize`), drop spans shorter than
700 ms before returning — sub-second chunks carry unstable embeddings and only cause
label churn in `assign_by_overlap` (which already keeps a segment's prior label when no
span overlaps).
3. Unit test: spans under the minimum are filtered; existing overlap tests still pass.
4. **Verify empirically**: re-run diarization over the user's retained 2-person recording
(the 0.5.2 test meeting) and confirm the cluster count lands near 2–4, not 83. If 0.7
still over-clusters, try 0.8 before considering per-meeting configurability.
Acceptance: 2-person retained recording diarizes to ≤4 speakers.
## Phase 1 — `reprocess_transcript` re-diarizes + persistent mic voiceprint (fixes "everything is You")
1. **Persist the voiceprint at stop.** In `stop_recording`
(`src-tauri/src/commands.rs` after the transcription worker join), when
`session.mic_voice_sample` exists **and** `session.retention` is on, write the mic
sample as 16 kHz mono `voiceprint.wav` next to `audio.wav` in `meeting_dir(&meeting_id)`.
ADR-0009 gate: it is retained audio of the user's voice, so it lives and dies with
`audio.wav` — every code path that deletes `audio.wav` (retention off at finalize,
delete-recording command) must delete `voiceprint.wav` too. Update
`docs/03-data-model.md` (meeting dir file list).
2. **Re-diarize on reprocess.** In `reprocess_transcript` (`src-tauri/src/commands.rs:1646`),
after `transcribe_file`: build the diarizer via `diarizer_from_installed_models`
(inside `spawn_blocking`, same as `stop_recording`), `diarize(&wav_path)`, then
`assign(&mut segments, &spans)`. Missing models → skip gracefully (same degradation as
live).
3. **Rebuild the name map instead of reusing the stale one.** Old labels are meaningless
after re-clustering. If `voiceprint.wav` exists, run
`voiceprint::match_mic_speaker` against the new spans to get a fresh
`You`/`Speaker N` map; otherwise use an empty map (raw `S1…` labels).
Pass `speaker_infos_from_segments(&segments, &new_names)` to `finalize_meeting`
instead of `meeting.speakers` (`src-tauri/src/commands.rs:1713`), and persist the new
names via the store the same way `stop_recording` does. Deliberate policy: user-typed
names from the original run are dropped on reprocess because they key to dead labels —
note this in the command's doc comment.
4. `import_media` reuses this path (`commands.rs:1825` "same batch path"); confirm imported
meetings get diarized speakers too (they have no voiceprint — expect raw labels).
5. Tests: unit test the fresh-name-map policy (stale map not reused); manual: re-transcribe
the 0.5.2 meeting with English Medium → segments show distinct speakers, the user's own
lines show "You".
Acceptance: re-transcription yields per-speaker labels again, with "You" on the mic
speaker when a voiceprint exists; never a single-speaker collapse.
## Phase 2 — Live labels reach the UI, including live "You" (fixes "S1 for everyone")
Backend (`src-tauri/src/commands.rs`, the 15s tick at 509–567):
1. After `diarizer.assign` in the tick, **re-emit every committed segment whose speaker
changed** via the existing `transcript://segment` event (ids are stable; the frontend
store already replaces by id — `src/lib/stores/recording.svelte.ts:49-53`). No new
event needed for relabeling.
2. In the same tick, run `voiceprint::match_mic_speaker` with
`session.mic_voice_sample.samples()` against the fresh spans (clusters re-shuffle every
tick, so match every tick; candidate audio is already capped at 10 s per cluster).
Merge results into `session.speaker_names` **without overwriting user-set names** —
same already-named guard as the post-stop pass (`commands.rs:676-683`). The tick's
existing `diarization://updated` emit then carries the "You" display name.
3. Frontend: add an `onDiarizationUpdated` listener to `src/lib/api.ts` (payload
`{ meetingId, speakers: SpeakerInfo[] }`, already documented in
`docs/04-api-contracts.md:188`). Recording store gains a `speakers` state updated by
it; live rendering at `TranscriptNotes.svelte:527` passes that list to `speakerName`
(the finalized path at :344 already does this).
4. Docs: note in `docs/04-api-contracts.md` that `transcript://segment` may re-emit a
committed segment with an updated `speaker` (replace-by-id contract).
5. Tests: frontend store test — a re-emitted segment with the same id replaces the old
one; manual: during a live 2-person call, own speech flips to "You" within ~15–30 s.
Acceptance: during recording, labels differentiate speakers and the mic speaker shows
"You" while talking (within one tick), not just after stop.
## Phase 3 — Per-stream attribution (accuracy end-game, larger change, needs design sign-off)
Blind clustering of a summed mono signal is the ceiling on accuracy. WA knows which
samples are mic before `spawn_mixer` sums them (`commands.rs:342-371`,
`src-tauri/src/audio/mod.rs` MicBridge/mixer). Design sketch — do NOT start without
reviewing ADR-0005 and getting sign-off, since it changes the diarization contract:
- Record a coarse mic-activity timeline during capture (e.g. per-100 ms mic-RMS-dominant
flags, negligible memory).
- At stop: mic-dominant ranges become "You" spans directly; run sherpa only over the
remaining (far-side) ranges to split the *other* participants; merge span lists before
`assign_by_overlap`.
- Voiceprint match remains as fallback for meetings without the timeline (imports, old
recordings).
- Update ADR-0005 (or add a new ADR) + `docs/02-architecture.md` data flow.
Acceptance: 2-person call yields exactly "You" + 1 speaker with near-zero attribution
errors on non-overlapping speech; overlapping speech attributes to the dominant stream.
---
## Cross-cutting rules for the executing agent
- CLAUDE.md applies in full: `cargo fmt` + `cargo clippy -- -D warnings`, no
`unwrap()`/`expect()` on user-reachable paths, conventional commits referencing
FR-SPK-*, commit each file right after finishing it (one commit per file), docs updated
in the same change when contracts move.
- Memory discipline: `memanto agent activate whispassist` first; `memanto remember` every
decision/tuning result (especially the empirical threshold from Phase 0) with full
metadata; record phase completion so the next agent knows where to resume.
- No new egress, no new settings defaults ON; `voiceprint.wav` is local retained audio
under the existing ADR-0009 consent/retention gate.
@@ -0,0 +1,150 @@
# Plan: dual-channel capture (mic = left, loopback = right) — FR-SPK / FR-CAP
Status: **implemented** 2026-07-14 (all 3 sign-off answers: DB column; retire MicActivity with a
code note; export → dual-mono). Migration 0009 + `write_wav_split` + `attribute_split` +
`fold_wav` + ADR-0005 Phase 3.5. Supersedes the summed-mono + masked-diarize approach
in [`2026-07-13-diarization-phase3-per-stream-attribution.md`](./2026-07-13-diarization-phase3-per-stream-attribution.md).
The proposed **ADR-0005 amendment** is at the bottom — apply it on approval.
## Motivation
Every diarization problem this month traces to one root: **`audio.wav` is a summed mono mix** of
mic + loopback, so the two sources are irrecoverable after the fact. Phase 3 works around that by
(a) recording a separate `mic_activity.json` "You" timeline and (b) diarizing the mix with the You
ranges masked out. It works at stop but is fragile on **reprocess** — the persisted timeline has to
re-align against freshly-transcribed segments over a mono mix, and it currently collapses to a
single "Speaker 2". The timelines are being written correctly (verified: real `you_spans`), so the
failure is the masked-mono re-alignment itself, not the capture.
**Fix the root, not the instance:** keep both sources *physically separate in the recording* by
recording **stereo — left channel = microphone ("You"), right channel = system/loopback
("Speaker")**. The separation then lives in the file forever; reprocess and diarization always have
both streams; no sidecar timeline, no masking, no summed-mono clustering.
## Key facts that make this cheap
- `audio.wav` is **already 2-channel** on a normal (stereo) render endpoint — today we *sum the mic
into both channels*. Writing mic→L and loopback(downmixed)→R is the **same channel count, same
bytes** — no size increase in the common case.
- Transcription already downmixes to mono: `read_wav_mono_16k` averaging L+R of a split file yields
`(mic + loopback)/2` = the same summed signal it gets today, so **transcription is unchanged**.
- The mic↔loopback alignment machinery (`MicBridge`, the drain fix, the capture-silence fix) carries
over unchanged — we just route the aligned samples to two channels instead of summing them.
## Design
### 1. Capture write path (`audio`)
- Force the recording spec to **2ch, 16-bit, native rate** when the mic is enabled ("split"
layout). Loopback-only (mic off) keeps today's behavior.
- New writer (replacing `write_wav_bytes`'s summing for split recordings): per frame,
**L = mic sample** (0 when the mic underran), **R = downmix(loopback channels)**.
- Idle path (loopback silent): **L = drained mic, R = 0** — same silence-synthesis + device-paced
drain we just fixed, now writing to L instead of summing.
### 2. Layout versioning (distinguish old "summed" from new "split")
Content alone can't tell a summed-stereo file from a split-stereo one, so mark it explicitly:
- Add `audio_layout TEXT` to the `meetings` row: `'summed'` (default / NULL for all existing rows
and imports) or `'split'` (new mic-enabled recordings).
- Include it in the **`MeetingBundle` manifest** so exported recordings stay interpretable on
another machine.
- Playback and diarization branch on it; old recordings keep every current code path untouched.
### 3. Read helpers (`audio`)
- `read_wav_mono_16k` — **unchanged** (downmix all channels → mono 16k). Used for transcription of
both layouts; on a split file it returns the summed signal, exactly as wanted.
- New `read_wav_channel_16k(path, channel)` — extract one channel → mono 16k. Diarization reads the
**right** channel (far side); "You" detection reads the **left** (mic).
### 4. Diarization / attribution (`commands`) — *simpler* than Phase 3
For a **split** recording, one path replaces the timeline + masking:
```
far_spans = diarizer.diarize_samples( read_wav_channel_16k(wav, RIGHT) ) // clean far side
you_spans = vad_spans( read_wav_channel_16k(wav, LEFT) ) // energy-windowed mic
merge + assign_by_overlap + build_name_map("You") // You + Speaker N
```
- The mic is **never** in the clustering input — structurally, not procedurally.
- `vad_spans` is the `MicActivity` windowing logic (energy floor + 100 ms windows) applied to the L
channel from the file — so **stop and reprocess share one path** and produce identical results.
- **This deletes `MicActivity`, `mic_activity.json` (persist + read + delete), and `mask_ranges`.**
Net: less code than the current Phase 3.
- Mic-off (`'summed'`) recordings keep the whole-signal pass + voiceprint fallback.
### 5. Playback (`commands::serve_recording`)
- A split file plays mic-in-left / computer-in-right — wrong for a listener. On playback, **downmix
L+R → mono** (dual-mono) so both sources come out both ears.
- `serve_recording` already reads+decrypts the whole file into memory; add a downmix-to-mono step
for `'split'` layout and serve that mono WAV (range/seek computed over the transcoded bytes).
`'summed'` files stream as today.
### 6. Migration & portability
- **Existing recordings** (`'summed'`): every current path unchanged. Their reprocess falls back to
the voiceprint match (acceptable — they predate split). We may retire the `MicActivity`/mask code
once split is default; the handful of recent test recordings lose Phase 3 on reprocess, which is
fine.
- **Bundle export**: export **downmixes the canonical split `audio.wav` to mono** so a shared file
plays normally in any player; the working file stays split for WA's own re-processing.
## What this removes / simplifies
| Removed | Replaced by |
|---|---|
| `MicActivity` struct + capture wiring | L-channel VAD read from the file |
| `mic_activity.json` (write/read/delete) | nothing — recompute from L channel |
| `mask_ranges` + masked diarize | diarize the R channel directly |
| stop vs reprocess divergence | one shared split-attribution path |
The capture-silence fix, the device-paced `drain`, `diarize_samples`, and the `build_name_map`
naming all **stay**.
## Costs / risks (none fatal)
- **New code:** split writer, `audio_layout` column + migration + manifest field, channel read
helper, playback downmix, `vad_spans`, and the split-vs-summed branch. Bigger than a patch, but
it *retires* comparable Phase 3 code.
- **Loopback stereo image lost** (R = loopback downmixed to mono) — irrelevant for speech.
- **Mono render endpoint** (rare): 1ch→2ch doubles that recording's size.
- **Raw file plays split** in a third-party player until exported (export downmixes) — minor.
- **Clock drift** between mic/loopback now shows as tiny L/R desync instead of dropped summed
samples — harmless (diarization tolerates it; playback sums it away).
## Open questions for sign-off
1. Layout marker: **DB column** (proposed) vs. a self-describing WAV chunk? DB column is simpler and
travels via the bundle manifest; recommend it.
2. Retire `MicActivity`/`mask_ranges` outright, or keep them for existing `'summed'` recordings'
reprocess? Recommend **retire** (summed → voiceprint fallback) to avoid two live systems.
3. Export downmix to mono — agree that's the right default for shared/synced copies?
## Implementation order
1. `audio_layout` column + `MeetingBundle` field + read on `get_meeting` (no behavior yet).
2. Split writer + force 2ch when mic on; set `audio_layout='split'`. (`audio` + `start_recording`.)
3. `read_wav_channel_16k` + `vad_spans`; split-attribution path in stop + reprocess.
4. Playback downmix for split; bundle-export downmix.
5. Delete `MicActivity` / `mic_activity.json` / `mask_ranges`; update tests.
6. Apply the ADR-0005 amendment + update `docs/02`/`03`.
---
## Proposed ADR-0005 amendment (apply on approval)
> ### Phase 3.5 refinement — dual-channel capture (FR-SPK/FR-CAP, 2026-07-14)
>
> The Phase 3 per-stream attribution (above) reconstructs the mic/far-side split *after the fact*
> from a summed-mono `audio.wav` plus a `mic_activity.json` timeline, then masks the mic ranges
> before clustering. This is reliable at stop but fragile on reprocess (re-aligning a sidecar
> timeline against a mono mix), and it carries a parallel persistence path.
>
> **Decision:** when the microphone is enabled, record `audio.wav` as **stereo with the streams
> separated — left = microphone ("You"), right = system/loopback ("Speaker")** — instead of summing
> them. The separation is then intrinsic to the recording:
> - Diarization runs on the **right channel only** → `Speaker N`; "You" comes from **left-channel**
> voice activity. The mic is never clustered, by construction.
> - Reprocess recomputes both from the file — no sidecar timeline, no masking.
> - Transcription still downmixes (L+R) to the same summed mono it uses today; playback and bundle
> export downmix to mono so shared/played audio is normal.
> - A `meetings.audio_layout` flag (`summed` | `split`, in the bundle manifest for portability)
> distinguishes recordings; existing `summed` recordings keep the original blind-clustering +
> voiceprint path. On a stereo render endpoint this is **size-neutral** (the file was already 2ch
> with the mic summed into both).
>
> This **supersedes** `MicActivity` / `mic_activity.json` / masked diarization from Phase 3; the
> capture-silence handling, `diarize_samples`, and the `build_name_map` naming are retained. Segment
> IDs and the names-in-DB rule are unchanged.
+1 -1
View File
@@ -1,7 +1,7 @@
{
"name": "whispassist",
"private": true,
"version": "0.5.1",
"version": "0.7.2",
"type": "module",
"description": "Privacy-first, fully local Windows meeting assistant.",
"license": "MIT OR Apache-2.0",
+57 -2
View File
@@ -124,6 +124,17 @@ version = "1.1.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1505bd5d3d116872e7271a6d4e16d81d0c8570876c8de68093a09ac269d8aac0"
[[package]]
name = "auto-launch"
version = "0.5.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1f012b8cc0c850f34117ec8252a44418f2e34a2cf501de89e29b241ae5f79471"
dependencies = [
"dirs 4.0.0",
"thiserror 1.0.69",
"winreg 0.10.1",
]
[[package]]
name = "autocfg"
version = "1.5.1"
@@ -848,6 +859,15 @@ dependencies = [
"subtle",
]
[[package]]
name = "dirs"
version = "4.0.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ca3aa72a6f96ea37bbc5aa912f6788242832f75369bdfdadcb0e38423f100059"
dependencies = [
"dirs-sys 0.3.7",
]
[[package]]
name = "dirs"
version = "5.0.1"
@@ -866,6 +886,17 @@ dependencies = [
"dirs-sys 0.5.0",
]
[[package]]
name = "dirs-sys"
version = "0.3.7"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1b1d1d91c932ef41c0f2663aa8b0ca0342d444d842c06914aa0a7e352d0bada6"
dependencies = [
"libc",
"redox_users 0.4.6",
"winapi",
]
[[package]]
name = "dirs-sys"
version = "0.4.1"
@@ -1043,7 +1074,7 @@ dependencies = [
"rustc_version",
"toml 1.1.2+spec-1.1.0",
"vswhom",
"winreg",
"winreg 0.55.0",
]
[[package]]
@@ -4935,6 +4966,20 @@ dependencies = [
"walkdir",
]
[[package]]
name = "tauri-plugin-autostart"
version = "2.5.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "459383cebc193cdd03d1ba4acc40f2c408a7abce419d64bdcd2d745bc2886f70"
dependencies = [
"auto-launch",
"serde",
"serde_json",
"tauri",
"tauri-plugin",
"thiserror 2.0.18",
]
[[package]]
name = "tauri-plugin-dialog"
version = "2.7.1"
@@ -6043,7 +6088,7 @@ dependencies = [
[[package]]
name = "whispassist"
version = "0.5.1"
version = "0.7.2"
dependencies = [
"argon2",
"async-trait",
@@ -6073,6 +6118,7 @@ dependencies = [
"sqlx",
"tauri",
"tauri-build",
"tauri-plugin-autostart",
"tauri-plugin-dialog",
"thiserror 1.0.69",
"tokio",
@@ -6767,6 +6813,15 @@ dependencies = [
"memchr",
]
[[package]]
name = "winreg"
version = "0.10.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "80d0f4e272c85def139476380b12f9ac60926689dd2e01d4923222f40580869d"
dependencies = [
"winapi",
]
[[package]]
name = "winreg"
version = "0.55.0"
+3 -1
View File
@@ -1,6 +1,6 @@
[package]
name = "whispassist"
version = "0.5.1"
version = "0.7.2"
description = "Privacy-first, fully local Windows meeting assistant"
authors = ["WhispAssist contributors"]
license = "MIT OR Apache-2.0"
@@ -81,6 +81,7 @@ ort = { version = "=2.0.0-rc.10", optional = true, default-features = false, fea
rustfft = { version = "6", optional = true }
sherpa-rs = { version = "0.6", optional = true, default-features = false, features = ["download-binaries"] } # sherpa-onnx bindings (Phase 4, ADR-0005)
tauri-plugin-dialog = "2" # native Save/choose-folder (Phase 2 export)
tauri-plugin-autostart = "2" # opt-in launch-on-login (per-user HKCU\Run, no admin; NFR-RES-4)
# notes export (Phase 8, FR-NOTE-4) — pure-Rust, no external binary/cloud
# conversion service, consistent with the fully-local invariant.
@@ -99,6 +100,7 @@ windows = { version = "0.58", features = [
"Win32_Devices_DeviceAndDriverInstallation", # SetupAPI: NPU detection (Phase 3, T3.4)
"Win32_System_Com",
"Win32_UI_Shell", # SetCurrentProcessExplicitAppUserModelID (Phase 8, T8.6)
"Win32_UI_WindowsAndMessaging", # MessageBoxW: fatal-startup error dialog (see run())
"UI_Notifications", # scheduled toast reminders (Phase 8, T8.6, FR-CAL-5) — the
"Data_Xml_Dom", # OS delivers these itself at the due time, no polling timer
"Foundation",
File diff suppressed because one or more lines are too long
+42
View File
@@ -176,6 +176,48 @@
"Identifier": {
"description": "Permission identifier",
"oneOf": [
{
"description": "This permission set configures if your\napplication can enable or disable auto\nstarting the application on boot.\n\n#### Granted Permissions\n\nIt allows all to check, enable and\ndisable the automatic start on boot.\n\n\n#### This default permission set includes:\n\n- `allow-enable`\n- `allow-disable`\n- `allow-is-enabled`",
"type": "string",
"const": "autostart:default",
"markdownDescription": "This permission set configures if your\napplication can enable or disable auto\nstarting the application on boot.\n\n#### Granted Permissions\n\nIt allows all to check, enable and\ndisable the automatic start on boot.\n\n\n#### This default permission set includes:\n\n- `allow-enable`\n- `allow-disable`\n- `allow-is-enabled`"
},
{
"description": "Enables the disable command without any pre-configured scope.",
"type": "string",
"const": "autostart:allow-disable",
"markdownDescription": "Enables the disable command without any pre-configured scope."
},
{
"description": "Enables the enable command without any pre-configured scope.",
"type": "string",
"const": "autostart:allow-enable",
"markdownDescription": "Enables the enable command without any pre-configured scope."
},
{
"description": "Enables the is_enabled command without any pre-configured scope.",
"type": "string",
"const": "autostart:allow-is-enabled",
"markdownDescription": "Enables the is_enabled command without any pre-configured scope."
},
{
"description": "Denies the disable command without any pre-configured scope.",
"type": "string",
"const": "autostart:deny-disable",
"markdownDescription": "Denies the disable command without any pre-configured scope."
},
{
"description": "Denies the enable command without any pre-configured scope.",
"type": "string",
"const": "autostart:deny-enable",
"markdownDescription": "Denies the enable command without any pre-configured scope."
},
{
"description": "Denies the is_enabled command without any pre-configured scope.",
"type": "string",
"const": "autostart:deny-is-enabled",
"markdownDescription": "Denies the is_enabled command without any pre-configured scope."
},
{
"description": "Default core plugins set.\n#### This default permission set includes:\n\n- `core:path:default`\n- `core:event:default`\n- `core:window:default`\n- `core:webview:default`\n- `core:app:default`\n- `core:image:default`\n- `core:resources:default`\n- `core:menu:default`\n- `core:tray:default`",
"type": "string",
+42
View File
@@ -176,6 +176,48 @@
"Identifier": {
"description": "Permission identifier",
"oneOf": [
{
"description": "This permission set configures if your\napplication can enable or disable auto\nstarting the application on boot.\n\n#### Granted Permissions\n\nIt allows all to check, enable and\ndisable the automatic start on boot.\n\n\n#### This default permission set includes:\n\n- `allow-enable`\n- `allow-disable`\n- `allow-is-enabled`",
"type": "string",
"const": "autostart:default",
"markdownDescription": "This permission set configures if your\napplication can enable or disable auto\nstarting the application on boot.\n\n#### Granted Permissions\n\nIt allows all to check, enable and\ndisable the automatic start on boot.\n\n\n#### This default permission set includes:\n\n- `allow-enable`\n- `allow-disable`\n- `allow-is-enabled`"
},
{
"description": "Enables the disable command without any pre-configured scope.",
"type": "string",
"const": "autostart:allow-disable",
"markdownDescription": "Enables the disable command without any pre-configured scope."
},
{
"description": "Enables the enable command without any pre-configured scope.",
"type": "string",
"const": "autostart:allow-enable",
"markdownDescription": "Enables the enable command without any pre-configured scope."
},
{
"description": "Enables the is_enabled command without any pre-configured scope.",
"type": "string",
"const": "autostart:allow-is-enabled",
"markdownDescription": "Enables the is_enabled command without any pre-configured scope."
},
{
"description": "Denies the disable command without any pre-configured scope.",
"type": "string",
"const": "autostart:deny-disable",
"markdownDescription": "Denies the disable command without any pre-configured scope."
},
{
"description": "Denies the enable command without any pre-configured scope.",
"type": "string",
"const": "autostart:deny-enable",
"markdownDescription": "Denies the enable command without any pre-configured scope."
},
{
"description": "Denies the is_enabled command without any pre-configured scope.",
"type": "string",
"const": "autostart:deny-is-enabled",
"markdownDescription": "Denies the is_enabled command without any pre-configured scope."
},
{
"description": "Default core plugins set.\n#### This default permission set includes:\n\n- `core:path:default`\n- `core:event:default`\n- `core:window:default`\n- `core:webview:default`\n- `core:app:default`\n- `core:image:default`\n- `core:resources:default`\n- `core:menu:default`\n- `core:tray:default`",
"type": "string",
@@ -0,0 +1,12 @@
-- WhispAssist dual-channel capture (FR-SPK/FR-CAP). Forward-only migration.
-- Mirrors docs/03-data-model.md.
PRAGMA foreign_keys = ON;
-- How audio.wav stores its channels:
-- NULL / 'summed' — mic summed into every loopback channel (all pre-2026-07-14
-- recordings and every import): one mixed mono-equivalent.
-- 'split' — stereo, left = microphone ("You"), right = system/loopback
-- ("Speaker"), kept separate so diarization runs on the far
-- side alone. New mic-enabled recordings.
-- Diarization, playback (downmix), and bundle export branch on this.
ALTER TABLE meetings ADD COLUMN audio_layout TEXT;
+346 -27
View File
@@ -45,9 +45,26 @@ pub enum AudioError {
pub struct CaptureHandle {
running: Arc<AtomicBool>,
paused: Arc<AtomicBool>,
/// Mic-only (FR-CAP-7): when set, the microphone stream emits silence instead
/// of real samples — the recording's mic-left channel and the live transcript
/// go quiet, the meter drops to zero, while loopback keeps recording. Toggled
/// live via `set_muted` (the "press M to mute" control).
muted: Arc<AtomicBool>,
thread: JoinHandle<Result<CaptureSummary, AudioError>>,
}
impl CaptureHandle {
/// Mute/unmute this stream live. Only meaningful for the microphone capture.
pub fn set_muted(&self, muted: bool) {
self.muted.store(muted, Ordering::SeqCst);
}
/// Whether this stream is currently muted.
pub fn is_muted(&self) -> bool {
self.muted.load(Ordering::SeqCst)
}
}
/// Where captured frames are delivered for live transcription: mono f32 @ 16kHz,
/// bounded so a slow/absent consumer can never stall the capture thread.
pub type FrameSink = SyncSender<Vec<f32>>;
@@ -201,16 +218,37 @@ impl MicBridge {
}
}
}
/// Loopback idle path: take everything buffered at once, device-paced. The
/// idle writer must NOT meter the mic by wall-clock — that lets the buffer
/// back up to `cap` and drop the oldest samples, which sounds like the audio
/// skipping/fast-forwarding late in a long silent stretch. Draining fully
/// keeps the mic flowing at its own clock.
fn drain(&self) -> Vec<f32> {
self.buf
.lock()
.map(|mut buf| buf.drain(..).collect())
.unwrap_or_default()
}
}
/// Below this per-chunk RMS the mic is treated as silence/room noise and not
/// collected into a voiceprint. Without it the voiceprint was just the first
/// few seconds of the recording regardless of whether the user had spoken —
/// so a user silent for the first 30s got a noise voiceprint that spuriously
/// matched a cluster and mislabeled everyone "You". ponytail: fixed floor for
/// normalized [-1,1] speech; promote to a calibration knob if real mics vary.
const VOICE_ENERGY_FLOOR: f32 = 0.01;
/// A one-shot, bounded capture of raw mic-only audio (16kHz mono, same format
/// the transcriber and diarizer both use) taken early in a recording — enough
/// to compute a voiceprint that identifies which diarized speaker cluster is
/// the mic (so it can be labeled "You" instead of a clustered "S1"/"S2"; see
/// `diarization::voiceprint`). Unlike `MicBridge`, this is filled once and
/// never drained: the first `cap` samples are kept and everything after is
/// dropped, since a voiceprint only needs a few seconds of real speech, not
/// the whole meeting.
/// the transcriber and diarizer both use) taken from the user's early *speech*
/// — enough to compute a voiceprint that identifies which diarized speaker
/// cluster is the mic (so it can be labeled "You" instead of a clustered
/// "S1"/"S2"; see `diarization::voiceprint`). Unlike `MicBridge`, this is filled
/// once and never drained: the first `cap` samples of speech-level audio are
/// kept and everything after is dropped, since a voiceprint only needs a few
/// seconds of real speech, not the whole meeting. Silence is skipped so the
/// window tracks when the user actually talks, not the wall clock.
pub struct VoiceSample {
cap: usize,
buf: Mutex<Vec<f32>>,
@@ -225,6 +263,18 @@ impl VoiceSample {
}
fn push(&self, samples: &[f32]) {
// Skip silence/room noise so the voiceprint is the user's voice, not
// whatever the mic picked up before they first spoke (see
// VOICE_ENERGY_FLOOR). RMS inline so VoiceSample stays independent of
// the `audio`-gated level helper.
let rms = if samples.is_empty() {
0.0
} else {
(samples.iter().map(|s| s * s).sum::<f32>() / samples.len() as f32).sqrt()
};
if rms < VOICE_ENERGY_FLOOR {
return;
}
if let Ok(mut buf) = self.buf.lock() {
if buf.len() < self.cap {
buf.extend_from_slice(samples);
@@ -240,6 +290,12 @@ impl VoiceSample {
}
}
// MicActivity (the live mic-speech timeline) was retired 2026-07-14 with the
// masked-mono diarization it fed — the split (mic-left/loopback-right) layout
// makes the mic/far-side separation intrinsic to audio.wav, so "You" is
// recomputed from the left channel via `vad_spans`. See the retirement note in
// commands.rs (attribute_split).
/// Number of audio frames in a raw WASAPI byte buffer of the given format.
#[cfg(feature = "audio")]
fn frame_count(bytes: &[u8], format: &WaveFormat) -> usize {
@@ -273,12 +329,15 @@ impl WasapiCapture {
event_sink: EventSink,
bridge: Option<Arc<MicBridge>>,
voice_sample: Option<Arc<VoiceSample>>,
split: bool,
emit_level: bool,
) -> Result<CaptureHandle, AudioError> {
let running = Arc::new(AtomicBool::new(true));
let paused = Arc::new(AtomicBool::new(false));
let muted = Arc::new(AtomicBool::new(false));
let running_th = running.clone();
let paused_th = paused.clone();
let muted_th = muted.clone();
let wav_path = wav_path.map(Path::to_path_buf);
let device_id = device_id.map(str::to_string);
@@ -293,8 +352,10 @@ impl WasapiCapture {
&event_sink,
&running_th,
&paused_th,
&muted_th,
bridge.as_ref(),
voice_sample.as_ref(),
split,
emit_level,
)
})
@@ -303,6 +364,7 @@ impl WasapiCapture {
Ok(CaptureHandle {
running,
paused,
muted,
thread,
})
}
@@ -327,7 +389,8 @@ impl WasapiCapture {
event_sink,
Some(bridge),
None,
true,
true, // split: mic-enabled loopback records mic-left / loopback-right
true, // emit_level
)
}
@@ -352,7 +415,8 @@ impl WasapiCapture {
event_sink,
Some(bridge),
voice_sample,
true,
false, // split: mic stream has no writer
true, // emit_level
)
}
}
@@ -375,7 +439,8 @@ impl AudioCapture for WasapiCapture {
event_sink,
None,
None,
true,
false, // split: loopback-only / mic stream — no split writer
true, // emit_level
)
}
@@ -394,7 +459,8 @@ impl AudioCapture for WasapiCapture {
event_sink,
None,
None,
true,
false, // split: loopback-only / mic stream — no split writer
true, // emit_level
)
}
@@ -540,6 +606,18 @@ fn audio_level(mono: &[f32], mic: bool) -> AudioLevel {
/// without flooding the frontend with an event per ~10ms WASAPI callback.
const LEVEL_EMIT_INTERVAL: Duration = Duration::from_millis(50);
/// When the loopback (render) endpoint is idle, WASAPI delivers no packets, so
/// the recording would stall and the bridged mic — the user talking over system
/// silence — would be lost as the MicBridge overflows. The loopback loop then
/// synthesizes silence for the elapsed gap and folds the mic in, keeping
/// `audio.wav` continuous and complete (FR-CAP-7). MIN gates on a *sustained*
/// gap so brief inter-packet stalls during active playback don't punch silence
/// into the loopback (audible clicks); it stays well under the MicBridge's
/// ~0.5s cap so the mic can't back up and drop samples between drains. MAX caps
/// the silence padding written in one chunk.
const SILENCE_TICK_MIN: Duration = Duration::from_millis(150);
const SILENCE_TICK_MAX: Duration = Duration::from_millis(500);
/// Runs on a dedicated OS thread for the lifetime of a `CaptureHandle`. Owns the
/// WASAPI client and the WAV writer; exits (and finalizes the WAV) once `running`
/// is cleared.
@@ -553,8 +631,15 @@ fn capture_loop(
event_sink: &EventSink,
running: &AtomicBool,
paused: &AtomicBool,
// Mic-only live mute (FR-CAP-7): zeroes the decoded mic samples so the
// recording, transcript, and meter all go silent while loopback continues.
muted: &AtomicBool,
bridge: Option<&Arc<MicBridge>>,
voice_sample: Option<&Arc<VoiceSample>>,
// FR-SPK: when true, the loopback WAV is stereo L=mic / R=loopback (the mic
// is a separate track, not summed in) so diarization can run on the far side
// alone. Only meaningful for the loopback writer.
split: bool,
emit_level: bool,
) -> Result<CaptureSummary, AudioError> {
wasapi::initialize_mta()
@@ -566,14 +651,18 @@ fn capture_loop(
// UI can overlay them — but only the loopback stream writes a WAV.
let mut session = open_capture_session(&direction, device_id)?;
// Loopback publishes its rate so the mic knows what to resample to before
// pushing into the shared bridge (mic-into-recording, FR-CAP-7).
// pushing into the shared bridge (mic-into-recording, FR-CAP-7), and so the
// Phase 3 mic-activity timeline maps frame positions to windows.
if is_loopback {
if let Some(b) = bridge {
b.rate
.store(session.format.get_samplespersec(), Ordering::Relaxed);
}
}
let spec = wav_spec_for(&session.format)?;
let mut spec = wav_spec_for(&session.format)?;
if split && is_loopback {
spec.channels = 2; // L = mic, R = loopback (downmixed)
}
let mut writer = match wav_path {
Some(path) => Some(WavWriter::create(path, spec).map_err(|e| {
AudioError::Capture(format!("could not create {}: {e}", path.display()))
@@ -593,6 +682,9 @@ fn capture_loop(
let mut queue: std::collections::VecDeque<u8> = std::collections::VecDeque::new();
let mut frames_written: u64 = 0;
let mut last_level_emit = Instant::now() - LEVEL_EMIT_INTERVAL;
// Wall-clock of the last WAV write (real or synthesized), so the loopback
// loop can fill idle gaps with silence+mic instead of stalling.
let mut last_write = Instant::now();
while running.load(Ordering::Relaxed) {
// Short timeout so we periodically re-check `running` even with no data.
@@ -636,8 +728,43 @@ fn capture_loop(
continue;
}
if queue.is_empty() {
// Loopback idle (nothing playing): keep audio.wav advancing so the
// bridged mic (user talking over system silence) is preserved
// instead of lost to MicBridge overflow. Only after a *sustained*
// gap, so brief inter-packet stalls during active playback don't
// punch silence into the loopback (clicks). Loopback + mic-enabled
// only; the mic capture stream has no writer.
if is_loopback && !paused.load(Ordering::Relaxed) {
if let (Some(w), Some(b)) = (writer.as_mut(), bridge) {
let elapsed = last_write.elapsed();
if elapsed >= SILENCE_TICK_MIN {
// Drain the whole mic buffer (device-paced). Metering the
// pull by wall-clock let the buffer back up to its cap and
// drop the oldest samples — the audio skipped/fast-
// forwarded late in a long gap. Draining fully avoids that.
let mic = b.drain();
let rate = session.format.get_samplespersec() as usize;
// Advance the timeline by the wall-clock gap (capped), but
// never fewer frames than the mic we must write.
let gap_frames = (elapsed.min(SILENCE_TICK_MAX).as_micros() as usize
* rate
/ 1_000_000)
.max(mic.len());
let bpf = session.format.get_nchannels() as usize
* (session.format.get_bitspersample() as usize / 8);
let silence = vec![0u8; gap_frames * bpf];
frames_written += if split {
write_wav_split(w, &silence, &session.format, &mic)?
} else {
write_wav_bytes(w, &silence, &session.format, &mic)?
};
last_write = Instant::now();
}
}
}
continue;
}
last_write = Instant::now();
let bytes: Vec<u8> = queue.drain(..).collect();
// Must keep pulling WASAPI buffers even while paused (required to avoid
@@ -653,9 +780,20 @@ fn capture_loop(
Some(b) if is_loopback => b.pull(frame_count(&bytes, &session.format)),
_ => Vec::new(),
};
frames_written += write_wav_bytes(w, &bytes, &session.format, &mic)?;
frames_written += if split {
write_wav_split(w, &bytes, &session.format, &mic)?
} else {
write_wav_bytes(w, &bytes, &session.format, &mic)?
};
}
let mut mono = decode_mono_f32(&bytes, &session.format)?;
// Mic muted: replace the decoded samples with silence before anything
// downstream sees them — the recording's mic channel, the bridge, the
// transcript feed, the meter, and the voiceprint sample all go quiet.
// Loopback (`is_loopback`) is never muted this way.
if !is_loopback && muted.load(Ordering::Relaxed) {
mono.iter_mut().for_each(|s| *s = 0.0);
}
let mono = decode_mono_f32(&bytes, &session.format)?;
// Mic: feed the shared bridge (resampled to the loopback's rate) so the
// loopback thread can fold it into the recording.
@@ -783,6 +921,53 @@ fn f32_to_i16(v: f32) -> i16 {
(v.clamp(-1.0, 1.0) * i16::MAX as f32) as i16
}
/// Split layout (FR-SPK): write frames as **stereo, L = mic, R = loopback**
/// (downmixed to mono), so the two sources stay separate in `audio.wav` and
/// diarization can run on the far side alone. `loopback_bytes` is the raw WASAPI
/// mix buffer (all-zero for a synthesized-silence chunk); `mic` may be shorter
/// than the frame count (rest silent) or empty (mic underran). Returns frames.
fn write_wav_split(
writer: &mut WavWriter<BufWriter<File>>,
loopback_bytes: &[u8],
format: &WaveFormat,
mic: &[f32],
) -> Result<u64, AudioError> {
let sample_type = format
.get_subformat()
.map_err(|e| AudioError::Device(format!("unrecognized mix format: {e}")))?;
let channels = format.get_nchannels() as usize;
if channels == 0 {
return Ok(0);
}
let (bytes_per_sample, decode): (usize, fn(&[u8]) -> f32) = match (sample_type, format.get_bitspersample()) {
(SampleType::Float, 32) => (4, |c| f32::from_le_bytes(c.try_into().unwrap())),
(SampleType::Int, 16) => (2, |c| {
i16::from_le_bytes(c.try_into().unwrap()) as f32 / i16::MAX as f32
}),
(st, bits) => {
return Err(AudioError::Device(format!(
"unsupported capture format: {st} {bits}-bit"
)))
}
};
let mut frames = 0u64;
for frame in loopback_bytes.chunks_exact(bytes_per_sample * channels) {
// R = loopback downmixed to mono.
let loop_mono =
frame.chunks_exact(bytes_per_sample).map(decode).sum::<f32>() / channels as f32;
// L = mic for this frame (silent if the mic underran).
let mic_s = mic.get(frames as usize).copied().unwrap_or(0.0);
writer
.write_sample(f32_to_i16(mic_s))
.map_err(|e| AudioError::Capture(format!("wav write: {e}")))?;
writer
.write_sample(f32_to_i16(loop_mono))
.map_err(|e| AudioError::Capture(format!("wav write: {e}")))?;
frames += 1;
}
Ok(frames)
}
/// Downmix raw WASAPI capture bytes to mono `f32` in `[-1.0, 1.0]`, at the
/// device's native sample rate (resampling to 16kHz happens separately).
fn decode_mono_f32(bytes: &[u8], format: &WaveFormat) -> Result<Vec<f32>, AudioError> {
@@ -897,6 +1082,22 @@ impl Resampler {
/// through hound's size-aware iterator; hound is only used to parse the
/// format/`fmt ` chunk.
pub fn read_wav_mono_16k(path: &Path) -> Result<Vec<f32>, AudioError> {
read_wav_16k(path, None)
}
/// Like [`read_wav_mono_16k`] but extracts a single channel instead of
/// downmixing — used by the Phase 3.5 split path to diarize the right
/// (loopback) channel and VAD the left (mic) channel (FR-SPK). `channel` is
/// clamped to the file's last channel, so it's safe on a mono/summed file.
pub fn read_wav_channel_16k(path: &Path, channel: usize) -> Result<Vec<f32>, AudioError> {
read_wav_16k(path, Some(channel))
}
/// Shared reader: decrypt (T8.8), reduce each frame to one f32 — averaging all
/// channels (`channel = None`) or picking one (`channel = Some`) — resample to
/// 16kHz. Reads the `data` payload as raw bytes (an unfinalized WAV's size
/// fields read back as 0), so it works on a mid-write / crash-left file (FR-REL-1).
fn read_wav_16k(path: &Path, channel: Option<usize>) -> Result<Vec<f32>, AudioError> {
let raw = std::fs::read(path).map_err(|e| AudioError::Capture(format!("read wav: {e}")))?;
// Decrypt if the recording was sealed at rest (T8.8); plaintext passes through.
let file =
@@ -906,6 +1107,8 @@ pub fn read_wav_mono_16k(path: &Path) -> Result<Vec<f32>, AudioError> {
.map_err(|e| AudioError::Capture(format!("open wav: {e}")))?
.spec();
let channels = spec.channels.max(1) as usize;
// Which channel index to pick (clamped), or None to downmix.
let pick = channel.map(|c| c.min(channels - 1));
let data_marker = file
.windows(4)
@@ -917,22 +1120,14 @@ pub fn read_wav_mono_16k(path: &Path) -> Result<Vec<f32>, AudioError> {
let mono: Vec<f32> = match (spec.sample_format, spec.bits_per_sample) {
(SampleFormat::Float, 32) => bytes
.chunks_exact(4 * channels)
.map(|frame| {
frame
.chunks_exact(4)
.map(|c| f32::from_le_bytes(c.try_into().unwrap()))
.sum::<f32>()
/ channels as f32
})
.map(|frame| reduce_frame(frame, 4, pick, channels, |c| f32::from_le_bytes(c.try_into().unwrap())))
.collect(),
(SampleFormat::Int, 16) => bytes
.chunks_exact(2 * channels)
.map(|frame| {
frame
.chunks_exact(2)
.map(|c| i16::from_le_bytes(c.try_into().unwrap()) as f32 / i16::MAX as f32)
.sum::<f32>()
/ channels as f32
reduce_frame(frame, 2, pick, channels, |c| {
i16::from_le_bytes(c.try_into().unwrap()) as f32 / i16::MAX as f32
})
})
.collect(),
(fmt, bits) => {
@@ -946,6 +1141,78 @@ pub fn read_wav_mono_16k(path: &Path) -> Result<Vec<f32>, AudioError> {
Ok(resampler.process(&mono))
}
/// Voice-activity "You" spans (FR-SPK): 100ms windows of 16kHz mono audio above
/// [`VOICE_ENERGY_FLOOR`], merged into `(start_ms, end_ms)` ranges. Run on the
/// left (mic) channel of a split recording to attribute segments to "You" —
/// replaces the old live `MicActivity` timeline (recomputed from the file, so
/// stop and reprocess agree).
pub fn vad_spans(samples: &[f32]) -> Vec<(u64, u64)> {
const WINDOW_MS: u64 = 100;
let win = (TARGET_SAMPLE_RATE as u64 * WINDOW_MS / 1000) as usize; // 1600 @16kHz
if win == 0 {
return Vec::new();
}
let mut spans = Vec::new();
let mut start: Option<u64> = None;
let mut w = 0u64;
let mut i = 0usize;
while i < samples.len() {
let chunk = &samples[i..(i + win).min(samples.len())];
let rms = (chunk.iter().map(|s| s * s).sum::<f32>() / chunk.len().max(1) as f32).sqrt();
match (rms >= VOICE_ENERGY_FLOOR, start) {
(true, None) => start = Some(w * WINDOW_MS),
(false, Some(s)) => {
spans.push((s, w * WINDOW_MS));
start = None;
}
_ => {}
}
i += win;
w += 1;
}
if let Some(s) = start {
spans.push((s, w * WINDOW_MS));
}
spans
}
/// Reduce one interleaved frame to a single f32: pick channel `pick` or average
/// all `channels`. `width` is bytes-per-sample; `decode` turns those bytes → f32.
fn reduce_frame(
frame: &[u8],
width: usize,
pick: Option<usize>,
channels: usize,
decode: impl Fn(&[u8]) -> f32,
) -> f32 {
match pick {
Some(c) => decode(&frame[c * width..c * width + width]),
None => frame.chunks_exact(width).map(decode).sum::<f32>() / channels as f32,
}
}
/// Write mono 16kHz `f32` samples as a Float32 WAV — the format
/// [`read_wav_mono_16k`] reads back. Used to persist the mic voiceprint sample
/// next to a retained recording so it can be re-matched on reprocess (FR-SPK).
pub fn write_wav_mono_16k(path: &Path, samples: &[f32]) -> Result<(), AudioError> {
let spec = WavSpec {
channels: 1,
sample_rate: 16_000,
bits_per_sample: 32,
sample_format: SampleFormat::Float,
};
let mut writer =
WavWriter::create(path, spec).map_err(|e| AudioError::Capture(format!("write wav: {e}")))?;
for &s in samples {
writer
.write_sample(s)
.map_err(|e| AudioError::Capture(format!("write wav: {e}")))?;
}
writer
.finalize()
.map_err(|e| AudioError::Capture(format!("write wav: {e}")))
}
/// Ceiling on how far the still-flowing stream may run ahead of a stalled
/// partner before the mixer forwards it alone — so a muted/dead microphone (or a
/// silent system output) can't hold up the live transcript. 0.5s @ 16kHz.
@@ -1051,6 +1318,31 @@ fn mixer_loop(
mod tests {
use super::*;
#[test]
fn vad_spans_ignores_silence() {
// 200ms of 16kHz silence → no "You" spans.
assert!(vad_spans(&[0.0; 3_200]).is_empty());
}
#[test]
fn vad_spans_collapses_speech_into_spans_with_gaps() {
// 16kHz: 1600 samples per 100ms window. Speech in windows 0,1 (0-200ms)
// and 4,5 (400-600ms), silence between.
let mut samples = vec![0.0f32; 9_600]; // 600ms
samples[0..3_200].fill(0.2); // windows 0,1
samples[6_400..9_600].fill(0.2); // windows 4,5
assert_eq!(vad_spans(&samples), vec![(0, 200), (400, 600)]);
}
#[test]
fn voice_sample_skips_silence_and_keeps_speech() {
let vs = VoiceSample::new(16_000);
vs.push(&[0.0; 4_000]); // silence: below the energy floor
assert!(vs.samples().is_empty(), "silence must not fill the voiceprint");
vs.push(&[0.2; 4_000]); // speech-level audio
assert_eq!(vs.samples().len(), 4_000, "speech-level audio is collected");
}
#[test]
fn mixer_sums_aligned_streams_and_keeps_the_unmatched_tail() {
let mut m = Mixer::new();
@@ -1127,6 +1419,33 @@ mod tests {
let _ = std::fs::remove_dir_all(&dir);
}
#[test]
fn write_wav_split_puts_mic_left_and_loopback_right() {
let dir = std::env::temp_dir().join(format!("wa-test-{}", uuid::Uuid::new_v4()));
std::fs::create_dir_all(&dir).unwrap();
let path = dir.join("audio.wav");
// 2ch float loopback input; split output is always 2ch 16-bit.
let format = WaveFormat::new(32, 32, &SampleType::Float, 48_000, 2, None);
let spec = WavSpec {
channels: 2,
sample_rate: 48_000,
bits_per_sample: 16,
sample_format: SampleFormat::Int,
};
let mut writer = WavWriter::create(&path, spec).unwrap();
// One 2ch loopback frame [0.4, 0.6] → R = downmix 0.5; mic 0.2 → L.
let loop_bytes: Vec<u8> = [0.4f32, 0.6].iter().flat_map(|s| s.to_le_bytes()).collect();
let n = write_wav_split(&mut writer, &loop_bytes, &format, &[0.2]).unwrap();
writer.finalize().unwrap();
assert_eq!(n, 1);
let mut reader = hound::WavReader::open(&path).unwrap();
let s: Vec<i16> = reader.samples::<i16>().map(|x| x.unwrap()).collect();
assert!((s[0] as f32 / i16::MAX as f32 - 0.2).abs() < 0.001); // L = mic
assert!((s[1] as f32 / i16::MAX as f32 - 0.5).abs() < 0.001); // R = loopback downmix
let _ = std::fs::remove_dir_all(&dir);
}
#[test]
fn f32_to_i16_clamps_and_scales() {
assert_eq!(f32_to_i16(0.0), 0);
+993 -57
View File
File diff suppressed because it is too large Load Diff
+178
View File
@@ -0,0 +1,178 @@
//! Enterprise deployment: seed default settings from an admin-supplied `.ini`
//! on **first run only** (before any `settings.json` exists).
//!
//! An admin mass-deploying WhispAssist (GPO / SCCM / Intune) drops a
//! `wa-defaults.ini` and every fresh install picks it up once, seeding
//! `settings.json` with their chosen defaults (record-by-default, preferred
//! backend, retention, model to auto-download, …) — all via native Windows file
//! deployment, no WiX custom actions. See `docs/enterprise-deployment.md`.
//!
//! **Guardrail (CLAUDE.md):** the file must never carry secrets. Keys that look
//! like credentials are ignored here as defense in depth — API keys / OAuth
//! tokens live only in the OS credential store.
use crate::models::Settings;
use serde_json::{Map, Value};
use std::path::PathBuf;
/// Special (non-`Settings`) INI key: when truthy, the first-run seed also fetches
/// the configured `whisper_model` in the background so the machine is ready
/// offline. Stripped before the settings merge.
const AUTO_DOWNLOAD_KEY: &str = "auto_download_model";
/// Candidate locations, first found wins:
/// 1. `%PROGRAMDATA%\WhispAssist\wa-defaults.ini` — machine-wide enterprise path.
/// 2. `<exe dir>\wa-defaults.ini` — the bundled template / per-install override.
fn candidate_paths() -> Vec<PathBuf> {
let mut paths = Vec::new();
if let Ok(program_data) = std::env::var("ProgramData") {
paths.push(PathBuf::from(program_data).join("WhispAssist").join("wa-defaults.ini"));
}
if let Ok(exe) = std::env::current_exe() {
if let Some(dir) = exe.parent() {
paths.push(dir.join("wa-defaults.ini"));
}
}
paths
}
/// Reads the first existing defaults file and produces the seeded settings plus
/// the whisper model id to auto-download (if `auto_download_model` was set).
/// `None` when no file exists or it contains no overrides (the shipped template
/// is fully commented, so normal installs get exactly today's behavior).
pub fn seed_settings_from_defaults() -> Option<(Settings, Option<String>)> {
let text = candidate_paths()
.into_iter()
.find_map(|p| std::fs::read_to_string(p).ok())?;
seed_from_ini(&text)
}
/// Testable core: parse INI text → merge onto the built-in defaults.
fn seed_from_ini(text: &str) -> Option<(Settings, Option<String>)> {
let mut overrides = parse_ini(text);
if overrides.is_empty() {
return None;
}
// Pull the non-Settings auto-download flag out before the merge.
let auto_download = overrides
.remove(AUTO_DOWNLOAD_KEY)
.map(|v| truthy(&v))
.unwrap_or(false);
// Merge overrides onto the default settings' JSON form, then deserialize.
// Unknown keys (typos) are ignored — `Settings` has no deny_unknown_fields.
let mut base = match serde_json::to_value(crate::commands::default_settings()) {
Ok(Value::Object(map)) => map,
_ => return None,
};
for (k, v) in overrides {
base.insert(k, v);
}
let settings: Settings = serde_json::from_value(Value::Object(base)).ok()?;
let model = if auto_download {
Some(settings.whisper_model.clone())
} else {
None
};
Some((settings, model))
}
/// Minimal INI reader: skips blanks, `;`/`#` comments and `[section]` headers;
/// splits each `key = value` on the first `=`; coerces values to bool / integer /
/// string so serde lands them on the typed `Settings` fields. Silently drops any
/// key that looks like a secret (guardrail — no credentials in the deploy file).
fn parse_ini(text: &str) -> Map<String, Value> {
let mut map = Map::new();
for line in text.lines() {
let line = line.trim();
if line.is_empty()
|| line.starts_with(';')
|| line.starts_with('#')
|| line.starts_with('[')
{
continue;
}
let Some((key, value)) = line.split_once('=') else {
continue;
};
let key = key.trim().to_string();
let value = value.trim();
if key.is_empty() || looks_like_secret(&key) {
continue;
}
map.insert(key, coerce(value));
}
map
}
/// `true`/`false` → bool, all-integer → number, everything else → string.
fn coerce(value: &str) -> Value {
match value.to_ascii_lowercase().as_str() {
"true" => return Value::Bool(true),
"false" => return Value::Bool(false),
_ => {}
}
if let Ok(n) = value.parse::<i64>() {
return Value::Number(n.into());
}
Value::String(value.to_string())
}
fn truthy(v: &Value) -> bool {
matches!(v, Value::Bool(true)) || matches!(v, Value::String(s) if s.eq_ignore_ascii_case("true"))
}
/// Defense in depth: never seed anything that smells like a credential.
fn looks_like_secret(key: &str) -> bool {
let k = key.to_ascii_lowercase();
["key", "token", "secret", "credential", "password"]
.iter()
.any(|needle| k.contains(needle))
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn fully_commented_file_is_a_noop() {
let ini = "; default_record = true\n# preferred_backend = cpu\n[general]\n\n";
assert!(seed_from_ini(ini).is_none());
}
#[test]
fn coerces_bool_int_and_string_fields() {
let ini = "default_record = true\nretention_max_age_days = 90\npreferred_backend = cpu\n";
let (settings, model) = seed_from_ini(ini).expect("overrides present");
assert!(settings.default_record);
assert_eq!(settings.retention_max_age_days, Some(90));
assert_eq!(settings.preferred_backend, "cpu");
assert!(model.is_none());
}
#[test]
fn auto_download_returns_the_configured_model() {
let ini = "whisper_model = base.en-q5_1\nauto_download_model = true\n";
let (_settings, model) = seed_from_ini(ini).expect("overrides present");
assert_eq!(model.as_deref(), Some("base.en-q5_1"));
}
#[test]
fn unset_fields_keep_their_defaults() {
let ini = "default_record = true\n";
let (settings, _) = seed_from_ini(ini).unwrap();
// microphone stays on, auto_start stays off — only the named key changed.
assert!(settings.microphone_enabled);
assert!(!settings.auto_start);
}
#[test]
fn secret_keys_are_ignored() {
let ini = "anthropic_api_key = sk-should-be-dropped\ndefault_record = true\n";
let map = parse_ini(ini);
assert!(!map.contains_key("anthropic_api_key"));
assert!(map.contains_key("default_record"));
}
}
+164 -2
View File
@@ -23,6 +23,11 @@ pub enum DiarError {
pub trait Diarizer: Send + Sync {
/// Partition audio into speaker spans.
fn diarize(&self, wav: &Path) -> Result<Vec<SpeakerSpan>, DiarError>;
/// Partition already-loaded 16kHz mono samples into speaker spans. Phase 3
/// (FR-SPK) diarizes a *masked* far-side signal (the mic's "You" ranges
/// zeroed), so the caller loads and masks the samples itself rather than
/// pointing at the raw WAV.
fn diarize_samples(&self, samples: Vec<f32>) -> Result<Vec<SpeakerSpan>, DiarError>;
/// Assign speaker labels to transcript segments by overlap with spans.
fn assign(&self, segments: &mut [TranscriptSegment], spans: &[SpeakerSpan]);
}
@@ -53,6 +58,55 @@ pub fn assign_by_overlap(segments: &mut [TranscriptSegment], spans: &[SpeakerSpa
}
}
/// Split-layout attribution (FR-SPK): decides per segment between "You" (mic
/// channel voice activity) and the far side's diarized speakers by comparing
/// the *total* voiced overlap on each channel, not by picking the single
/// longest span — a long far-side diarizer span could otherwise swallow a
/// segment the user spoke most of, showing their words under "Speaker N".
/// The mic channel is physically the user's voice alone, so channel evidence
/// outranks cluster evidence; ties go to "You" (mislabeling the user's own
/// words as someone else is the worse failure). A segment with no voiced
/// overlap on either channel keeps its prior label rather than guessing.
// ponytail: whole-segment labels — a segment genuinely containing both sides
// still gets one speaker; the upgrade path is transcribing each channel
// separately so segments can never mix voices.
pub fn assign_split(
segments: &mut [TranscriptSegment],
you_spans: &[(u64, u64)],
far_vad: &[(u64, u64)],
far_spans: &[SpeakerSpan],
) {
fn overlap(a0: u64, a1: u64, b0: u64, b1: u64) -> u64 {
a1.min(b1).saturating_sub(a0.max(b0))
}
for seg in segments.iter_mut() {
let mic_ms: u64 = you_spans
.iter()
.map(|&(s, e)| overlap(seg.start_ms, seg.end_ms, s, e))
.sum();
let far_ms: u64 = far_vad
.iter()
.map(|&(s, e)| overlap(seg.start_ms, seg.end_ms, s, e))
.sum();
if mic_ms == 0 && far_ms == 0 {
continue;
}
if mic_ms >= far_ms {
seg.speaker = "You".to_string();
} else if let Some(span) = far_spans
.iter()
.map(|sp| (overlap(seg.start_ms, seg.end_ms, sp.start_ms, sp.end_ms), sp))
.filter(|(o, _)| *o > 0)
.max_by_key(|(o, _)| *o)
.map(|(_, sp)| sp)
{
seg.speaker = span.speaker.clone();
}
// Far side voiced but no diarizer span overlaps (e.g. a sub-700ms span
// was filtered): keep the prior label rather than guess.
}
}
/// sherpa-onnx-backed diarizer: pyannote segmentation + speaker-embedding +
/// fast clustering (ADR-0005, T4.1). `Diarize::compute` needs `&mut self`; it's
/// wrapped in a `Mutex` to satisfy `Diarizer: Sync` — diarization is a
@@ -70,7 +124,13 @@ impl SherpaDiarizer {
// sherpa-onnx to pick the cluster count itself from `threshold`
// instead of forcing a fixed number of speakers.
num_clusters: Some(-1),
threshold: Some(0.5),
// Tuning evidence: 0.5 over the summed mic+loopback mono WAV
// exploded a 2-speaker conversation into 83 clusters (overlapped
// speech + short pyannote chunks yield unstable embeddings). sherpa
// fast-clustering: larger threshold → fewer clusters. 0.7 lands a
// 2-person meeting near 2–4. ponytail: code constant, no settings
// knob until 0.7 proves insufficient in the field.
threshold: Some(0.7),
..Default::default()
};
let engine = sherpa_rs::diarize::Diarize::new(segmentation_model, embedding_model, config)
@@ -86,11 +146,28 @@ impl Diarizer for SherpaDiarizer {
fn diarize(&self, wav: &Path) -> Result<Vec<SpeakerSpan>, DiarError> {
let samples =
crate::audio::read_wav_mono_16k(wav).map_err(|e| DiarError::Run(e.to_string()))?;
self.diarize_samples(samples)
}
fn diarize_samples(&self, samples: Vec<f32>) -> Result<Vec<SpeakerSpan>, DiarError> {
// Guard the native FFI against degenerate input (a fully-masked far side
// is all zeros; a tiny buffer has no usable window). sherpa can crash
// rather than error on these, so skip cleanly instead.
if samples.len() < MIN_DIARIZE_SAMPLES {
return Ok(Vec::new());
}
let mut engine = self.engine.lock().unwrap_or_else(|e| e.into_inner());
let segments = engine
.compute(samples, None)
.map_err(|e| DiarError::Run(e.to_string()))?;
Ok(segments.into_iter().map(segment_to_span).collect())
// Drop sub-700ms spans: their embeddings are unstable and only cause
// label churn in assign_by_overlap (which keeps the prior label when no
// span overlaps). See MIN_SPAN_MS.
Ok(segments
.into_iter()
.map(segment_to_span)
.filter(span_long_enough)
.collect())
}
fn assign(&self, segments: &mut [TranscriptSegment], spans: &[SpeakerSpan]) {
@@ -98,6 +175,23 @@ impl Diarizer for SherpaDiarizer {
}
}
/// Minimum span duration kept from a diarization pass. Sub-second pyannote
/// chunks carry unstable ERes2Net embeddings and only churn labels.
#[cfg(feature = "diarization")]
const MIN_SPAN_MS: u64 = 700;
/// Below this many 16kHz samples (~1s) the diarizer skips the sherpa FFI: a
/// fully-masked far side is all zeros and a tiny buffer has no usable
/// segmentation window, either of which can crash the native code.
#[cfg(feature = "diarization")]
const MIN_DIARIZE_SAMPLES: usize = 16_000;
/// A span is kept only if it lasts at least [`MIN_SPAN_MS`].
#[cfg(feature = "diarization")]
fn span_long_enough(span: &SpeakerSpan) -> bool {
span.end_ms.saturating_sub(span.start_ms) >= MIN_SPAN_MS
}
/// sherpa-onnx speaker indices are 0-based; WA's internal labels are 1-based ("S1"…).
#[cfg(feature = "diarization")]
fn segment_to_span(seg: sherpa_rs::diarize::Segment) -> SpeakerSpan {
@@ -165,6 +259,58 @@ mod overlap_tests {
assign_by_overlap(&mut segments, &[]);
assert_eq!(segments[0].speaker, "S1");
}
#[test]
fn split_labels_a_mic_dominant_segment_you_even_against_a_longer_far_span() {
// The user spoke 0-4000ms; the far side 4000-6000ms — but the far
// cluster span covers the whole window, so the old merged max-overlap
// pick handed the entire segment (the user's words included) to the
// far speaker. Channel totals must side with the mic instead.
let mut segments = vec![segment(0, 6000)];
let you = vec![(0u64, 4000u64)];
let far_vad = vec![(4000u64, 6000u64)];
let far_spans = vec![span(0, 6000, "S1")]; // long far cluster span
assign_split(&mut segments, &you, &far_vad, &far_spans);
assert_eq!(segments[0].speaker, "You");
}
#[test]
fn split_ties_go_to_you() {
let mut segments = vec![segment(0, 2000)];
let you = vec![(0u64, 1000u64)];
let far_vad = vec![(1000u64, 2000u64)];
let far_spans = vec![span(1000, 2000, "S1")];
assign_split(&mut segments, &you, &far_vad, &far_spans);
assert_eq!(segments[0].speaker, "You");
}
#[test]
fn split_assigns_the_best_far_span_when_the_far_side_dominates() {
let mut segments = vec![segment(0, 3000)];
let you = vec![(0u64, 500u64)];
let far_vad = vec![(500u64, 3000u64)];
let far_spans = vec![span(500, 1000, "S1"), span(1000, 3000, "S2")];
assign_split(&mut segments, &you, &far_vad, &far_spans);
assert_eq!(segments[0].speaker, "S2");
}
#[test]
fn split_keeps_the_prior_label_when_both_channels_are_silent() {
let mut segments = vec![segment(5000, 6000)];
segments[0].speaker = "S9".to_string();
assign_split(&mut segments, &[(0, 1000)], &[(0, 1000)], &[span(0, 1000, "S1")]);
assert_eq!(segments[0].speaker, "S9");
}
#[test]
fn split_keeps_the_prior_label_when_far_is_voiced_but_no_far_span_overlaps() {
// Far VAD hears speech but every diarizer span was filtered (sub-700ms):
// don't guess a label.
let mut segments = vec![segment(0, 1000)];
segments[0].speaker = "S3".to_string();
assign_split(&mut segments, &[], &[(0, 1000)], &[span(2000, 3000, "S1")]);
assert_eq!(segments[0].speaker, "S3");
}
}
#[cfg(all(test, feature = "diarization"))]
@@ -184,6 +330,22 @@ mod tests {
assert_eq!(span.speaker, "S1");
}
#[test]
fn span_long_enough_filters_sub_minimum_spans() {
let short = SpeakerSpan {
start_ms: 0,
end_ms: MIN_SPAN_MS - 1,
speaker: "S1".to_string(),
};
let exact = SpeakerSpan {
start_ms: 0,
end_ms: MIN_SPAN_MS,
speaker: "S1".to_string(),
};
assert!(!span_long_enough(&short));
assert!(span_long_enough(&exact));
}
#[test]
fn new_surfaces_a_load_error_for_missing_models_instead_of_panicking() {
let result =
+9 -4
View File
@@ -110,8 +110,9 @@ pub fn match_mic_speaker(
}
/// Distinct speaker labels in first-appearance order — spans come back from
/// the diarizer already sorted by start time.
fn first_appearance_order(spans: &[SpeakerSpan]) -> Vec<String> {
/// the diarizer already sorted by start time. `pub(crate)` so the Phase 3
/// per-stream path can reuse it to name its merged You + far-side spans.
pub(crate) fn first_appearance_order(spans: &[SpeakerSpan]) -> Vec<String> {
let mut seen = std::collections::HashSet::new();
spans
.iter()
@@ -152,8 +153,12 @@ fn cosine_similarity(a: &[f32], b: &[f32]) -> f32 {
/// `mic_label` -> "You"; every other label, in first-appearance order ->
/// "Speaker 2", "Speaker 3", … (numbering starts at 2 — "You" stands in for
/// "Speaker 1" without ever being called that).
fn build_name_map(labels_in_order: &[String], mic_label: &str) -> HashMap<String, String> {
/// "Speaker 1" without ever being called that). `pub(crate)` so Phase 3 reuses
/// it for its merged You + far-side spans, keeping naming uniform app-wide.
pub(crate) fn build_name_map(
labels_in_order: &[String],
mic_label: &str,
) -> HashMap<String, String> {
let mut names = HashMap::new();
let mut next_speaker_number = 2;
for label in labels_in_order {
+158 -6
View File
@@ -9,6 +9,7 @@ pub mod audio;
pub mod briefs;
pub mod calendar;
pub mod commands;
pub mod deploy;
pub mod diarization;
pub mod error;
pub mod hardware;
@@ -27,7 +28,8 @@ pub mod vault;
use std::path::PathBuf;
use std::sync::{Arc, Mutex as StdMutex};
use std::thread::JoinHandle;
use tauri::tray::TrayIcon;
use tauri::menu::{Menu, MenuItem};
use tauri::tray::{MouseButton, MouseButtonState, TrayIcon, TrayIconBuilder, TrayIconEvent};
use tauri::Manager;
use tokio::sync::Mutex;
@@ -85,6 +87,10 @@ pub struct RecordingSession {
/// auto-labeled "You" instead of a clustered "S1"/"S2". `None` when the
/// mic is disabled (same conditions as `mic_capture`).
pub mic_voice_sample: Option<Arc<audio::VoiceSample>>,
/// `audio.wav` channel layout for this recording (FR-SPK): `"split"` (mic
/// enabled → stereo mic-left/loopback-right) or `"summed"` (mic off). Passed
/// to `finalize_meeting` so playback/diarization/export can branch on it.
pub audio_layout: &'static str,
/// Live notes redesign: raw user-authored notes accumulated *during* the
/// recording (freeform text + per-moment annotations) — see
/// `models::ManualNotes`. Mutated by `update_live_notes`/`set_segment_note`
@@ -98,18 +104,67 @@ pub struct RecordingSession {
/// tooltip on recording state changes (FR-CAP-4).
pub struct TrayHandle(pub TrayIcon);
/// Show a blocking native error dialog and exit non-zero. Used for failures
/// that happen before the Tauri app (and its window) exist, where a silent
/// `panic=abort` fast-fail would otherwise close with no explanation. Exits
/// cleanly (code 1) rather than aborting so the OS reports no crash.
fn fatal_startup_error(body: &str) -> ! {
tracing::error!("fatal startup error: {body}");
#[cfg(windows)]
{
use windows::core::PCWSTR;
use windows::Win32::UI::WindowsAndMessaging::{
MessageBoxW, MB_ICONERROR, MB_OK, MB_SYSTEMMODAL,
};
// Null-terminated UTF-16 for the Win32 wide API.
let to_wide = |s: &str| s.encode_utf16().chain(std::iter::once(0)).collect::<Vec<u16>>();
let text = to_wide(body);
let caption = to_wide("WhispAssist can't start");
// SAFETY: both buffers are null-terminated and outlive the call.
unsafe {
MessageBoxW(
None,
PCWSTR(text.as_ptr()),
PCWSTR(caption.as_ptr()),
MB_OK | MB_ICONERROR | MB_SYSTEMMODAL,
);
}
}
#[cfg(not(windows))]
eprintln!("WhispAssist can't start: {body}");
std::process::exit(1);
}
/// Build state, register commands/events, and run the app.
pub fn run() {
tracing_subscriber::fmt().with_env_filter("info").init();
let store: Arc<dyn storage::Store> = Arc::new(
tauri::async_runtime::block_on(storage::SqliteStore::connect())
.expect("failed to initialize storage (wa.db)"),
);
let store: Arc<dyn storage::Store> =
match tauri::async_runtime::block_on(storage::SqliteStore::connect()) {
Ok(s) => Arc::new(s),
// A migration checksum mismatch (an edited-after-shipping migration)
// or a locked/corrupt file lands here. Tell the user instead of
// fast-failing into a silent no-launch.
Err(e) => fatal_startup_error(&format!(
"WhispAssist couldn't open its database (wa.db):\n\n{e}\n\n\
Your recordings and notes are not deleted. This usually means the \
database is from a newer version of WhispAssist, or a data migration \
is incompatible. Reinstalling the matching version, or moving wa.db \
aside from %LOCALAPPDATA%\\WhispAssist, will let the app start."
)),
};
let store_for_setup = store.clone();
tauri::Builder::default()
.plugin(tauri_plugin_dialog::init())
// Opt-in launch-at-login (NFR-RES-4). The macOS launcher arg is required
// by the signature but unused on Windows, where enable/disable writes a
// per-user HKCU\...\Run entry (no admin). Off until the user (or an
// enterprise deploy file) turns `auto_start` on.
.plugin(tauri_plugin_autostart::init(
tauri_plugin_autostart::MacosLauncher::LaunchAgent,
None,
))
// In-memory streaming of recordings for the player (FR-REC-5): decrypts
// on the fly so no plaintext audio is ever written to disk.
.register_uri_scheme_protocol("waaudio", |_ctx, request| {
@@ -120,13 +175,84 @@ pub fn run() {
session: Mutex::new(None),
})
.setup(move |app| {
// Single tray icon (the `trayIcon` in tauri.conf.json was removed so
// this is the only one). It carries a Show/Quit menu and, on
// left-click, restores the window — the always-available way back
// from "close to tray".
let icon = tauri::image::Image::from_bytes(include_bytes!("../icons/tray.png"))?;
let tray = tauri::tray::TrayIconBuilder::new()
let show_item = MenuItem::with_id(app, "show", "Show WhispAssist", true, None::<&str>)?;
let quit_item = MenuItem::with_id(app, "quit", "Quit", true, None::<&str>)?;
let menu = Menu::with_items(app, &[&show_item, &quit_item])?;
let tray = TrayIconBuilder::new()
.icon(icon)
.tooltip("WhispAssist — idle")
.menu(&menu)
.show_menu_on_left_click(false)
.on_menu_event(|app, event| match event.id.as_ref() {
"show" => show_main_window(app),
"quit" => app.exit(0),
_ => {}
})
.on_tray_icon_event(|tray, event| {
if let TrayIconEvent::Click {
button: MouseButton::Left,
button_state: MouseButtonState::Up,
..
} = event
{
show_main_window(tray.app_handle());
}
})
.build(app)?;
app.manage(TrayHandle(tray));
// First-run enterprise deploy seeding (deploy.rs): if no settings.json
// exists yet and an admin dropped a wa-defaults.ini, seed settings once
// and optionally fetch the configured model in the background. One-shot
// — guarded by the settings file's absence, so it never re-runs and adds
// nothing to idle cost (NFR-RES-1).
if !crate::paths::settings_path().exists() {
if let Some((seeded, model_to_download)) = deploy::seed_settings_from_defaults() {
match commands::save_settings(&seeded) {
Ok(()) => {
tracing::info!("seeded settings.json from wa-defaults.ini");
if let Some(id) = model_to_download {
let app_handle = app.handle().clone();
tauri::async_runtime::spawn(async move {
if let Err(e) = commands::download_model(
app_handle,
commands::DownloadModelArgs {
kind: "whisper".into(),
id,
},
)
.await
{
tracing::warn!("deploy auto-download of model failed: {e:?}");
}
});
}
}
Err(e) => {
tracing::error!("first-run deploy seeding failed to write settings: {e:?}")
}
}
}
}
// Reconcile launch-at-login with the persisted preference (NFR-RES-4):
// if the user opted in but the OS entry is missing (e.g. after a
// reinstall or a deploy file that set auto_start), restore it. One-shot.
{
use tauri_plugin_autostart::ManagerExt;
let manager = app.autolaunch();
if commands::load_settings().auto_start && !manager.is_enabled().unwrap_or(false) {
if let Err(e) = manager.enable() {
tracing::warn!("failed to restore auto-start entry: {e}");
}
}
}
// Reminders (Phase 8, T8.6, FR-CAL-5) are Windows-scheduled toasts, not
// an app-side timer — Windows itself is what's "polling", so this stays
// within NFR-RES-1. init() just registers the AppUserModelID.
@@ -220,6 +346,17 @@ pub fn run() {
});
Ok(())
})
// Close to tray (keep running in background): when the setting is on,
// the window X hides instead of quitting; the tray "Quit" is the real
// exit. Off → default behavior (closing the window quits the app).
.on_window_event(|window, event| {
if let tauri::WindowEvent::CloseRequested { api, .. } = event {
if commands::load_settings().close_to_tray {
api.prevent_close();
let _ = window.hide();
}
}
})
.invoke_handler(tauri::generate_handler![
commands::start_recording,
commands::stop_recording,
@@ -227,6 +364,7 @@ pub fn run() {
commands::recording_playback_path,
commands::pause_recording,
commands::resume_recording,
commands::toggle_microphone_mute,
commands::set_recording_retention,
commands::acknowledge_recording_consent,
commands::update_live_notes,
@@ -238,6 +376,9 @@ pub fn run() {
commands::list_audio_devices,
commands::list_input_devices,
commands::set_preferred_backend,
commands::set_auto_start,
commands::monitor_audio_level,
commands::stress_test_hardware,
commands::list_models,
commands::list_whisper_languages,
commands::download_npu_package,
@@ -266,6 +407,7 @@ pub fn run() {
commands::generate_summary,
commands::confirm_action_items,
commands::generate_tags,
commands::enhance_notes,
commands::llm_setup_suggestions,
commands::pull_ollama_model,
commands::import_pst,
@@ -310,6 +452,16 @@ pub fn run() {
.expect("error while running WhispAssist");
}
/// Restore the main window from the tray (show + unminimize + focus). Shared by
/// the tray left-click and the "Show WhispAssist" menu item.
fn show_main_window(app: &tauri::AppHandle) {
if let Some(w) = app.get_webview_window("main") {
let _ = w.show();
let _ = w.unminimize();
let _ = w.set_focus();
}
}
/// Used by `commands.rs` to keep the tray tooltip honest about capture state (FR-CAP-4).
pub(crate) fn update_tray_tooltip(app: &tauri::AppHandle, text: &str) {
if let Some(tray) = app.try_state::<TrayHandle>() {
+16
View File
@@ -214,6 +214,11 @@ pub struct MeetingBundle {
pub tags: Vec<String>,
pub speakers: Vec<SpeakerInfo>,
pub action_items: Vec<ActionItem>,
/// `audio.wav` channel layout (FR-SPK/FR-CAP): `"split"` or `"summed"`.
/// `default` so bundles exported before this field deserialize as `None`
/// (treated as `"summed"`).
#[serde(default)]
pub audio_layout: Option<String>,
}
#[derive(Debug, Clone, Serialize, Deserialize)]
@@ -349,6 +354,17 @@ pub struct Settings {
pub mcp_expose: String, // none|selected|all
#[serde(default)]
pub mcp_expose_recordings: bool,
/// Launch WhispAssist automatically at login (opt-in, NFR-RES-4). OFF by
/// default; toggled via `set_auto_start`, which writes a per-user
/// `HKCU\...\Run` entry through `tauri-plugin-autostart` (no admin). An
/// enterprise deploy file may set this to `true` (see `deploy.rs`).
#[serde(default)]
pub auto_start: bool,
/// Closing the window hides WhispAssist to the system tray instead of
/// quitting, so it keeps running in the background (tray "Quit" really
/// exits). ON by default; the tray icon is the always-available way back.
#[serde(default = "default_true")]
pub close_to_tray: bool,
}
fn default_mcp_transport() -> String {
+45 -2
View File
@@ -74,6 +74,10 @@ pub struct FinalizeMeeting {
pub language: Option<String>,
pub backend_used: Option<String>,
pub model_used: Option<String>,
/// How `audio.wav`'s channels are laid out (FR-SPK/FR-CAP): `"split"` =
/// stereo mic-left/loopback-right, `"summed"`/`None` = mic mixed into every
/// channel. `None` leaves the stored value unchanged (e.g. on reprocess).
pub audio_layout: Option<String>,
}
/// Full meeting detail: DB row + transcript + speakers + notes (`get_meeting`'s
@@ -91,6 +95,10 @@ pub struct Meeting {
pub language: Option<String>,
pub backend_used: Option<String>,
pub model_used: Option<String>,
/// `audio.wav` channel layout (FR-SPK/FR-CAP): `"split"` (mic-left /
/// loopback-right) or `"summed"`/`None` (mic mixed into every channel).
/// Diarization, playback downmix, and bundle export branch on it.
pub audio_layout: Option<String>,
pub segments: Vec<TranscriptSegment>,
pub speakers: Vec<SpeakerInfo>,
pub notes_markdown: String,
@@ -268,6 +276,10 @@ pub trait Store: Send + Sync {
) -> Result<Vec<MeetingListItem>, StoreError>;
async fn get_meeting(&self, id: &MeetingId) -> Result<Meeting, StoreError>;
async fn delete_meeting(&self, id: &MeetingId) -> Result<(), StoreError>;
/// Overwrite a meeting's lifecycle `status` (e.g. mark a background import
/// `transcribing` while it runs, or `error` if it fails). `finalize_meeting`
/// is still the only path to `ready`.
async fn set_meeting_status(&self, id: &MeetingId, status: &str) -> Result<(), StoreError>;
async fn update_notes(&self, id: &MeetingId, markdown: &str) -> Result<(), StoreError>;
/// (Re)builds this meeting's FTS index row from the current title and
/// whatever's on disk/in the DB for transcript/notes/summary/tags (Phase
@@ -284,6 +296,14 @@ pub trait Store: Send + Sync {
label: &str,
name: &str,
) -> Result<(), StoreError>;
/// Delete every `speakers` row for a meeting (labels, names, participant
/// links, merges). Used before a full re-diarization rebuild
/// (`reprocess_transcript`, FR-SPK): the old labels key to the previous
/// clustering and are meaningless once the audio is re-clustered, so the
/// caller re-inserts the fresh set via `finalize_meeting`. Without this,
/// stale labels from an over-split run linger in the DB (and the
/// Participants pane) even though the transcript no longer references them.
async fn clear_speakers(&self, id: &MeetingId) -> Result<(), StoreError>;
/// Fold over-split speaker labels into one canonical label (T4.5,
/// FR-SPK-3). Segment speaker IDs in storage are never rewritten
/// (FR-SPK-5) — `get_meeting` resolves `from` labels to `into` when it
@@ -734,7 +754,8 @@ impl Store for SqliteStore {
let now = now_unix();
sqlx::query(
"UPDATE meetings SET status = 'ready', ended_at = ?, duration_secs = ?, recorded = ?,
language = ?, backend_used = ?, model_used = ?, updated_at = ? WHERE id = ?",
language = ?, backend_used = ?, model_used = ?,
audio_layout = COALESCE(?, audio_layout), updated_at = ? WHERE id = ?",
)
.bind(now)
.bind(s.duration_secs)
@@ -742,6 +763,7 @@ impl Store for SqliteStore {
.bind(&s.language)
.bind(&s.backend_used)
.bind(&s.model_used)
.bind(&s.audio_layout)
.bind(now)
.bind(id)
.execute(&self.pool)
@@ -857,7 +879,7 @@ impl Store for SqliteStore {
async fn get_meeting(&self, id: &MeetingId) -> Result<Meeting, StoreError> {
let row = sqlx::query(
"SELECT id, title, started_at, ended_at, duration_secs, status, recorded, language, backend_used, model_used, calendar_event_id, template_id
"SELECT id, title, started_at, ended_at, duration_secs, status, recorded, language, backend_used, model_used, calendar_event_id, template_id, audio_layout
FROM meetings WHERE id = ?",
)
.bind(id)
@@ -928,6 +950,7 @@ impl Store for SqliteStore {
language: row.get("language"),
backend_used: row.get("backend_used"),
model_used: row.get("model_used"),
audio_layout: row.get("audio_layout"),
segments,
speakers,
notes_markdown,
@@ -957,6 +980,16 @@ impl Store for SqliteStore {
Ok(())
}
async fn set_meeting_status(&self, id: &MeetingId, status: &str) -> Result<(), StoreError> {
sqlx::query("UPDATE meetings SET status = ?, updated_at = ? WHERE id = ?")
.bind(status)
.bind(now_unix())
.bind(id)
.execute(&self.pool)
.await?;
Ok(())
}
async fn update_notes(&self, id: &MeetingId, markdown: &str) -> Result<(), StoreError> {
write_artifact(
&paths::meeting_dir(id).join("notes.md"),
@@ -1252,6 +1285,14 @@ impl Store for SqliteStore {
.await
}
async fn clear_speakers(&self, id: &MeetingId) -> Result<(), StoreError> {
sqlx::query("DELETE FROM speakers WHERE meeting_id = ?")
.bind(id)
.execute(&self.pool)
.await?;
Ok(())
}
async fn merge_speakers(
&self,
id: &MeetingId,
@@ -2015,6 +2056,7 @@ mod tests {
language: Some("fr".to_string()), // what auto-detect resolved to
backend_used: Some("cpu".to_string()),
model_used: Some("small-q5_1".to_string()),
audio_layout: None,
},
)
.await
@@ -2058,6 +2100,7 @@ mod tests {
language: Some("en".to_string()),
backend_used: Some("cpu".to_string()),
model_used: Some("small-q5_1".to_string()),
audio_layout: None,
},
)
.await
+6 -6
View File
@@ -1,7 +1,7 @@
{
"$schema": "https://schema.tauri.app/config/2",
"productName": "WhispAssist",
"version": "0.5.1",
"version": "0.7.2",
"identifier": "bet.dou.whispassist",
"build": {
"frontendDist": "../dist",
@@ -22,18 +22,18 @@
],
"security": {
"csp": "default-src 'self'; connect-src 'self' http://localhost:* http://127.0.0.1:*; img-src 'self' data:; media-src 'self' http://waaudio.localhost; style-src 'self' 'unsafe-inline'"
},
"trayIcon": {
"iconPath": "icons/tray.png",
"tooltip": "WhispAssist"
}
},
"bundle": {
"active": true,
"targets": ["msi", "nsis"],
"icon": ["icons/icon.ico"],
"resources": ["wa-defaults.ini"],
"windows": {
"webviewInstallMode": { "type": "downloadBootstrapper" }
"webviewInstallMode": { "type": "downloadBootstrapper" },
"nsis": {
"installMode": "both"
}
}
}
}
+1 -1
View File
@@ -1,6 +1,6 @@
{
"$schema": "gen/schemas/desktop-schema.json",
"bundle": {
"resources": ["vulkan-1.dll"]
"resources": ["vulkan-1.dll", "wa-defaults.ini"]
}
}
+60
View File
@@ -0,0 +1,60 @@
; ============================================================================
; WhispAssist enterprise deployment defaults (wa-defaults.ini)
; ============================================================================
; Read ONCE, on a machine's FIRST launch (before settings.json exists), to
; seed the app's default settings. After that the user's own settings.json
; wins and this file is ignored. Deploy it with native Windows tooling
; (Group Policy / SCCM / Intune file copy) to either location — the first
; one found wins:
;
; 1. %PROGRAMDATA%\WhispAssist\wa-defaults.ini (machine-wide)
; 2. <install dir>\wa-defaults.ini (this bundled template)
;
; As shipped, every setting below is COMMENTED OUT, so a normal install
; behaves exactly as if this file were absent. Uncomment and edit the lines
; you want to preset. Format is flat "key = value" — no sections required;
; [section] headers, ';' and '#' comment lines are ignored.
;
; Values: true / false for switches; a plain number for numeric fields;
; text otherwise. Unknown / misspelled keys are ignored.
;
; SECURITY: never put secrets here. API keys, OAuth tokens and sync
; passwords live only in the OS credential store; any key containing
; "key", "token", "secret", "credential" or "password" is dropped on read.
; ----------------------------------------------------------------------------
; ---- Recording (ADR-0009) --------------------------------------------------
; Record every meeting to disk by default (consent notice still applies).
; default_record = false
; ---- Transcription backend & model -----------------------------------------
; preferred_backend = auto ; auto | npu | nvidia | amd | intel | cpu
; whisper_model = base.en-q5_1 ; a catalog model id
; whisper_language = auto ; auto | ISO-639-1 code (multilingual model only)
; low_overhead = false
; Download whisper_model in the background on first launch so the machine is
; ready offline. Requires network at first run.
; auto_download_model = false
; ---- Storage & retention (FR-STORE-2) --------------------------------------
; storage_root = C:\ProgramData\WhispAssist\data
; retention_max_age_days = 90
; retention_max_size_gb = 20
; ---- Local LLM / summaries (ADR-0007) --------------------------------------
; llm_provider = ollama ; ollama | custom | anthropic | off
; llm_endpoint = http://localhost:11434
; llm_model = llama3
; ---- Capture & UX ----------------------------------------------------------
; microphone_enabled = true
; auto_record_calendar = false
; theme = system ; system | light | dark
; ---- Startup (NFR-RES-4) ---------------------------------------------------
; Launch WhispAssist automatically at login for the user.
; auto_start = false
; ---- Sync master switch (ADR-0010; targets/creds configured in-app) --------
; sync_enabled = false
+74 -24
View File
@@ -12,6 +12,7 @@
import { recording } from "./lib/stores/recording.svelte";
import { settings } from "./lib/stores/settings.svelte";
import { meetings } from "./lib/stores/meetings.svelte";
import { imports } from "./lib/stores/imports.svelte";
import { calendar } from "./lib/stores/calendar.svelte";
import { api, type NoteTemplate, type CalendarEvent } from "./lib/api";
import { onMount } from "svelte";
@@ -24,6 +25,8 @@
Square,
Trash2,
FilePlus,
Mic,
MicOff,
Settings as SettingsIcon,
AlertTriangle,
PanelLeftClose,
@@ -131,6 +134,7 @@
recording.init();
settings.load();
meetings.init();
imports.init(); // live background-import progress for the tracker
calendar.load(); // events power the auto-record timer above
checkVault();
api
@@ -207,6 +211,18 @@
} else if (e.ctrlKey && e.key === ",") {
e.preventDefault();
showSettings = !showSettings;
} else if (
// Press "M" to mute/unmute the mic mid-meeting (FR-CAP-7). Bare key (no
// modifiers) and only while recording with the mic on.
e.key.toLowerCase() === "m" &&
!e.ctrlKey &&
!e.metaKey &&
!e.altKey &&
recording.state !== "idle" &&
settings.settings.microphone_enabled
) {
e.preventDefault();
recording.toggleMute();
}
}
@@ -232,10 +248,16 @@
<div class="app" data-theme={resolvedTheme}>
<div class="sr-only" role="status" aria-live="polite">{recordingAnnouncement}</div>
<header class="bar">
<strong>WhispAssist</strong>
<span class="muted">{t("app.tagline")}</span>
<div class="spacer"></div>
{#if recording.state === "idle"}
<button
class="record-btn"
onclick={startRecording}
title={t("app.record_title")}
aria-keyshortcuts="Control+Shift+R"
>
<Circle size={11} fill="currentColor" aria-hidden="true" />
{t("app.record")}
</button>
<select
class="theme-select"
bind:value={selectedTemplateId}
@@ -247,23 +269,6 @@
<option value={tpl.id}>{tpl.name}</option>
{/each}
</select>
<button
class="import-btn"
onclick={() => (showImport = true)}
title={t("app.add_meeting_title")}
>
<FilePlus size={13} aria-hidden="true" />
{t("app.add_meeting")}
</button>
<button
class="record-btn"
onclick={startRecording}
title={t("app.record_title")}
aria-keyshortcuts="Control+Shift+R"
>
<Circle size={11} fill="currentColor" aria-hidden="true" />
{t("app.record")}
</button>
{:else}
<button
class="stop-btn"
@@ -278,6 +283,18 @@
<Trash2 size={12} aria-hidden="true" />
{t("app.cancel")}
</button>
{/if}
<div class="spacer"></div>
{#if recording.state === "idle"}
<button
class="import-btn"
onclick={() => (showImport = true)}
title={t("app.add_meeting_title")}
>
<FilePlus size={13} aria-hidden="true" />
{t("app.add_meeting")}
</button>
{:else}
<span class="rec">
<span class="rec-dot" aria-hidden="true"></span>
{t("app.recording")}
@@ -289,6 +306,23 @@
micPeak={recording.levelPeakMic}
showMic={settings.settings.microphone_enabled}
/>
{#if settings.settings.microphone_enabled}
<button
class="mute-btn"
class:muted={recording.micMuted}
onclick={() => recording.toggleMute()}
aria-pressed={recording.micMuted}
aria-keyshortcuts="M"
title={recording.micMuted ? t("app.unmute_title") : t("app.mute_title")}
>
{#if recording.micMuted}
<MicOff size={14} aria-hidden="true" />
{:else}
<Mic size={14} aria-hidden="true" />
{/if}
<span class="sr-only">{recording.micMuted ? t("app.unmute") : t("app.mute")}</span>
</button>
{/if}
{#if settings.hardware}
<span class="backend" title={t("app.backend_title")}>{settings.hardware.active}</span>
{/if}
@@ -588,10 +622,6 @@
border-bottom: 1px solid var(--border);
background: var(--bg-elevated);
}
.bar strong {
font-size: 0.95rem;
letter-spacing: -0.01em;
}
.spacer {
flex: 1;
}
@@ -718,6 +748,26 @@
border-radius: var(--radius-full);
padding: 0.15rem 0.5rem;
}
.mute-btn {
display: inline-flex;
align-items: center;
justify-content: center;
width: 30px;
height: 30px;
color: var(--fg);
background: var(--bg);
border: 1px solid var(--border);
border-radius: var(--radius-full);
cursor: pointer;
}
.mute-btn:hover {
background: var(--bg-hover);
}
.mute-btn.muted {
color: var(--danger, #d33);
border-color: var(--danger, #d33);
background: color-mix(in srgb, var(--danger, #d33) 12%, transparent);
}
.retention {
display: flex;
align-items: center;
+72 -4
View File
@@ -56,6 +56,19 @@ export interface AudioDeviceInfo {
name: string;
}
// Quick hardware stress test (Settings ▸ Hardware): per-(backend, model)
// real-time factor, plus the recommended real-time-capable pairing.
export interface StressResult {
backend: string;
model: string;
rtf: number;
realtime: boolean;
}
export interface StressTestResult {
results: StressResult[];
recommended: { backend: string; model: string } | null;
}
export interface LlmStatus {
provider: string; // ollama|custom|anthropic|off (ADR-0011; "openai" not yet wired)
reachable: boolean;
@@ -83,6 +96,19 @@ export interface LanguageOption {
export type MeetingStatus = "recording" | "transcribing" | "ready" | "recovering" | "error";
// The four ordered phases of a background media import (import://progress).
export type ImportPhase = "prepare" | "transcribe" | "diarize" | "finalize";
// One `import://progress` tick. `state` is active (running), done (finished,
// `elapsedMs` set) or error (`error` message set) for the given `phase`.
export interface ImportProgress {
meetingId: MeetingId;
phase: ImportPhase;
state: "active" | "done" | "error";
elapsedMs: number | null;
error: string | null;
}
export interface MeetingListItem {
id: MeetingId;
title: string;
@@ -336,6 +362,12 @@ export interface AppSettings {
audio_output_device: string | null;
microphone_enabled: boolean;
audio_input_device: string | null;
/** Launch WhispAssist at login (opt-in, off by default; NFR-RES-4). Toggled
* via setAutoStart, which writes a per-user Run entry (no admin). */
auto_start: boolean;
/** Closing the window hides to the tray (keep running in background) instead
* of quitting; on by default. Tray "Quit" is the real exit. */
close_to_tray: boolean;
}
// Feature brief — agent-ready spec distilled from a meeting (ADR-0011).
@@ -398,6 +430,10 @@ export const api = {
invoke<string>("recording_playback_path", { meetingId }),
pauseRecording: (meetingId: MeetingId) => invoke<void>("pause_recording", { meetingId }),
resumeRecording: (meetingId: MeetingId) => invoke<void>("resume_recording", { meetingId }),
// Toggle mic mute for the active recording (FR-CAP-7); returns the new muted
// state. Errors if the meeting was started with the mic off.
toggleMicrophoneMute: (meetingId: MeetingId) =>
invoke<boolean>("toggle_microphone_mute", { meetingId }),
setRecordingRetention: (meetingId: MeetingId, record: boolean) =>
invoke<void>("set_recording_retention", { meetingId, record }),
acknowledgeRecordingConsent: () => invoke<void>("acknowledge_recording_consent"),
@@ -417,6 +453,11 @@ export const api = {
listInputDevices: () => invoke<AudioDeviceInfo[]>("list_input_devices"),
setPreferredBackend: (backend: BackendId | "auto") =>
invoke<void>("set_preferred_backend", { args: { backend } }),
setAutoStart: (enabled: boolean) => invoke<void>("set_auto_start", { enabled }),
// Test a device: stream device://level for a few seconds. Resolves when done.
monitorAudioLevel: (kind: "input" | "loopback", deviceId: string | null, durationMs = 6000) =>
invoke<void>("monitor_audio_level", { kind, deviceId, durationMs }),
stressTestHardware: () => invoke<StressTestResult>("stress_test_hardware"),
downloadNpuPackage: () => invoke<void>("download_npu_package"),
downloadDirectmlPackage: () => invoke<void>("download_directml_package"),
listModels: () => invoke<ModelInfo[]>("list_models"),
@@ -437,10 +478,12 @@ export const api = {
invoke<void>("reprocess_transcript", { meetingId, model, language }),
// Manually add a meeting from an existing recording — a local audio/video
// file path or a URL (YouTube/streaming page or direct media URL). Requires
// ffmpeg (and yt-dlp for URLs) on PATH; neither is bundled. Returns the new
// meeting's id once transcription + diarization have finished.
importMedia: (source: string, title?: string) =>
invoke<MeetingId>("import_media", { source, title }),
// ffmpeg (and yt-dlp for URLs) on PATH; neither is bundled. `model` overrides
// the Settings whisper model for this one import. Returns the new meeting's id
// *immediately*; transcode/transcribe/diarize run in the background and stream
// `import://progress` ticks, finishing with `transcript://finalized`.
importMedia: (source: string, title?: string, model?: string) =>
invoke<MeetingId>("import_media", { source, title, model }),
resumeTranscription: (meetingId: MeetingId) =>
invoke<void>("resume_transcription", { meetingId }),
listMeetings: (filter?: MeetingFilter) =>
@@ -460,6 +503,10 @@ export const api = {
deleteMeeting: (meetingId: MeetingId) => invoke<void>("delete_meeting", { meetingId }),
updateNotes: (meetingId: MeetingId, markdown: string) =>
invoke<void>("update_notes", { meetingId, markdown }),
// AI-enhance rough notes into structured Markdown grounded in the transcript
// (Granola-style). Returns the enhanced text; the caller decides to keep it.
enhanceNotes: (meetingId: MeetingId, notes: string) =>
invoke<string>("enhance_notes", { meetingId, notes }),
// dest is a file path for md/pdf/docx/obsidian, a folder for bundle.
// "obsidian" writes one self-contained vault note (no audio) — FR-STORE-4.
exportMeeting: (
@@ -588,12 +635,24 @@ export const events = {
onDeviceChanged: (
cb: (p: { meetingId: string; recovered: boolean; message: string }) => void,
): Promise<UnlistenFn> => listen("recording://device", (e) => cb(e.payload as never)),
// Mic mute toggled for the active recording (FR-CAP-7).
onMicMuted: (
cb: (p: { meetingId: string; muted: boolean }) => void,
): Promise<UnlistenFn> => listen("recording://mic", (e) => cb(e.payload as never)),
onSegment: (
cb: (p: { meetingId: string; segment: TranscriptSegment }) => void,
): Promise<UnlistenFn> => listen("transcript://segment", (e) => cb(e.payload as never)),
onFinalized: (
cb: (p: { meetingId: string; segmentCount: number }) => void,
): Promise<UnlistenFn> => listen("transcript://finalized", (e) => cb(e.payload as never)),
// Per-phase progress of a background media import (feeds the import tracker).
onImportProgress: (cb: (p: ImportProgress) => void): Promise<UnlistenFn> =>
listen("import://progress", (e) => cb(e.payload as never)),
// Live diarization refined the speaker list mid-recording (FR-SPK): updated
// labels/display names, including the mic speaker resolved to "You".
onDiarizationUpdated: (
cb: (p: { meetingId: string; speakers: SpeakerInfo[] }) => void,
): Promise<UnlistenFn> => listen("diarization://updated", (e) => cb(e.payload as never)),
onLlmToken: (cb: (p: { meetingId: string; text: string }) => void): Promise<UnlistenFn> =>
listen("llm://token", (e) => cb(e.payload as never)),
onLlmDone: (cb: (p: { meetingId: string; summary: SummaryFile }) => void): Promise<UnlistenFn> =>
@@ -604,6 +663,15 @@ export const events = {
onHardwareChanged: (
cb: (p: { active: BackendId; reason: string }) => void,
): Promise<UnlistenFn> => listen("hardware://changed", (e) => cb(e.payload as never)),
// Live level meter for a device test (Settings ▸ Hardware). `done` marks the
// end of the monitor window.
onDeviceLevel: (
cb: (p: { kind: string; rms?: number; peak?: number; done?: boolean }) => void,
): Promise<UnlistenFn> => listen("device://level", (e) => cb(e.payload as never)),
// Per-(backend, model) progress ticks during the quick stress test.
onStressProgress: (
cb: (p: { backend: string; model: string }) => void,
): Promise<UnlistenFn> => listen("stress://progress", (e) => cb(e.payload as never)),
onNpuDownload: (
cb: (p: {
stage: "model" | "runtime" | "done";
+99 -18
View File
@@ -1,22 +1,43 @@
<script lang="ts">
// Manually add a meeting from an existing recording (feature: "add a meeting
// + upload a video URL or audio file"). Transcoding is done by the backend
// via ffmpeg (+ yt-dlp for URLs) — both external, not bundled — so this is
// just a small form: pick a local file or paste a URL, optional title, go.
import { api, errorMessage } from "../api";
// via ffmpeg (+ yt-dlp for URLs) — both external, not bundled — so this is a
// small form: pick a file or URL, choose the transcription model, go. Import
// runs in the background (import_media returns as soon as the meeting row
// exists), so this closes immediately and the meetings list shows progress.
import { api, errorMessage, type ModelInfo } from "../api";
import { open } from "@tauri-apps/plugin-dialog";
import { onMount } from "svelte";
import { trapFocus } from "../actions/trapFocus";
import { t } from "../i18n/index.svelte";
import { X, FileUp, Link as LinkIcon } from "@lucide/svelte";
import { X, FileUp, Download } from "@lucide/svelte";
let { onClose, onImported }: { onClose: () => void; onImported: (id: string) => void } = $props();
// External download pages for the two tools this feature shells out to.
const FFMPEG_URL = "https://github.com/BtbN/FFmpeg-Builds/releases/latest";
const YTDLP_URL = "https://github.com/yt-dlp/yt-dlp/releases/latest";
// `source` is either a local file path (set via Browse) or a URL (typed).
let source = $state("");
let title = $state("");
let model = $state("");
let models = $state<ModelInfo[]>([]);
let busy = $state(false);
let error = $state<string | null>(null);
// Only installed whisper models are selectable; default to the active one so
// the pick matches the user's Settings default unless they change it here.
onMount(async () => {
try {
const all = await api.listModels();
models = all.filter((m) => m.installed);
model = models.find((m) => m.active)?.id ?? models[0]?.id ?? "";
} catch {
models = [];
}
});
async function browse() {
const path = await open({
multiple: false,
@@ -51,7 +72,7 @@
busy = true;
error = null;
try {
const id = await api.importMedia(source.trim(), title.trim() || undefined);
const id = await api.importMedia(source.trim(), title.trim() || undefined, model || undefined);
onImported(id);
onClose();
} catch (e) {
@@ -101,17 +122,32 @@
</div>
</label>
<label class="wide">
<div class="grid">
<label>
{t("import.model_label")}
<select bind:value={model} disabled={busy || models.length === 0}>
{#each models as m (m.id)}
<option value={m.id}>{m.label}</option>
{/each}
</select>
<span class="hint">{t("import.model_hint")}</span>
</label>
<label>
{t("import.title_label")} <em>({t("import.optional")})</em>
<input bind:value={title} placeholder={t("import.title_placeholder")} disabled={busy} />
</label>
</div>
<p class="muted small">
<LinkIcon size={12} aria-hidden="true" />
{t("import.requires_1")} <code>ffmpeg</code>
{t("import.requires_2")} <code>yt-dlp</code>
{t("import.requires_3")}
</p>
<div class="tools">
<span class="muted small">{t("import.requires")}</span>
<button class="tool" type="button" onclick={() => api.openUrl(FFMPEG_URL)}>
<Download size={12} aria-hidden="true" /> ffmpeg
</button>
<button class="tool" type="button" onclick={() => api.openUrl(YTDLP_URL)}>
<Download size={12} aria-hidden="true" /> yt-dlp
</button>
</div>
{#if error}
<p class="error">{error}</p>
@@ -121,6 +157,7 @@
<button class="primary" onclick={doImport} disabled={!source.trim() || busy}>
{busy ? t("import.importing") : t("import.import")}
</button>
<span class="muted small note">{t("import.background_note")}</span>
<button class="link" onclick={onClose} disabled={busy}>{t("import.cancel")}</button>
</div>
</div>
@@ -143,7 +180,7 @@
border: 1px solid var(--border);
border-radius: var(--radius-lg);
padding: 1.25rem;
width: min(520px, 100%);
width: min(540px, 100%);
max-height: 90vh;
overflow: auto;
box-shadow: 0 12px 40px rgba(0, 0, 0, 0.3);
@@ -179,7 +216,8 @@
font-weight: 400;
color: var(--muted);
}
input {
input,
select {
width: 100%;
box-sizing: border-box;
padding: 0.4rem 0.55rem;
@@ -189,10 +227,18 @@
color: var(--fg);
font: inherit;
}
input:focus-visible {
input:focus-visible,
select:focus-visible {
border-color: var(--accent);
outline: none;
}
.hint {
display: block;
margin-top: 0.25rem;
font-size: 0.75rem;
font-weight: 400;
color: var(--muted);
}
.row {
display: flex;
gap: 0.4rem;
@@ -213,18 +259,49 @@
padding: 0.4rem 0.6rem;
cursor: pointer;
}
/* Model + title side by side on wide panels, stacked when cramped. */
.grid {
display: grid;
grid-template-columns: 1fr 1fr;
gap: 0 0.75rem;
}
@media (max-width: 460px) {
.grid {
grid-template-columns: 1fr;
}
}
.tools {
display: flex;
align-items: center;
flex-wrap: wrap;
gap: 0.4rem;
margin-top: 0.9rem;
}
.tool {
display: inline-flex;
align-items: center;
gap: 0.25rem;
background: var(--bg-hover, transparent);
color: var(--accent);
border: 1px solid var(--border);
border-radius: var(--radius-sm);
padding: 0.2rem 0.5rem;
font-size: 0.78rem;
cursor: pointer;
}
.tool:hover {
border-color: var(--accent);
}
.muted {
color: var(--muted);
}
.small {
font-size: 0.8rem;
display: flex;
align-items: center;
gap: 0.3rem;
}
.error {
color: var(--danger, #d33);
font-size: 0.85rem;
margin-top: 0.75rem;
}
.actions {
display: flex;
@@ -232,6 +309,10 @@
gap: 0.6rem;
margin-top: 1rem;
}
.actions .note {
flex: 1;
line-height: 1.2;
}
.actions .primary {
background: var(--accent);
color: var(--accent-fg, #fff);
+214
View File
@@ -0,0 +1,214 @@
<script lang="ts">
// Domino's-pizza-tracker-style progress for a background media import: four
// ordered steps, the running one pulses, finished ones show how long they
// took. Fed by the `imports` store (import://progress events). Renders nothing
// until the first tick arrives. Design per ui-ux-pro-max: color is never the
// only signal (icon + label + time), tabular figures for the timers, and the
// pulse is dropped under prefers-reduced-motion.
import { imports } from "../stores/imports.svelte";
import { t } from "../i18n/index.svelte";
import { AudioLines, Captions, Users, FileCheck2, Check, X } from "@lucide/svelte";
let { meetingId }: { meetingId: string } = $props();
const run = $derived(imports.get(meetingId));
const ICONS = {
prepare: AudioLines,
transcribe: Captions,
diarize: Users,
finalize: FileCheck2,
} as const;
// ms → compact, human duration for a finished step ("820 ms", "4.3s", "2m 05s").
function fmtDur(ms: number | null): string {
if (ms == null) return "";
if (ms < 1000) return `${ms} ms`;
const s = ms / 1000;
if (s < 60) return `${s.toFixed(1)}s`;
const m = Math.floor(s / 60);
const rem = Math.round(s % 60);
return `${m}m ${String(rem).padStart(2, "0")}s`;
}
</script>
{#if run}
<section class="tracker" aria-label={t("import.tracker.label")}>
<header>
{#if run.error}
<span class="head err">{t("import.tracker.failed")}</span>
{:else if run.done}
<span class="head ok">{t("import.tracker.done")}</span>
{:else}
<span class="head">{t("import.tracker.running")}</span>
{/if}
</header>
<ol class="steps" aria-live="polite">
{#each run.phases as p (p.phase)}
{@const Icon = ICONS[p.phase]}
<li class="step {p.state}">
<div class="node">
{#if p.state === "done"}
<Check size={18} aria-hidden="true" />
{:else if p.state === "error"}
<X size={18} aria-hidden="true" />
{:else}
<Icon size={18} aria-hidden="true" />
{/if}
</div>
<div class="meta">
<span class="name">{t(`import.phase.${p.phase}`)}</span>
<span class="time">
{#if p.state === "done"}{fmtDur(p.elapsedMs)}
{:else if p.state === "active"}{t("import.tracker.working")}
{:else if p.state === "error"}{t("import.tracker.stopped")}
{/if}
</span>
</div>
</li>
{/each}
</ol>
{#if run.error}
<p class="msg">{run.error}</p>
{/if}
</section>
{/if}
<style>
.tracker {
border: 1px solid var(--border);
border-radius: var(--radius-sm);
background: var(--panel, var(--bg-elevated));
padding: 0.85rem 1rem 1rem;
}
header {
margin-bottom: 0.9rem;
}
.head {
font-size: 0.85rem;
font-weight: 600;
color: var(--fg);
}
.head.ok {
color: var(--success);
}
.head.err {
color: var(--danger);
}
.steps {
display: flex;
list-style: none;
margin: 0;
padding: 0;
}
.step {
flex: 1;
position: relative;
text-align: center;
min-width: 0;
}
/* Connector from the previous node's center to this one's (each step is the
same width, so -50%→+50% spans center to center), sitting behind the node. */
.step::before {
content: "";
position: absolute;
top: 17px;
left: -50%;
width: 100%;
height: 2px;
background: var(--border);
z-index: 0;
}
.step:first-child::before {
display: none;
}
.step.done::before,
.step.active::before,
.step.error::before {
background: var(--accent);
}
.node {
position: relative;
z-index: 1;
width: 36px;
height: 36px;
margin: 0 auto 0.45rem;
display: grid;
place-items: center;
border-radius: var(--radius-full);
border: 2px solid var(--border);
background: var(--bg);
color: var(--muted);
}
.step.active .node {
border-color: var(--accent);
background: var(--accent-soft, transparent);
color: var(--accent);
animation: pulse 1.4s ease-out infinite;
}
.step.done .node {
border-color: var(--success);
background: var(--success);
color: #fff;
}
.step.error .node {
border-color: var(--danger);
background: var(--danger);
color: #fff;
}
.meta {
display: flex;
flex-direction: column;
gap: 0.1rem;
padding: 0 0.2rem;
}
.name {
font-size: 0.78rem;
font-weight: 500;
color: var(--muted);
line-height: 1.2;
}
.step.active .name,
.step.done .name {
color: var(--fg);
}
.time {
font-size: 0.72rem;
color: var(--muted);
font-variant-numeric: tabular-nums;
min-height: 1em;
}
.step.active .time {
color: var(--accent);
}
.msg {
margin: 0.85rem 0 0;
font-size: 0.8rem;
color: var(--danger);
word-break: break-word;
}
@keyframes pulse {
0% {
box-shadow: 0 0 0 0 color-mix(in srgb, var(--accent) 45%, transparent);
}
70% {
box-shadow: 0 0 0 8px color-mix(in srgb, var(--accent) 0%, transparent);
}
100% {
box-shadow: 0 0 0 0 color-mix(in srgb, var(--accent) 0%, transparent);
}
}
@media (prefers-reduced-motion: reduce) {
.step.active .node {
animation: none;
box-shadow: 0 0 0 3px var(--accent-soft, transparent);
}
}
</style>
+54 -5
View File
@@ -24,6 +24,25 @@
"settings.transcription.switch_multilingual": "Switch to a multilingual model above to choose a language.",
"settings.hardware.title": "Hardware",
"settings.hardware.refresh": "Refresh",
"settings.hardware.test_output": "Test",
"settings.hardware.test_mic": "Test",
"settings.hardware.testing": "Listening…",
"settings.hardware.play_tone": "Play tone",
"settings.hardware.stress_title": "Quick stress test",
"settings.hardware.stress_hint": "Benchmarks your installed models on each available backend and recommends the most accurate one that still keeps up with live speech. Takes a moment.",
"settings.hardware.stress_run": "Run stress test",
"settings.hardware.stress_running": "Running…",
"settings.hardware.stress_progress": "Benchmarking {pair}…",
"settings.hardware.stress_recommend": "Recommended: {backend} + {model}",
"settings.hardware.stress_apply": "Apply",
"settings.hardware.stress_none": "No installed model keeps up with live speech on this hardware — try a smaller model.",
"settings.hardware.stress_backend": "Backend",
"settings.hardware.stress_model": "Model",
"settings.hardware.stress_rtf": "Speed (×real-time)",
"settings.hardware.stress_realtime": "Live?",
"settings.hardware.stress_yes": "Yes",
"settings.hardware.stress_no": "No",
"settings.hardware.active_backend": "Active backend",
"settings.hardware.model_meta": "· model {size}",
"settings.hardware.preferred_backend": "Preferred backend",
@@ -278,6 +297,7 @@
"settings.privacy.locked_word": "locked",
"settings.privacy.vault_locked_2": ". Unlock to read encrypted meetings.",
"settings.privacy.password": "Password",
"settings.privacy.show_password": "Show password",
"settings.privacy.unlock": "Unlock",
"settings.privacy.vault_unlocked_1": "Vault is ",
"settings.privacy.unlocked_word": "unlocked",
@@ -320,12 +340,23 @@
"import.optional": "optional",
"import.title_placeholder": "Defaults to the file name",
"import.filter_av": "Audio / video",
"import.requires_1": "Requires",
"import.requires_2": "installed and on your PATH (plus",
"import.requires_3": "for URLs). WhispAssist doesn't bundle them.",
"import.importing": "Importing… this can take a while",
"import.model_label": "Transcription model",
"import.model_hint": "Recorded with the meeting so you can see how it was transcribed.",
"import.requires": "Needs these on your PATH (not bundled):",
"import.background_note": "Runs in the background — track it in the list.",
"import.importing": "Starting…",
"import.import": "Import",
"import.cancel": "Cancel",
"import.tracker.label": "Import progress",
"import.tracker.running": "Importing…",
"import.tracker.done": "Import complete",
"import.tracker.failed": "Import failed",
"import.tracker.working": "working…",
"import.tracker.stopped": "stopped",
"import.phase.prepare": "Transcode",
"import.phase.transcribe": "Transcribe",
"import.phase.diarize": "Identify speakers",
"import.phase.finalize": "Finalize",
"tagchip.filter": "Filter meetings tagged \"{tag}\"",
"tagchip.remove": "Remove tag {tag}",
@@ -345,6 +376,10 @@
"app.cancel": "Cancel",
"app.cancel_title": "Discard this recording and delete it",
"app.recording": "Recording…",
"app.mute": "Mute microphone",
"app.unmute": "Unmute microphone",
"app.mute_title": "Mute microphone (M)",
"app.unmute_title": "Unmute microphone (M)",
"app.backend_title": "Active transcription backend",
"app.retention_title": "Save audio as .wav for this meeting",
"app.saving": "saving",
@@ -397,6 +432,7 @@
"transcript.heading": "Transcript",
"transcript.title_aria": "Meeting title",
"transcript.transcribed_with": "Transcribed with",
"transcript.lang_title": "Transcription language",
"transcript.lang_auto": "auto-detecting…",
"transcript.show": "Show transcript",
@@ -428,9 +464,18 @@
"notes.italic": "Italic",
"notes.h1": "Heading 1",
"notes.h2": "Heading 2",
"notes.h3": "Heading 3",
"notes.bullet": "Bullet list",
"notes.numbered": "Numbered list",
"notes.quote": "Quote",
"notes.divider": "Divider",
"notes.checkbox_title": "Checkbox",
"notes.checkbox_aria": "Checkbox list item",
"notes.enhance": "Enhance",
"notes.enhancing": "Enhancing…",
"notes.enhance_title": "Expand these notes into structured Markdown using the transcript (AI)",
"notes.enhanced_note": "Notes enhanced from the transcript.",
"notes.undo_enhance": "Undo",
"notes.edit_raw": "Edit the raw markdown",
"notes.render": "Render the markdown",
"notes.editor": "Editor",
@@ -527,5 +572,9 @@
"settings.recording.consent_ack": "acknowledged",
"settings.recording.consent_not": "not yet acknowledged",
"settings.recording.auto_label": "Auto-start recording when a calendar event begins",
"settings.recording.auto_hint": "Only while WhispAssist is open. When an imported calendar event's start time arrives, a recording begins automatically (using your default retention setting above). Nothing runs in the background — the timer is armed only while the app is running. Import events under Settings → Calendar."
"settings.recording.auto_hint": "Only while WhispAssist is open. When an imported calendar event's start time arrives, a recording begins automatically (using your default retention setting above). Nothing runs in the background — the timer is armed only while the app is running. Import events under Settings → Calendar.",
"settings.recording.autostart_label": "Launch WhispAssist at login",
"settings.recording.autostart_hint": "Starts WhispAssist automatically when you sign in to Windows. Off by default; installs a per-user startup entry (no admin required) and does not begin recording on its own.",
"settings.recording.close_tray_label": "Close to system tray",
"settings.recording.close_tray_hint": "Closing the window keeps WhispAssist running in the background instead of quitting. Reopen it from the tray icon; use the tray's Quit to exit fully. On by default."
}
+63
View File
@@ -0,0 +1,63 @@
// Live per-meeting progress of background media imports (feeds ImportTracker).
// Fed entirely by `import://progress` events emitted by `import_media`; kept in
// memory only (the meeting's `status` badge is the persistent story after a
// restart). See commands.rs `run_import_pipeline`.
import { events, type ImportPhase, type ImportProgress, type MeetingId } from "../api";
export type PhaseState = "pending" | "active" | "done" | "error";
// The four phases in the order the backend runs (and the tracker renders) them.
export const IMPORT_PHASES: ImportPhase[] = ["prepare", "transcribe", "diarize", "finalize"];
export interface PhaseInfo {
phase: ImportPhase;
state: PhaseState;
elapsedMs: number | null;
}
export interface ImportRun {
meetingId: MeetingId;
phases: PhaseInfo[];
error: string | null;
done: boolean;
}
function freshRun(meetingId: MeetingId): ImportRun {
return {
meetingId,
phases: IMPORT_PHASES.map((phase) => ({ phase, state: "pending", elapsedMs: null })),
error: null,
done: false,
};
}
class ImportsStore {
runs = $state<Record<MeetingId, ImportRun>>({});
get(meetingId: MeetingId): ImportRun | undefined {
return this.runs[meetingId];
}
async init() {
await events.onImportProgress((p) => this.apply(p));
}
private apply(p: ImportProgress) {
// Re-read through the record after inserting so we mutate the $state proxy,
// not the raw object (Svelte 5 deep reactivity only tracks the proxy).
if (!this.runs[p.meetingId]) this.runs[p.meetingId] = freshRun(p.meetingId);
const run = this.runs[p.meetingId];
const info = run.phases.find((x) => x.phase === p.phase);
if (!info) return;
info.state = p.state;
if (p.state === "done") info.elapsedMs = p.elapsedMs;
if (p.state === "error") {
run.error = p.error;
run.done = true;
}
if (p.phase === "finalize" && p.state === "done") run.done = true;
}
}
export const imports = new ImportsStore();
+34 -2
View File
@@ -1,7 +1,7 @@
// Recording state store (Svelte 5 runes-friendly via a small class).
// Subscribes to recording/transcript events and exposes reactive state.
import { api, events, type TranscriptSegment, type MeetingId } from "../api";
import { api, events, type TranscriptSegment, type SpeakerInfo, type MeetingId } from "../api";
import { settings } from "./settings.svelte";
import { SvelteMap } from "svelte/reactivity";
@@ -10,6 +10,9 @@ class RecordingStore {
state = $state<"idle" | "recording" | "paused">("idle");
elapsedMs = $state(0);
segments = $state<TranscriptSegment[]>([]);
/** Live speaker list from provisional diarization (FR-SPK): resolves labels
* to display names ("You" for the mic speaker) for the live transcript. */
speakers = $state<SpeakerInfo[]>([]);
/** Whether audio is being retained as .wav for the in-flight meeting (ADR-0009). */
retention = $state(false);
/** Live system/loopback level for the waveform/meter (FR-CAP-5); 0 when not recording. */
@@ -19,6 +22,9 @@ class RecordingStore {
* (FR-CAP-7); stays 0 when the mic is disabled or not recording. */
levelRmsMic = $state(0);
levelPeakMic = $state(0);
/** Mic muted for the in-flight meeting (FR-CAP-7): mic channel goes silent
* while loopback keeps recording. Toggled by the "M" key / mute button. */
micMuted = $state(false);
/** Set while a capture-device reconnect is in progress; cleared on recovery (FR-CAP-6). */
deviceNotice = $state<string | null>(null);
/** Live notes redesign: freeform text typed in the Notes pane while recording. */
@@ -47,11 +53,16 @@ class RecordingStore {
this.retention = p.record;
});
await events.onSegment(({ segment }) => {
// Replace an interim segment with the same id, else append.
// Replace an interim segment with the same id, else append. Live
// diarization also re-emits committed segments here with a refined
// speaker (FR-SPK) — same replace-by-id path applies.
const i = this.segments.findIndex((s) => s.id === segment.id);
if (i >= 0) this.segments[i] = segment;
else this.segments.push(segment);
});
await events.onDiarizationUpdated(({ speakers }) => {
this.speakers = speakers;
});
await events.onLevel(({ rms, peak, mic }) => {
if (mic) {
this.levelRmsMic = rms;
@@ -64,12 +75,30 @@ class RecordingStore {
await events.onDeviceChanged(({ recovered, message }) => {
this.deviceNotice = recovered ? null : message;
});
// Keep mute state in sync even if it was toggled elsewhere (e.g. a future
// tray control), not just from this store's toggleMute().
await events.onMicMuted(({ muted }) => {
this.micMuted = muted;
});
}
/** Toggle mic mute for the active recording (FR-CAP-7); no-op if not
* recording. Optimistically flips, then reconciles with the backend result. */
async toggleMute() {
if (!this.meetingId || this.state === "idle") return;
try {
this.micMuted = await api.toggleMicrophoneMute(this.meetingId);
} catch {
// Mic off for this meeting (or capture gone) — nothing to mute.
}
}
async start(title?: string, record = false, templateId?: string, calendarEventId?: string) {
this.segments = [];
this.speakers = [];
this.retention = record;
this.deviceNotice = null;
this.micMuted = false;
this.notesText = "";
this.segmentNotes.clear();
// T8.7/FR-TRX-4: whatever language is currently configured in Settings
@@ -89,6 +118,7 @@ class RecordingStore {
this.levelPeak = 0;
this.levelRmsMic = 0;
this.levelPeakMic = 0;
this.micMuted = false;
this.deviceNotice = null;
}
@@ -101,11 +131,13 @@ class RecordingStore {
if (this.meetingId) await api.cancelRecording(this.meetingId);
this.meetingId = null;
this.segments = [];
this.speakers = [];
this.state = "idle";
this.levelRms = 0;
this.levelPeak = 0;
this.levelRmsMic = 0;
this.levelPeakMic = 0;
this.micMuted = false;
this.deviceNotice = null;
this.notesText = "";
this.segmentNotes.clear();
+14
View File
@@ -48,6 +48,8 @@ const DEFAULT_SETTINGS: AppSettings = {
audio_output_device: null, // system default render device (FR-CAP-1)
microphone_enabled: true, // capture the user's mic into the transcript (FR-CAP-7)
audio_input_device: null, // system default capture device
auto_start: false, // launch at login — opt-in, off by default (NFR-RES-4)
close_to_tray: true, // closing the window hides to tray; on by default
};
class SettingsStore {
@@ -383,6 +385,18 @@ class SettingsStore {
return this.patch({ default_record: on });
}
/** Launch-at-login toggle (NFR-RES-4). Goes through its own command (not
* patch) since the backend also writes the per-user OS Run entry; that
* command persists auto_start itself, so we just mirror it locally. */
async setAutoStart(on: boolean) {
this.settings = { ...this.settings, auto_start: on };
try {
await api.setAutoStart(on);
} catch {
this.backendStub = true;
}
}
setRetentionPolicy(maxAgeDays: number | null, maxSizeGb: number | null) {
return this.patch({ retention_max_age_days: maxAgeDays, retention_max_size_gb: maxSizeGb });
}
+396 -24
View File
@@ -9,9 +9,16 @@
import { t, i18n, LOCALES } from "../i18n/index.svelte";
import ConsentNotice from "../components/ConsentNotice.svelte";
import HostedAiBanner from "../components/HostedAiBanner.svelte";
import LevelMeter from "../components/LevelMeter.svelte";
import { open } from "@tauri-apps/plugin-dialog";
import { api, errorMessage, events } from "../api";
import type { BackendId, SyncKind, SyncTargetConfig, SyncTargetInfo } from "../api";
import type {
BackendId,
StressTestResult,
SyncKind,
SyncTargetConfig,
SyncTargetInfo,
} from "../api";
import { trapFocus } from "../actions/trapFocus";
import {
X,
@@ -23,7 +30,11 @@
CalendarDays,
UploadCloud,
ShieldCheck,
Lock,
LockOpen,
Sparkles,
Volume2,
Zap,
RefreshCw,
ChevronRight,
RotateCcw,
@@ -381,11 +392,98 @@
let showConsent = $state(false);
let testResult = $state<{ ok: boolean; message: string } | null>(null);
// ---- Audio device test (live level meter) ----
let monitorKind = $state<"input" | "loopback" | null>(null);
let monitorRms = $state(0);
let monitorPeak = $state(0);
let monitorUnlisten: (() => void) | null = null;
function stopMonitor() {
monitorUnlisten?.();
monitorUnlisten = null;
monitorKind = null;
monitorRms = 0;
monitorPeak = 0;
}
async function testDevice(kind: "input" | "loopback") {
if (monitorKind) return;
monitorKind = kind;
monitorRms = 0;
monitorPeak = 0;
monitorUnlisten = await events.onDeviceLevel((p) => {
if (p.kind !== kind) return;
if (p.done) {
stopMonitor();
return;
}
monitorRms = p.rms ?? 0;
monitorPeak = p.peak ?? 0;
});
const deviceId =
kind === "input"
? (settings.settings.audio_input_device ?? null)
: (settings.settings.audio_output_device ?? null);
try {
await api.monitorAudioLevel(kind, deviceId, 6000);
} catch (e) {
testResult = { ok: false, message: errorMessage(e) };
} finally {
stopMonitor();
}
}
// A 440Hz beep to the default output so the user can confirm speakers work.
function playTone() {
try {
const ctx = new AudioContext();
const osc = ctx.createOscillator();
const gain = ctx.createGain();
osc.frequency.value = 440;
gain.gain.value = 0.15;
osc.connect(gain).connect(ctx.destination);
osc.start();
osc.stop(ctx.currentTime + 0.5);
osc.onended = () => ctx.close();
} catch {
/* no Web Audio available */
}
}
// ---- Quick hardware stress test ----
let stressRunning = $state(false);
let stressProgress = $state<string | null>(null);
let stressResult = $state<StressTestResult | null>(null);
let stressError = $state<string | null>(null);
async function runStressTest() {
if (stressRunning) return;
stressRunning = true;
stressError = null;
stressResult = null;
const un = await events.onStressProgress((p) => {
stressProgress = `${p.backend} · ${p.model}`;
});
try {
stressResult = await api.stressTestHardware();
} catch (e) {
stressError = errorMessage(e);
} finally {
un();
stressProgress = null;
stressRunning = false;
}
}
async function applyRecommendation() {
const r = stressResult?.recommended;
if (!r) return;
await settings.setPreferredBackend(r.backend as BackendId | "auto");
await settings.patch({ whisper_model: r.model });
}
// ---- At-rest encryption vault (T8.8, FR-SEC-3) ----
let vault = $state<{ enabled: boolean; unlocked: boolean } | null>(null);
let vaultPw = $state("");
let vaultPw2 = $state("");
let vaultMsg = $state<string | null>(null);
let vaultMsgError = $state(false);
let showVaultPw = $state(false);
async function loadVault() {
try {
vault = await api.vaultStatus();
@@ -394,42 +492,46 @@
}
}
onMount(loadVault);
function setVaultMsg(msg: string | null, isError = false) {
vaultMsg = msg;
vaultMsgError = isError;
}
async function enableVault() {
vaultMsg = null;
setVaultMsg(null);
try {
await api.enableVault(vaultPw);
vaultPw = "";
vaultMsg = "Vault enabled and unlocked.";
setVaultMsg("Vault enabled and unlocked.");
await loadVault();
} catch (e) {
vaultMsg = errorMessage(e);
setVaultMsg(errorMessage(e), true);
}
}
async function unlockVault() {
vaultMsg = null;
setVaultMsg(null);
try {
await api.unlockVault(vaultPw);
vaultPw = "";
vaultMsg = "Unlocked.";
setVaultMsg("Unlocked.");
await loadVault();
} catch (e) {
vaultMsg = errorMessage(e);
setVaultMsg(errorMessage(e), true);
}
}
async function lockVault() {
await api.lockVault();
vaultMsg = "Locked.";
setVaultMsg("Locked.");
await loadVault();
}
async function changeVaultPassword() {
vaultMsg = null;
setVaultMsg(null);
try {
await api.changeVaultPassword(vaultPw, vaultPw2);
vaultPw = "";
vaultPw2 = "";
vaultMsg = "Password changed.";
setVaultMsg("Password changed.");
} catch (e) {
vaultMsg = errorMessage(e);
setVaultMsg(errorMessage(e), true);
}
}
@@ -754,13 +856,40 @@
</label>
<p class="muted">{t("settings.recording.auto_hint")}</p>
<label class="row">
<input
type="checkbox"
checked={settings.settings.auto_start}
onchange={(e) => settings.setAutoStart((e.target as HTMLInputElement).checked)}
/>
<span>{t("settings.recording.autostart_label")}</span>
</label>
<p class="muted">{t("settings.recording.autostart_hint")}</p>
<label class="row">
<input
type="checkbox"
checked={settings.settings.close_to_tray}
onchange={(e) =>
settings.patch({ close_to_tray: (e.target as HTMLInputElement).checked })}
/>
<span>{t("settings.recording.close_tray_label")}</span>
</label>
<p class="muted">{t("settings.recording.close_tray_hint")}</p>
{#if showConsent}
<ConsentNotice onAccept={acceptConsent} onCancel={() => (showConsent = false)} />
{/if}
</section>
{:else if section === "hardware"}
<section>
<div class="actions">
<h3>{t("settings.hardware.title")}</h3>
<button class="ghost" onclick={() => settings.loadHardware()}>
<RefreshCw size={14} aria-hidden="true" />
{t("settings.hardware.refresh")}
</button>
</div>
{#if settings.hardware}
<div class="row">
{t("settings.hardware.active_backend")}<code>{settings.hardware.active}</code>
@@ -804,6 +933,20 @@
</select>
</label>
<p class="muted">{t("settings.hardware.recording_device_hint")}</p>
<div class="device-test">
<button onclick={() => testDevice("loopback")} disabled={monitorKind !== null}>
<Volume2 size={13} aria-hidden="true" />
{monitorKind === "loopback"
? t("settings.hardware.testing")
: t("settings.hardware.test_output")}
</button>
<button onclick={playTone} disabled={monitorKind !== null}>
{t("settings.hardware.play_tone")}
</button>
{#if monitorKind === "loopback"}
<LevelMeter rms={monitorRms} peak={monitorPeak} />
{/if}
</div>
<label
>{t("settings.hardware.microphone")}
@@ -825,6 +968,75 @@
</select>
</label>
<p class="muted">{t("settings.hardware.mic_hint")}</p>
<div class="device-test">
<button
onclick={() => testDevice("input")}
disabled={monitorKind !== null || !settings.settings.microphone_enabled}
>
<Mic size={13} aria-hidden="true" />
{monitorKind === "input"
? t("settings.hardware.testing")
: t("settings.hardware.test_mic")}
</button>
{#if monitorKind === "input"}
<LevelMeter rms={monitorRms} peak={monitorPeak} />
{/if}
</div>
<h4>{t("settings.hardware.stress_title")}</h4>
<p class="muted">{t("settings.hardware.stress_hint")}</p>
<button onclick={runStressTest} disabled={stressRunning}>
<Zap size={13} aria-hidden="true" />
{stressRunning ? t("settings.hardware.stress_running") : t("settings.hardware.stress_run")}
</button>
{#if stressProgress}
<p class="muted">{t("settings.hardware.stress_progress", { pair: stressProgress })}</p>
{/if}
{#if stressError}<p class="muted err">{stressError}</p>{/if}
{#if stressResult}
{#if stressResult.recommended}
<div class="stress-rec">
<ShieldCheck size={14} aria-hidden="true" />
<span
>{t("settings.hardware.stress_recommend", {
backend: stressResult.recommended.backend,
model: stressResult.recommended.model,
})}</span
>
<button class="primary" onclick={applyRecommendation}
>{t("settings.hardware.stress_apply")}</button
>
</div>
{:else}
<p class="muted">{t("settings.hardware.stress_none")}</p>
{/if}
<table class="stress-table">
<thead>
<tr>
<th>{t("settings.hardware.stress_backend")}</th>
<th>{t("settings.hardware.stress_model")}</th>
<th>{t("settings.hardware.stress_rtf")}</th>
<th>{t("settings.hardware.stress_realtime")}</th>
</tr>
</thead>
<tbody>
{#each stressResult.results as r (r.backend + r.model)}
<tr>
<td>{r.backend}</td>
<td>{r.model}</td>
<td class="num">{r.rtf.toFixed(2)}×</td>
<td>
{#if r.realtime}
<Check size={13} aria-hidden="true" /> {t("settings.hardware.stress_yes")}
{:else}
{t("settings.hardware.stress_no")}
{/if}
</td>
</tr>
{/each}
</tbody>
</table>
{/if}
{#if settings.hardware.npu?.present}
{@const npu = settings.hardware.npu}
@@ -1918,34 +2130,81 @@
{/if}
{#if vault}
<div
class="vault-card"
class:locked={vault.enabled && !vault.unlocked}
class:unlocked={vault.enabled && vault.unlocked}
>
<div class="vault-head">
{#if !vault.enabled}
<ShieldCheck size={18} aria-hidden="true" />
{:else if !vault.unlocked}
<Lock size={18} aria-hidden="true" />
{:else}
<LockOpen size={18} aria-hidden="true" />
{/if}
<h4>{t("settings.privacy.vault_title")}</h4>
{#if vault.enabled}
<span class="badge" class:busy={!vault.unlocked}>
{vault.unlocked
? t("settings.privacy.unlocked_word")
: t("settings.privacy.locked_word")}
</span>
{/if}
</div>
{#if !vault.enabled}
<p class="muted">{t("settings.privacy.vault_intro")}</p>
<div class="grid">
<label class="wide"
>{t("settings.privacy.vault_password")}<input
type="password"
<div class="pw-row">
<input
type={showVaultPw ? "text" : "password"}
bind:value={vaultPw}
/></label
placeholder={t("settings.privacy.vault_password")}
aria-label={t("settings.privacy.vault_password")}
/>
<button
type="button"
class="icon pw-toggle"
onclick={() => (showVaultPw = !showVaultPw)}
aria-label={t("settings.privacy.show_password")}
title={t("settings.privacy.show_password")}
>
{#if showVaultPw}<EyeOff size={14} aria-hidden="true" />{:else}<Eye
size={14}
aria-hidden="true"
/>{/if}
</button>
</div>
<button class="primary" onclick={enableVault} disabled={vaultPw.length < 8}
>{t("settings.privacy.enable_vault")}</button
>
<p class="muted">{t("settings.privacy.vault_pw_hint")}</p>
<p class="muted hint">{t("settings.privacy.vault_pw_hint")}</p>
{:else if !vault.unlocked}
<p class="muted">
{t("settings.privacy.vault_locked_1")}<strong
>{t("settings.privacy.locked_word")}</strong
>{t("settings.privacy.vault_locked_2")}
</p>
<div class="grid">
<label class="wide"
>{t("settings.privacy.password")}<input
type="password"
<div class="pw-row">
<input
type={showVaultPw ? "text" : "password"}
bind:value={vaultPw}
/></label
placeholder={t("settings.privacy.password")}
aria-label={t("settings.privacy.password")}
onkeydown={(e) => e.key === "Enter" && vaultPw && unlockVault()}
/>
<button
type="button"
class="icon pw-toggle"
onclick={() => (showVaultPw = !showVaultPw)}
aria-label={t("settings.privacy.show_password")}
title={t("settings.privacy.show_password")}
>
{#if showVaultPw}<EyeOff size={14} aria-hidden="true" />{:else}<Eye
size={14}
aria-hidden="true"
/>{/if}
</button>
</div>
<button class="primary" onclick={unlockVault} disabled={!vaultPw}
>{t("settings.privacy.unlock")}</button
@@ -1956,7 +2215,10 @@
>{t("settings.privacy.unlocked_word")}</strong
>{t("settings.privacy.vault_unlocked_2")}
</p>
<button onclick={lockVault}>{t("settings.privacy.lock_now")}</button>
<button onclick={lockVault}>
<Lock size={14} aria-hidden="true" />
{t("settings.privacy.lock_now")}
</button>
<details>
<summary>{t("settings.privacy.change_password")}</summary>
<div class="grid">
@@ -1978,7 +2240,8 @@
>
</details>
{/if}
{#if vaultMsg}<p class="muted">{vaultMsg}</p>{/if}
{#if vaultMsg}<p class="vault-msg" class:error={vaultMsgError}>{vaultMsg}</p>{/if}
</div>
{/if}
</section>
{:else if section === "language"}
@@ -2429,6 +2692,115 @@
color: var(--accent, #2563eb);
border-color: currentColor;
}
.vault-card {
margin-top: 0.6rem;
padding: 0.85rem 1rem;
border: 1px solid var(--border);
border-radius: var(--radius-md, 8px);
background: var(--bg-elevated);
display: flex;
flex-direction: column;
gap: 0.6rem;
}
.vault-card.locked {
border-color: color-mix(in srgb, var(--accent) 45%, var(--border));
}
.vault-card.unlocked {
border-color: color-mix(in srgb, var(--success, #16a34a) 45%, var(--border));
}
.vault-head {
display: flex;
align-items: center;
gap: 0.5rem;
}
.vault-head h4 {
margin: 0;
}
.vault-head :global(svg) {
color: var(--muted);
}
.vault-card.locked .vault-head :global(svg) {
color: var(--accent);
}
.vault-card.unlocked .vault-head :global(svg) {
color: var(--success, #16a34a);
}
.vault-head .badge {
margin-left: auto;
}
.pw-row {
display: flex;
align-items: center;
gap: 0.4rem;
max-width: 22rem;
}
.pw-row input {
flex: 1;
}
.pw-toggle {
flex: none;
}
.vault-card .hint {
margin: 0;
}
.vault-msg {
margin: 0;
font-size: 0.85rem;
color: var(--success, #16a34a);
}
.vault-msg.error {
color: var(--danger);
}
.device-test {
display: flex;
align-items: center;
gap: 0.5rem;
flex-wrap: wrap;
margin: 0.35rem 0 0.6rem;
}
.device-test :global(.meter) {
flex: 1;
min-width: 8rem;
}
.stress-rec {
display: flex;
align-items: center;
gap: 0.5rem;
margin: 0.6rem 0;
padding: 0.55rem 0.75rem;
border: 1px solid color-mix(in srgb, var(--accent) 40%, var(--border));
border-radius: var(--radius-md, 8px);
background: color-mix(in srgb, var(--accent) 10%, var(--bg));
}
.stress-rec :global(svg) {
color: var(--accent);
}
.stress-rec span {
flex: 1;
}
.stress-table {
width: 100%;
border-collapse: collapse;
margin-top: 0.5rem;
font-size: 0.82rem;
}
.stress-table th,
.stress-table td {
text-align: left;
padding: 0.3rem 0.5rem;
border-bottom: 1px solid var(--border);
}
.stress-table th {
color: var(--muted);
font-weight: 600;
}
.stress-table .num {
font-variant-numeric: tabular-nums;
text-align: right;
}
.err {
color: var(--danger);
}
.npu-package {
margin-top: 0.6rem;
padding: 0.6rem 0.75rem;
+35 -14
View File
@@ -64,7 +64,7 @@
// nonce so re-clicking the same segment still jumps). Reading seekNonce is
// what makes this effect re-run.
$effect(() => {
player.seekNonce;
void player.seekNonce;
const ms = player.seekMs;
if (ms == null || !audioEl) return;
audioEl.currentTime = ms / 1000;
@@ -770,9 +770,13 @@
{#if editableItems.length === 0}
<p class="muted">{t("summary.ai_empty")}</p>
{:else}
<!-- Two-row card per item: the action text owns the full first row (it
was unreadable when six controls shared one row in this narrow
pane); owner/due/reminder are a secondary meta row beneath it. -->
<ul class="action-items">
{#each editableItems as item, i (i)}
<li>
<div class="ai-main">
<input
type="checkbox"
bind:checked={item.confirmed}
@@ -785,6 +789,16 @@
placeholder={t("summary.ai_text_placeholder")}
aria-label={t("summary.ai_text_aria")}
/>
<button
class="ai-del"
onclick={() => removeActionItem(i)}
title={t("summary.ai_delete_title")}
aria-label={t("summary.ai_delete_aria")}
>
<X size={14} aria-hidden="true" />
</button>
</div>
<div class="ai-meta">
<input
class="ai-owner"
value={item.owner ?? ""}
@@ -803,14 +817,7 @@
<input type="checkbox" bind:checked={item.reminder_set} disabled={!item.due_at} />
<Bell size={14} aria-hidden="true" />
</label>
<button
class="ai-del"
onclick={() => removeActionItem(i)}
title={t("summary.ai_delete_title")}
aria-label={t("summary.ai_delete_aria")}
>
<X size={14} aria-hidden="true" />
</button>
</div>
</li>
{/each}
</ul>
@@ -1080,9 +1087,9 @@
}
ul.action-items li {
display: flex;
align-items: center;
flex-direction: column;
gap: 0.35rem;
padding: 0.25rem 0;
padding: 0.45rem 0;
border-bottom: 1px solid var(--border);
}
ul.action-items label {
@@ -1090,14 +1097,28 @@
align-items: center;
gap: 0.4rem;
}
.ai-main {
display: flex;
align-items: center;
gap: 0.35rem;
}
.ai-text {
flex: 1;
min-width: 0;
font-size: 0.82rem;
font-size: 0.85rem;
}
/* Meta row indented under the text (past the confirm checkbox), wrapping
rather than crushing its inputs when the pane is narrow. */
.ai-meta {
display: flex;
align-items: center;
flex-wrap: wrap;
gap: 0.35rem;
padding-left: 1.4rem;
}
.ai-owner {
flex: none;
width: 5rem;
flex: 1;
min-width: 5rem;
font-size: 0.75rem;
}
.ai-del {
+207 -6
View File
@@ -6,19 +6,27 @@
import { meetings } from "../stores/meetings.svelte";
import { settings } from "../stores/settings.svelte";
import { player } from "../stores/player.svelte";
import { api, type SpeakerInfo } from "../api";
import { api, errorMessage, type SpeakerInfo } from "../api";
import { t } from "../i18n/index.svelte";
import { renderMarkdown } from "../markdown";
import { save, open } from "@tauri-apps/plugin-dialog";
import { layout, clamp } from "../stores/layout.svelte";
import { imports } from "../stores/imports.svelte";
import Splitter from "../components/Splitter.svelte";
import ImportTracker from "../components/ImportTracker.svelte";
import {
Bold,
Italic,
Heading1,
Heading2,
Heading3,
List,
ListOrdered,
ListChecks,
Quote,
Minus,
Sparkles,
Undo2,
FileDown,
FileText,
FolderOutput,
@@ -66,6 +74,10 @@
let editorEl: HTMLTextAreaElement | undefined = $state();
let saveTimer: ReturnType<typeof setTimeout> | undefined;
let loadedForId: string | null = null;
// The server copy the buffer was last synced against — lets the effect below
// tell a server-side notes change (speaker rename, reprocess) apart from the
// user's own unsaved edits.
let lastServerNotes: string | null = null;
// Transcript/notes split (FR-UX-1): resizable (drag the Splitter) and
// each side independently hideable, shared across the finalized-meeting
@@ -103,14 +115,23 @@
selectedSegmentMs = null;
});
// Sync the editor buffer whenever a different meeting is selected.
// Sync the editor buffer whenever a different meeting is selected — and when
// the *server* copy of the same meeting's notes changes underneath us (a
// speaker rename rewrites notes.md's dialogue tags, reprocess regenerates it).
// A buffer with unsaved local edits is never clobbered: it only adopts the
// server copy when it still equals the last-synced one.
$effect(() => {
const m = meetings.selected;
if (m && m.id !== loadedForId) {
notesText = m.notes_markdown;
lastServerNotes = m.notes_markdown;
loadedForId = m.id;
} else if (m && m.id === loadedForId && m.notes_markdown !== lastServerNotes) {
if (notesText === lastServerNotes) notesText = m.notes_markdown;
lastServerNotes = m.notes_markdown;
} else if (!m) {
loadedForId = null;
lastServerNotes = null;
}
});
@@ -158,6 +179,70 @@
scheduleSave();
}
// Slash commands: typing "/todo" (etc.) at the start of a line and pressing
// Space/Enter swaps it for the matching Markdown prefix. Reuses the same
// line-prefix model as the toolbar buttons — no rich inline menu.
// ponytail: line-prefix slash only; add a picker popover if users ask.
const SLASH_COMMANDS: Record<string, string> = {
h1: "# ",
h2: "## ",
h3: "### ",
todo: "- [ ] ",
bullet: "- ",
num: "1. ",
quote: "> ",
divider: "---\n",
};
function handleNotesKeydown(e: KeyboardEvent) {
if (e.key !== "Enter" && e.key !== " ") return;
const el = editorEl;
if (!el) return;
const { selectionStart: s, value } = el;
const lineStart = value.lastIndexOf("\n", s - 1) + 1;
const match = /^\/(\w+)$/.exec(value.slice(lineStart, s));
if (!match) return;
const prefix = SLASH_COMMANDS[match[1].toLowerCase()];
if (prefix === undefined) return;
e.preventDefault();
const head = value.slice(0, lineStart) + prefix;
notesText = head + value.slice(s);
queueMicrotask(() => {
el.focus();
el.selectionStart = el.selectionEnd = head.length;
});
scheduleSave();
}
// AI-enhance (Granola-style): expand the user's rough notes into structured
// Markdown grounded in the transcript, via the configured LlmProvider (local
// by default, no new egress). Keeps a one-step Undo so we never silently lose
// what the user typed.
let enhancing = $state(false);
let enhanceError = $state<string | null>(null);
let notesBeforeEnhance = $state<string | null>(null);
async function enhanceNotes() {
const m = meetings.selected;
if (!m || enhancing) return;
enhancing = true;
enhanceError = null;
try {
const enhanced = await api.enhanceNotes(m.id, notesText);
notesBeforeEnhance = notesText;
notesText = enhanced;
scheduleSave();
} catch (e) {
enhanceError = errorMessage(e);
} finally {
enhancing = false;
}
}
function undoEnhance() {
if (notesBeforeEnhance === null) return;
notesText = notesBeforeEnhance;
notesBeforeEnhance = null;
scheduleSave();
}
async function exportMd() {
const m = meetings.selected;
if (!m) return;
@@ -247,6 +332,15 @@
reprocessing = true;
try {
await meetings.reprocess(m.id, reprocessModel, reprocessLanguage || undefined);
// Re-transcribe rebuilds notes.md server-side (merging saved manual notes),
// but the meeting stays selected (same id), so the buffer-sync $effect —
// which only fires on an id change — won't pick it up. Resync explicitly so
// the notes pane updates in place instead of only after a restart.
const updated = meetings.selected;
if (updated && updated.id === m.id) {
notesText = updated.notes_markdown;
loadedForId = updated.id;
}
} finally {
reprocessing = false;
}
@@ -262,14 +356,23 @@
onchange={onTitleChange}
aria-label={t("transcript.title_aria")}
/>
{#if m.status === "transcribing" || imports.get(m.id)}
<div class="import-strip"><ImportTracker meetingId={m.id} /></div>
{:else if m.model_used}
<p class="engine-meta" title={t("transcript.transcribed_with")}>
{t("transcript.transcribed_with")}
<strong>{m.model_used}</strong>{#if m.backend_used}
· {m.backend_used}{/if}
</p>
{/if}
<div
class="split"
style="grid-template-columns: {splitColumns()};"
bind:clientWidth={splitWidth}
>
<!-- svelte-ignore a11y_no_static_element_interactions -- wheel/touchmove
here only note "the user scrolled by hand" to pause playback
auto-scroll; the pane isn't an interactive control. -->
<!-- wheel/touchmove here only note "the user scrolled by hand" to pause
playback auto-scroll; the pane isn't an interactive control. -->
<!-- svelte-ignore a11y_no_static_element_interactions -->
<div
class="pane transcript"
class:collapsed={layout.transcriptCollapsed}
@@ -423,6 +526,43 @@
>
<ListChecks size={14} aria-hidden="true" />
</button>
<button
onclick={() => insertLinePrefix("### ")}
title={t("notes.h3")}
aria-label={t("notes.h3")}
>
<Heading3 size={14} aria-hidden="true" />
</button>
<button
onclick={() => insertLinePrefix("1. ")}
title={t("notes.numbered")}
aria-label={t("notes.numbered")}
>
<ListOrdered size={14} aria-hidden="true" />
</button>
<button
onclick={() => insertLinePrefix("> ")}
title={t("notes.quote")}
aria-label={t("notes.quote")}
>
<Quote size={14} aria-hidden="true" />
</button>
<button
onclick={() => insertLinePrefix("---\n")}
title={t("notes.divider")}
aria-label={t("notes.divider")}
>
<Minus size={14} aria-hidden="true" />
</button>
<button
class="enhance"
onclick={enhanceNotes}
disabled={enhancing}
title={t("notes.enhance_title")}
>
<Sparkles size={14} aria-hidden="true" class={enhancing ? "spin" : ""} />
{enhancing ? t("notes.enhancing") : t("notes.enhance")}
</button>
<button
class="toggle"
onclick={() => (notesPreview = !notesPreview)}
@@ -459,6 +599,17 @@
Obsidian
</button>
</div>
{#if enhanceError}
<p class="enhance-bar error" role="alert">{enhanceError}</p>
{:else if notesBeforeEnhance !== null}
<div class="enhance-bar">
<span>{t("notes.enhanced_note")}</span>
<button class="link" onclick={undoEnhance}>
<Undo2 size={13} aria-hidden="true" />
{t("notes.undo_enhance")}
</button>
</div>
{/if}
<div class="editor-preview">
{#if notesPreview}
<!-- eslint-disable-next-line svelte/no-at-html-tags -- sanitized via renderMarkdown() -->
@@ -468,6 +619,7 @@
bind:this={editorEl}
bind:value={notesText}
oninput={scheduleSave}
onkeydown={handleNotesKeydown}
placeholder={t("notes.placeholder")}
></textarea>
{/if}
@@ -524,7 +676,7 @@
onclick={() => (selectedSegmentMs = open ? null : s.start_ms)}
>
<span class="ts">{fmtTs(s.start_ms)}</span>
<strong>{speakerName(s.speaker)}:</strong>
<strong>{speakerName(s.speaker, recording.speakers)}:</strong>
{s.text}
{#if hasNote}
<span class="note-badge" title={t("transcript.has_note")}>📝</span>
@@ -610,6 +762,21 @@
background: var(--border);
outline: none;
}
.import-strip {
flex: none;
padding: 0 1rem 0.5rem;
}
.engine-meta {
flex: none;
margin: 0;
padding: 0 1rem 0.4rem;
font-size: 0.75rem;
color: var(--muted);
}
.engine-meta strong {
font-weight: 600;
color: var(--fg);
}
.pad {
padding: 1rem;
max-width: 760px;
@@ -842,6 +1009,40 @@
.toolbar .spacer {
flex: 1;
}
.toolbar .enhance {
color: var(--accent);
border-color: color-mix(in srgb, var(--accent) 40%, var(--border));
font-weight: 600;
}
.toolbar .enhance:hover:not(:disabled) {
background: color-mix(in srgb, var(--accent) 12%, var(--bg));
}
.toolbar .enhance:disabled {
opacity: 0.6;
cursor: default;
}
.enhance-bar {
display: flex;
align-items: center;
gap: 0.5rem;
margin-bottom: 0.5rem;
font-size: 0.8rem;
color: var(--muted);
}
.enhance-bar.error {
color: var(--danger);
}
.enhance-bar .link {
display: inline-flex;
align-items: center;
gap: 0.25rem;
background: none;
border: none;
color: var(--accent);
cursor: pointer;
font-size: 0.8rem;
padding: 0;
}
.editor-preview {
height: calc(100% - 2.5rem);