124 Commits
Author SHA1 Message Date
iamdoubz 87fb665e9e Merge pull request 'Feature chore bug 010' (#25) from feature_chore_bug_010 into main
Reviewed-on: #25
2026-07-15 20:42:24 -05:00
iamdoubz 470fe6a7e7 docs(readme): refresh to v0.7.3 — NPU+iGPU split, live 'You' diarization, mic mute, tray, background import, AI-enhance, libpst button; drop stale 'signed installer' claim 2026-07-15 20:39:29 -05:00
iamdoubz 403d652c6d style: cargo fmt sweep (reflow-only; verified rustfmt(HEAD) == worktree per file) 2026-07-15 20:36:48 -05:00
iamdoubz e8a6ddca71 docs(release): v0.7.3 release notes with SHA-256 checksums 2026-07-15 20:35:12 -05:00
iamdoubz c9643cd79b feat(calendar): libpst (readpst) download button in Settings, matching the Import dialog's tool buttons (FR-CAL-1, ADR-0008) 2026-07-15 20:28:19 -05:00
iamdoubz 29bb636cb5 feat(i18n): Settings calendar 'requires readpst' string 2026-07-15 20:28:18 -05:00
iamdoubz 89bd7c3b7e chore: bump version 0.7.3 2026-07-15 20:14:30 -05:00
iamdoubz 4a3f8de679 docs(adr): 0004 update — NPU path is OpenVINO; decoder split to iGPU with KV cache 2026-07-15 19:48:16 -05:00
iamdoubz 7f7de48ecc feat(npu): engine log names the decoder EP alongside the encoder EP 2026-07-15 19:48:15 -05:00
iamdoubz 402ffe4586 feat(npu): KV-cache greedy decode + decoder on the OpenVINO GPU EP (T3.4)
Merged-decoder (use_cache_branch/past_key_values) loop discovered from the
graph at load; step 1 runs the no-cache branch over the prompt, later steps
feed one token + cached KVs (decoder slots refreshed, encoder cross-attn KVs
kept from step 1). Decoder session lands on the Intel iGPU via the OpenVINO
GPU plugin (same staged runtime) with CPU-EP fallback and a
WA_ONNX_DECODER_DEVICE=cpu|gpu override; DirectML backends keep the CPU
decoder. Legacy decoder_model.onnx is removed once the merged graph loads.

Measured (Core Ultra 5 135U, base.en, 19s TTS clip, identical transcripts):
decoder on OpenVINO/GPU infer=1288ms vs CPU-EP infer=6098ms
2026-07-15 19:48:14 -05:00
iamdoubz 177f1f7d90 feat(npu): catalog the merged Whisper decoder (decoder_model_merged.onnx) for KV-cache decode (T3.4) 2026-07-15 19:35:16 -05:00
iamdoubz 21f3d76ed0 Merge pull request 'Feature chore bug 009' (#24) from feature_chore_bug_009 into main
Reviewed-on: #24
2026-07-15 18:13:11 -05:00
iamdoubz 19bd5815c0 chore: bump version 0.7.2 2026-07-15 18:09:07 -05:00
iamdoubz a552ddb1fa fix(ui): action items as two-row cards so the action text is always readable in the narrow pane
Text row (confirm + full-width text + delete) over a wrapping meta row
(owner, due date, reminder). Also void the seekNonce dependency read to
satisfy no-unused-expressions.
2026-07-15 17:41:38 -05:00
iamdoubz de25ac70e7 fix(notes): adopt server-side notes.md changes (speaker rename, reprocess) into the open editor buffer
Also untangles the a11y svelte-ignore comment from its prose so eslint's
no-unused-svelte-ignore stops parsing the explanation as ignore codes.
2026-07-15 17:41:37 -05:00
iamdoubz 28d9b2988a fix(spk): live split attribution + honest model reporting + notes rename rewrite (FR-SPK, T3.4)
- live diarization tick on split recordings now uses attribute_split (mic
  channel = 'You' from the first pass) instead of whole-mix clustering +
  voiceprint matching, which never reliably surfaced 'You' before stop
- attribute_split assigns via diarization::assign_split (channel-total
  overlap) so the user's speech can't be swallowed by a longer far-side span
- stop_recording overlays live user renames over the automatic name map
- rename_speaker/map_speaker_to_participant rewrite notes.md's baked-in
  **Name:** dialogue tags so the Notes pane follows the Speakers pane
- model_used records the ONNX model when the NPU/DirectML engine is routed
  (was reporting the unused ggml model id) + info log naming the encoder EP
2026-07-15 17:41:28 -05:00
iamdoubz f24adc3a59 fix(diarization): channel-total split attribution so mic speech can't be swallowed by a far span (FR-SPK) 2026-07-15 17:33:40 -05:00
iamdoubz 94425d456c docs(release): v0.7.1 release notes with SHA-256 checksums 2026-07-15 14:44:01 -05:00
iamdoubz 49cdbd6a54 chore: bump version 0.7.1 2026-07-15 14:38:31 -05:00
iamdoubz b4d72622da docs(data): close_to_tray setting in settings.json schema 2026-07-15 14:02:22 -05:00
iamdoubz c1ccdce497 docs(api): toggle_microphone_mute command + recording://mic event 2026-07-15 14:02:21 -05:00
iamdoubz a319dd9e97 feat(i18n): mute + close-to-tray strings 2026-07-15 14:01:43 -05:00
iamdoubz 6a01afbba6 feat(settings): default close_to_tray in settings store 2026-07-15 14:01:42 -05:00
iamdoubz 8c6a0ea25c feat(settings): close-to-tray toggle in Settings 2026-07-15 14:01:41 -05:00
iamdoubz 4a000f07a2 feat(app): M-key + mic mute button in recording controls 2026-07-15 14:01:40 -05:00
iamdoubz 0e2a3bb110 feat(recording): micMuted state + toggleMute synced to recording://mic 2026-07-15 13:59:04 -05:00
iamdoubz 95b3d34713 feat(api): toggleMicrophoneMute + recording://mic event + close_to_tray setting 2026-07-15 13:58:18 -05:00
iamdoubz 588354f488 fix(tray): remove duplicate trayIcon from config — one tray icon only 2026-07-15 13:57:38 -05:00
iamdoubz ee08f2bc88 feat(app): single tray with Show/Quit menu, close-to-tray, register mute cmd 2026-07-15 13:57:37 -05:00
iamdoubz a71c17965f feat(recording): toggle_microphone_mute command + recording://mic event 2026-07-15 13:57:36 -05:00
iamdoubz 3c73ec1cd4 feat(settings): close_to_tray setting (default on) 2026-07-15 13:57:35 -05:00
iamdoubz 8f805b8ef5 feat(audio): live mic mute — zero mic samples via CaptureHandle.set_muted (FR-CAP-7) 2026-07-15 13:57:33 -05:00
iamdoubz f83234bde1 docs(api): document import_media (model, background) + import://progress event 2026-07-15 13:46:50 -05:00
iamdoubz d4b1578478 feat(i18n): transcript.transcribed_with label 2026-07-15 13:46:01 -05:00
iamdoubz 185e4ec803 feat(import): show import tracker + 'transcribed with' model in meeting view 2026-07-15 13:46:00 -05:00
iamdoubz 30f4a4d3ec feat(import): init imports store for live progress tracking 2026-07-15 13:45:59 -05:00
iamdoubz 86c3587d19 feat(i18n): import model/tracker/phase strings; background import copy 2026-07-15 13:44:00 -05:00
iamdoubz d736b78ecb feat(import): model selector + ffmpeg/yt-dlp download links in add-meeting dialog 2026-07-15 13:43:26 -05:00
iamdoubz 98f6fcdc16 feat(import): ImportTracker Domino's-style 4-step progress component 2026-07-15 13:42:19 -05:00
iamdoubz 683838bef1 feat(import): imports store tracking per-meeting import phase progress 2026-07-15 13:41:08 -05:00
iamdoubz c63cabaf88 feat(import): api model param + import://progress event and types 2026-07-15 13:39:55 -05:00
iamdoubz fe355f034d feat(import): run import in background with per-phase progress + model override 2026-07-15 13:39:13 -05:00
iamdoubz e0e9377273 feat(storage): set_meeting_status for background import status transitions 2026-07-15 13:35:06 -05:00
iamdoubz a95eafe952 docs(api): enhance_notes, monitor_audio_level, stress_test_hardware + events 2026-07-15 09:50:17 -05:00
iamdoubz 5b9e0ed1f7 feat(app): register enhance_notes, monitor_audio_level, stress_test_hardware 2026-07-15 09:50:16 -05:00
iamdoubz 6c343c4204 feat(hardware): enhance_notes, monitor_audio_level, stress_test_hardware commands 2026-07-15 09:50:15 -05:00
iamdoubz 7d16c864e8 feat(ui): i18n for notes editor, vault, and hardware test/stress 2026-07-15 09:50:13 -05:00
iamdoubz e461d4eb22 feat(ui): enhanceNotes/monitorAudioLevel/stressTestHardware bindings + events + types 2026-07-15 09:50:12 -05:00
iamdoubz b14728e1c8 feat(ui): vault lock card, audio device test meter, quick hardware stress test 2026-07-15 09:50:00 -05:00
iamdoubz b7e6f54d91 feat(ui): header reshuffle — actions far-left, template beside them, drop wordmark 2026-07-15 09:49:58 -05:00
iamdoubz cf919f1499 fix(notes): refresh notes pane after re-transcribe; add AI-enhance + slash/toolbar
Re-sync the editor buffer after reprocess (was stale until restart). Adds a
Granola-style AI-enhance button (grounded in transcript, one-step undo), H3/
numbered/quote/divider toolbar buttons, and slash commands.
2026-07-15 09:49:57 -05:00
iamdoubz c616e0bfec Merge pull request 'Chores customize install' (#23) from chores_customize_install into main
Reviewed-on: #23
2026-07-14 21:40:02 -05:00
iamdoubz ecbe3e464c docs(release): v0.7.0 release notes with SHA-256 checksums 2026-07-14 20:01:04 -05:00
iamdoubz 7f6c0fac81 chore: bump version 0.7.0 2026-07-14 19:53:03 -05:00
iamdoubz 6fa9916f7e docs(readme): opt-in startup, install scope, enterprise deployment pointer 2026-07-14 16:27:02 -05:00
iamdoubz d78a426a7e docs(api): set_auto_start command 2026-07-14 16:27:01 -05:00
iamdoubz 1b36df020a docs(data-model): auto_start setting + wa-defaults.ini first-run seeding 2026-07-14 16:27:00 -05:00
iamdoubz 4af008da0f docs(deploy): enterprise deployment guide (wa-defaults.ini, install flags) 2026-07-14 16:26:59 -05:00
iamdoubz 223dc5f34b docs(adr): ADR-0012 launch-at-login & enterprise deployment defaults 2026-07-14 16:26:58 -05:00
iamdoubz 27773a03e5 feat(ui): i18n for launch-at-login and hardware refresh 2026-07-14 16:24:28 -05:00
iamdoubz 203e7b1bf6 feat(ui): launch-at-login toggle + refresh-hardware button 2026-07-14 16:24:27 -05:00
iamdoubz a6e1728346 feat(ui): setAutoStart store action + auto_start default 2026-07-14 16:24:26 -05:00
iamdoubz 26808970df feat(ui): setAutoStart binding + auto_start in AppSettings 2026-07-14 16:24:25 -05:00
iamdoubz 4a63b38ee0 chore(bundle): include wa-defaults.ini in the Vulkan release resources 2026-07-14 16:21:43 -05:00
iamdoubz 5d6563231c chore(bundle): bundle wa-defaults.ini + NSIS installMode both (per-user/all-users) 2026-07-14 16:21:42 -05:00
iamdoubz 2edd0382d3 feat(deploy): ship commented wa-defaults.ini template (no-op until edited) 2026-07-14 16:21:41 -05:00
iamdoubz 1530bca976 feat(app): register autostart plugin, first-run deploy seed, auto_start reconcile 2026-07-14 16:21:35 -05:00
iamdoubz 55c5143de0 feat(deploy): first-run wa-defaults.ini settings seeding
Reads an admin-supplied .ini once before settings.json exists; coerces
values onto typed Settings via serde; drops secret-looking keys.
2026-07-14 16:21:34 -05:00
iamdoubz 47071b5861 feat(app): set_auto_start command + default_settings/save_settings visibility 2026-07-14 16:21:33 -05:00
iamdoubz ded5e00750 feat(app): add auto_start setting (opt-in, default off, NFR-RES-4) 2026-07-14 16:21:26 -05:00
iamdoubz 2fa3ea6824 feat(app): add tauri-plugin-autostart dependency
Pulls in the official launch-at-login plugin (per-user HKCU\Run, no admin).
Regenerated ACL/schema manifests reflect the new plugin.
2026-07-14 16:21:25 -05:00
iamdoubz 14300fccee Remove temp changelog files 2026-07-14 15:54:41 -05:00
iamdoubz 2c8958da01 Bump version 0.6.0 2026-07-14 15:48:50 -05:00
iamdoubz f2c945838c docs(plan): mark dual-channel capture implemented (FR-SPK) 2026-07-14 12:22:59 -05:00
iamdoubz 0e0f3a09f1 docs(architecture): per-stream diarization via split channels in stop data flow (FR-SPK) 2026-07-14 12:22:57 -05:00
iamdoubz 3d183fdd7c docs(data-model): audio.wav split layout + audio_layout column; drop mic_activity.json (FR-SPK) 2026-07-14 12:22:56 -05:00
iamdoubz 32bb4057ec docs(adr-0005): Phase 3.5 dual-channel capture supersedes the masked-mono timeline (FR-SPK) 2026-07-14 12:22:55 -05:00
iamdoubz 90a9cbd476 feat(recording): fold split stereo to mono on playback, dual-mono on export (FR-SPK)
fold_wav folds a WAV's channels to their average across N output channels.
serve_recording plays it as mono so a split (mic-left/loopback-right) recording
plays both sources in both ears; bundle export decrypts + folds to dual-mono so
the shared file is a normal stereo file (manifest records it as summed). Test
covers mono + dual-mono.
2026-07-14 12:21:12 -05:00
iamdoubz 5bd76b80e8 feat(diarization): channel-based split attribution; retire masked-mono path (FR-SPK)
stop + reprocess now, for a split recording, diarize the right (loopback)
channel into Speaker N and take You from left-channel (mic) VAD — read back from
the file so both agree. Fixes reprocess collapsing to one speaker. Summed
recordings keep the whole-signal + voiceprint fallback. Removes MicTimelineFile,
phase3_attribute, mask_ranges, mic_activity.json (retirement note retained).
2026-07-14 12:16:03 -05:00
iamdoubz a1d71d1c73 refactor(recording): drop RecordingSession.mic_activity (retired) (FR-SPK) 2026-07-14 12:16:02 -05:00
iamdoubz e287fa5ebe feat(audio): read_wav_channel_16k + vad_spans; retire MicActivity (FR-SPK)
Add a channel-selective 16k reader (share the decode with read_wav_mono_16k via
read_wav_16k) and vad_spans (100ms energy-windowed You detection). Remove the
MicActivity live timeline + its capture wiring — the split layout makes the
mic/far-side separation intrinsic to the file, so You is recomputed from the
left channel instead.
2026-07-14 12:16:01 -05:00
iamdoubz 534f5ca350 feat(commands): mark recordings split/summed at stop from the session layout (FR-SPK) 2026-07-14 12:04:46 -05:00
iamdoubz 1e99c4c747 feat(recording): RecordingSession.audio_layout (split when mic on) (FR-SPK) 2026-07-14 12:04:45 -05:00
iamdoubz 5e5f61f3b7 feat(audio): write_wav_split — stereo mic-left/loopback-right capture (FR-SPK)
New split writer + a `split` flag through the capture path: when the mic is on,
audio.wav is recorded 2ch with the mic on the left channel and the loopback
(downmixed) on the right, keeping the two sources separate in the file. Same
byte size as today's summed stereo. Transcription still downmixes L+R (= the
summed signal). Unit test covers L=mic / R=loopback.
2026-07-14 12:04:44 -05:00
iamdoubz dec9251e7b feat(commands): set audio_layout at each finalize/export site (summed for now) (FR-SPK) 2026-07-14 11:58:25 -05:00
iamdoubz d4e2ab494c feat(models): MeetingBundle.audio_layout (serde default) for portable layout (FR-SPK) 2026-07-14 11:58:23 -05:00
iamdoubz 845903a772 feat(storage): thread audio_layout through FinalizeMeeting + Meeting + get_meeting (FR-SPK) 2026-07-14 11:58:22 -05:00
iamdoubz 998de62df2 feat(storage): migration 0009 — meetings.audio_layout column (FR-SPK/FR-CAP) 2026-07-14 11:58:21 -05:00
iamdoubz e449c194ef docs(plan): draft dual-channel (mic-L/loopback-R) capture plan + ADR-0005 amendment for review (FR-SPK) 2026-07-14 11:28:16 -05:00
iamdoubz 6cea767dec fix(audio): drain mic device-paced during silence to stop playback skips/ticks (FR-CAP-7)
The capture-silence fix metered the mic pull by wall-clock, but the mic fills
the bridge at its own device clock. Over a long silent stretch they drifted, the
bridge backed up to its 0.5s cap and dropped oldest samples -> the recording
skipped/'fast-forwarded' late in the gap. And a 10ms idle threshold punched
silence into the loopback on brief inter-packet stalls -> audible ticks. Now:
drain the whole bridge (device-paced, no overflow) and only synthesize after a
sustained 150ms gap.
2026-07-14 11:10:07 -05:00
iamdoubz 60ae97c771 docs(plan): mark Phase 3 slices a–d landed (FR-SPK) 2026-07-14 00:53:07 -05:00
iamdoubz d8587b8e93 docs(architecture): note per-stream diarization in the stop data flow (FR-SPK) 2026-07-14 00:53:06 -05:00
iamdoubz 695a12dc2d docs(data-model): add mic_activity.json to the meeting dir list (FR-SPK) 2026-07-14 00:53:05 -05:00
iamdoubz 6385e38b33 docs(adr-0005): amend with Phase 3 per-stream 'You' attribution (FR-SPK) 2026-07-14 00:53:04 -05:00
iamdoubz 0b5a85f461 feat(diarization): persist mic_activity.json; reprocess uses the Phase 3 timeline (FR-SPK)
Factor the Phase 3 attribution (you_spans -> mask -> diarize far side -> merge +
name) into a shared phase3_attribute() used by both stop and reprocess. stop now
writes mic_activity.json (the 'You' spans) next to a retained audio.wav
(plaintext timing metadata, local-only, lives/dies with audio.wav); reprocess
prefers that timeline over the voiceprint fallback so re-transcription keeps
correct per-stream 'You'/'Speaker' attribution.
2026-07-14 00:51:41 -05:00
iamdoubz fd9311c482 feat(diarization): Phase 3 stop attribution — mic timeline is 'You', diarize far side masked (FR-SPK)
stop_recording now, when the mic is on, takes 'You' spans straight from the
MicActivity timeline and diarizes the recording with those ranges zeroed, so
sherpa only ever clusters the far side into Speaker N. Merges the span lists and
names them via the existing voiceprint map (You + Speaker 2..). Falls back to
the whole-signal pass + voiceprint match when the mic is off. Adds mask_ranges
helper + tests.
2026-07-14 00:04:58 -05:00
iamdoubz 39294adcbc feat(audio): populate MicActivity timeline from the loopback writer (FR-SPK) 2026-07-14 00:04:57 -05:00
iamdoubz fee18ebc59 refactor(diarization): expose first_appearance_order + build_name_map to crate (FR-SPK) 2026-07-14 00:04:56 -05:00
iamdoubz 51f5ff085c feat(diarization): Diarizer::diarize_samples + short-input guard for masked far-side pass (FR-SPK) 2026-07-14 00:04:55 -05:00
iamdoubz 6d4204fb22 feat(recording): RecordingSession.mic_activity for Phase 3 attribution (FR-SPK) 2026-07-14 00:04:54 -05:00
iamdoubz 185489fa18 feat(audio): MicActivity timeline for Phase 3 per-stream 'You' attribution (FR-SPK)
Records, per 100ms of audio.wav frame time, when the mic (the user) was
speech-level — captured in the loopback writer where mic and loopback exist
separately in the recording's own timebase. you_spans() collapses active
windows into merged 'You' ranges. The far side will be diarized with these
ranges masked out, so clustering never sees the mic. Struct + span logic land
first (tested); capture wiring + stop attribution follow.
2026-07-13 21:15:49 -05:00
iamdoubz 41c2c79797 docs(plan): finalize Phase 3 design — mic=You, diarize loopback-masked (FR-SPK) 2026-07-13 21:15:48 -05:00
iamdoubz 6dd6c3c69f fix(audio): capture mic during system-audio silence (FR-CAP-7)
WASAPI loopback delivers no packets when the render endpoint is idle, so the
recording only advanced while system audio played. The bridged mic (the user
talking over silence) was lost to MicBridge overflow, and audio.wav lost those
stretches entirely — corrupting diarization and dropping the user's solo
remarks in real meetings. When loopback is idle, synthesize silence for the
elapsed gap and fold the mic in, keeping audio.wav continuous. Loopback +
mic-enabled only.
2026-07-13 20:58:23 -05:00
iamdoubz 9468781218 fix(audio): gate voiceprint capture on mic energy so it's real speech (FR-SPK)
VoiceSample kept the first N samples unconditionally, so a user silent for the
first 30s got a voiceprint of room noise/a cough. That noise embedding
spuriously matched a cluster and mislabeled every speaker 'You', even during
silence. Skip sub-threshold (VOICE_ENERGY_FLOOR) chunks so the window tracks
actual speech; until ~1s of real voice accumulates, the match returns empty
(no premature 'You').
2026-07-13 20:35:25 -05:00
iamdoubz b39352dbff fix(diarization): clear stale speaker rows on reprocess so Participants pane matches (FR-SPK)
Reprocess re-diarizes and upserts the new labels but never deleted the old
ones, so get_meeting (Participants pane) kept returning the discarded run's
labels (e.g. 83) while the transcript showed the new 5. Clear the meeting's
speaker rows before finalize re-inserts the fresh set.
2026-07-13 20:09:03 -05:00
iamdoubz 0d263637df feat(storage): Store::clear_speakers to drop a meeting's speaker rows (FR-SPK) 2026-07-13 20:09:01 -05:00
iamdoubz bb6e222101 docs(plan): Phase 3 per-stream attribution design sketch + ADR-0005 review (FR-SPK) 2026-07-13 19:32:15 -05:00
iamdoubz 85ed3b5587 docs(api-contracts): note transcript://segment re-emit + live diarization://updated (FR-SPK) 2026-07-13 19:25:31 -05:00
iamdoubz 68d692fbe8 fix(transcript): resolve live speaker names incl 'You' in the recording view (FR-SPK) 2026-07-13 19:25:30 -05:00
iamdoubz fa664e5147 feat(recording): live speakers state from diarization://updated (FR-SPK) 2026-07-13 19:25:29 -05:00
iamdoubz b5c99621b5 feat(api): onDiarizationUpdated event listener for live speaker updates (FR-SPK) 2026-07-13 19:25:28 -05:00
iamdoubz db52b9e812 feat(diarization): live tick re-emits relabeled segments + live 'You' (FR-SPK)
Phase 2 backend. The 15s provisional pass now (1) runs the mic voiceprint
match every tick and merges 'You'/'Speaker N' into the session name map without
overwriting user-set names, and (2) re-emits every committed segment whose
speaker changed via transcript://segment (replace-by-id) so the live transcript
reflects the refined labels instead of a frozen 'S1'.
2026-07-13 19:25:27 -05:00
iamdoubz 319e7a31cc docs(data-model): add voiceprint.wav to the meeting dir file list (FR-SPK) 2026-07-13 19:19:06 -05:00
iamdoubz 25011d0f27 fix(diarization): reprocess re-diarizes + persist mic voiceprint.wav (FR-SPK)
Phase 1. stop_recording writes voiceprint.wav next to a retained audio.wav
(ADR-0009 gated, sealed at rest with the vault); reprocess_transcript now
re-diarizes the fresh transcript and rebuilds the speaker name map from the
voiceprint instead of reusing the stale original-run names, fixing the
collapse to a single 'You' speaker on re-transcription.
2026-07-13 19:19:06 -05:00
iamdoubz 7733e86cfd feat(audio): write_wav_mono_16k helper for persisting the mic voiceprint (FR-SPK) 2026-07-13 19:19:05 -05:00
iamdoubz a481f5dcbf fix(diarization): tame cluster explosion with threshold 0.7 + 700ms min span (FR-SPK)
Phase 0 of the speaker-accuracy plan. sherpa fast-clustering threshold 0.5
over summed mic+loopback mono exploded a 2-speaker call into 83 clusters;
raise to 0.7 (larger threshold -> fewer clusters) and drop sub-700ms spans
whose embeddings are unstable and only churn labels in assign_by_overlap.
2026-07-13 19:12:04 -05:00
iamdoubz bd6519b65e docs(plan): phased fix plan for diarization speaker accuracy (FR-SPK) 2026-07-13 18:40:44 -05:00
iamdoubz 4f3af98bfa Add Clippy marketing image 2026-07-13 16:15:18 -05:00
iamdoubz 77f9a8b518 docs(readme): bump status to v0.5.2 and add UI-localization comparison row 2026-07-13 16:07:12 -05:00
iamdoubz 2a7e6229f4 docs(readme): list the 20 available interface languages and RTL support 2026-07-13 16:05:12 -05:00
iamdoubz 64e7baad66 docs(release): v0.5.2 release notes with SHA-256 checksums 2026-07-13 16:00:00 -05:00
iamdoubz 804d451bfa chore(release): bump version to 0.5.2 2026-07-13 15:59:58 -05:00
iamdoubz a29e60b47d fix(storage): show native error dialog on startup DB failure instead of panic
A migration-checksum mismatch (an edited-after-shipping migration) made
SqliteStore::connect() return Err, which .expect() turned into a panic=abort
fast-fail: the window never opened and the OS reported only a 0xc0000409 crash.
Replace the expect with a blocking MessageBoxW that names the failure and points
at wa.db, then exit(1) cleanly.
2026-07-13 15:52:55 -05:00
iamdoubz 969ed6022e build(deps): enable Win32_UI_WindowsAndMessaging for startup error dialog 2026-07-13 15:52:54 -05:00
50 changed files with 5080 additions and 629 deletions
+98 -59
View File
@@ -1,3 +1,7 @@
<p align="center">
<img src="bonus/clippy-says-hej.png" alt="WhispAssist logo with friendly Clippy saying hi there showing open source, private, note taker, and meeting advisor">
</p>
# WhispAssist (WA) # WhispAssist (WA)
**A privacy-first, Windows-native meeting assistant that runs entirely on-device.** **A privacy-first, Windows-native meeting assistant that runs entirely on-device.**
@@ -7,35 +11,39 @@ on-device acceleration (**NPU → GPU → CPU**), labels speakers, structures th
Markdown notes, and optionally augments them with a locally hosted LLM (Ollama). Audio and Markdown notes, and optionally augments them with a locally hosted LLM (Ollama). Audio and
transcripts **never leave the machine** unless you explicitly configure a destination. transcripts **never leave the machine** unless you explicitly configure a destination.
> **Status: working application (v0.4.0).** Capture (system audio **+ your microphone**, with a > **Status: working application (v0.7.3).** Capture (system audio **+ your microphone**, with a
> live **dual level meter**), transcription (CPU / Intel NPU / Vulkan GPU) with a **fluid live > live **dual level meter** and an **M-key mic mute**), transcription (CPU / Intel NPU / Vulkan
> transcript**, speaker diarization, storage + crash recovery, local-LLM summaries, AI tags, > GPU) with a **fluid live transcript**, speaker diarization that labels **You** live from your
> own mic channel, storage + crash recovery, local-LLM summaries, AI tags, **AI-enhanced notes**,
> opt-in recording with **in-app playback**, at-rest encryption, self-hosted sync, **Outlook > opt-in recording with **in-app playback**, at-rest encryption, self-hosted sync, **Outlook
> `.pst`/calendar import with optional auto-record**, **importing an existing recording from a > `.pst`/calendar import with optional auto-record**, **background import of an existing
> file or URL**, and a loopback **MCP server** for coding-agent handoff are all implemented and > recording from a file or URL** (with a step-by-step progress tracker), **close-to-tray**, and a
> ship as a single signed **MSI + NSIS** universal installer. Build order and remaining tasks > loopback **MCP server** for coding-agent handoff are all implemented and ship as a single
> are in [`docs/05-roadmap.md`](docs/05-roadmap.md). > **MSI + NSIS** universal installer. On Intel Core Ultra machines, transcription runs with the
> **CPU near-idle**: Whisper's encoder on the **NPU**, its decoder on the **iGPU**. Build order
> and remaining tasks are in [`docs/05-roadmap.md`](docs/05-roadmap.md).
## Why WhispAssist — Granola vs Meetily vs WhispAssist ## Why WhispAssist — Granola vs Meetily vs WhispAssist
| | **Granola** | **Meetily** | **WhispAssist** | | | **Granola** | **Meetily** | **WhispAssist** |
|---|---|---|---| | ---------------------------------------------- | ----------------------------------------------- | --------------------------------- | ----------------------------------------------------------- |
| **Positioning** | Cloud AI notepad | Open-source self-hosted assistant | Local, Windows-native assistant | | **Positioning** | Cloud AI notepad | Open-source self-hosted assistant | Local, Windows-native assistant |
| **Platform** | macOS, Windows | macOS, Windows, Linux | Windows 10/11 | | **Platform** | macOS, Windows | macOS, Windows, Linux | Windows 10/11 |
| **Bot-free system-audio capture** | ✅ | ✅ | ✅ WASAPI loopback | | **Bot-free system-audio capture** | ✅ | ✅ | ✅ WASAPI loopback |
| **Transcription** | ☁️ cloud | ✅ local (Whisper) | ✅ local (whisper.cpp) | | **Transcription** | ☁️ cloud | ✅ local (Whisper) | ✅ local (whisper.cpp) |
| **NPU / GPU auto-acceleration** | n/a (cloud) | ⚠️ CPU/GPU, manual | ✅ **NPU→NVIDIA→AMD→Intel→CPU** ladder, one binary | | **NPU / GPU auto-acceleration** | n/a (cloud) | ⚠️ CPU/GPU, manual | ✅ **NPU→NVIDIA→AMD→Intel→CPU** ladder, one binary |
| **Speaker diarization** | ✅ cloud | ⚠️ limited | ✅ offline (sherpa-onnx) | | **Speaker diarization** | ✅ cloud | ⚠️ limited | ✅ offline (sherpa-onnx) |
| **Summaries / notes AI** | ☁️ cloud LLM | ✅ local (Ollama) / BYO | ✅ local (Ollama, localhost **or LAN**) + optional hosted | | **Summaries / notes AI** | ☁️ cloud LLM | ✅ local (Ollama) / BYO | ✅ local (Ollama, localhost **or LAN**) + optional hosted |
| **Default data egress** | ☁️ audio + notes to cloud | 🔒 local (cloud optional) | 🔒 **none** — everything off by default, allowlist-enforced | | **Interface localization (UI language)** | ⚠️ English-only UI (multilingual transcription) | ⚠️ English-only UI (more planned) | ✅ **20 languages** + RTL (Arabic/Urdu) |
| **Calendar / Outlook `.pst` context** | ✅ cloud calendar | ❌ | ✅ local `.pst` import + **auto-record on events** | | **Default data egress** | ☁️ audio + notes to cloud | 🔒 local (cloud optional) | 🔒 **none** — everything off by default, allowlist-enforced |
| **Import an existing recording (file/URL)** | ❌ live capture only | ❌ | ✅ any file or URL (ffmpeg + yt-dlp) | | **Calendar / Outlook `.pst` context** | ✅ cloud calendar | ❌ | ✅ local `.pst` import + **auto-record on events** |
| **Opt-in recording + consent notice** | ⚠️ partial | ❌ | ✅ off by default, one-time consent | | **Import an existing recording (file/URL)** | ❌ live capture only | ❌ | ✅ any file or URL (ffmpeg + yt-dlp) |
| **At-rest encryption** | ☁️ server-side | ❌ | ✅ vault: Argon2id + XChaCha20-Poly1305 | | **Opt-in recording + consent notice** | ⚠️ partial | ❌ | ✅ off by default, one-time consent |
| **Self-hosted sync w/ client-side encryption** | ❌ | ⚠️ | ✅ WebDAV + OAuth, **encrypt-before-upload** | | **At-rest encryption** | ☁️ server-side | ❌ | ✅ vault: Argon2id + XChaCha20-Poly1305 |
| **Coding-agent (MCP) server** | ✅ **cloud** MCP (notes via their servers) | ❌ | ✅ **local, loopback-only**, token-gated, adds no egress | | **Self-hosted sync w/ client-side encryption** | ❌ | ⚠️ | ✅ WebDAV + OAuth, **encrypt-before-upload** |
| **License** | Proprietary | Open source (MIT) | Open source (MIT / Apache-2.0) | | **Coding-agent (MCP) server** | ✅ **cloud** MCP (notes via their servers) | ❌ | ✅ **local, loopback-only**, token-gated, adds no egress |
| **Cost** | Subscription | Free | Free | | **License** | Proprietary | Open source (MIT) | Open source (MIT / Apache-2.0) |
| **Cost** | Subscription | Free | Free |
<sub>Comparison reflects each project's public positioning as of mid-2026. Granola now ships an <sub>Comparison reflects each project's public positioning as of mid-2026. Granola now ships an
MCP server too, but it is **cloud-hosted** — an agent reaching it pulls your notes through MCP server too, but it is **cloud-hosted** — an agent reaching it pulls your notes through
@@ -51,29 +59,36 @@ accelerated, zero-egress-by-default** option: it exploits the NPU/GPU in modern
everything on the device unless you opt in, and adds Windows-specific context (Outlook) and a everything on the device unless you opt in, and adds Windows-specific context (Outlook) and a
coding-agent handoff. coding-agent handoff.
## What's built (v0.4.0) ## What's built (v0.7.3)
- **Bot-free capture — now both sides, with a live dual meter** — WASAPI loopback records the - **Bot-free capture — now both sides, with a live dual meter and mic mute** — WASAPI loopback
system mix (all participants), and an optional **microphone** path captures your own voice, records the system mix (all participants), and an optional **microphone** path captures your
mixed into both the live transcript and the saved recording. While recording, a **level meter own voice, mixed into both the live transcript and the saved recording. While recording, a
overlays the system and microphone signals in two colours** so you can see both sides are being **level meter overlays the system and microphone signals in two colours** so you can see both
picked up. Pick a specific output/input or turn the mic off in **Settings ▸ Hardware**. No sides are being picked up, and the **M key (or the mic button) mutes/unmutes your mic
mid-meeting**. Pick a specific output/input or turn the mic off in **Settings ▸ Hardware**. No
meeting bot, no per-app plumbing. meeting bot, no per-app plumbing.
- **Local transcription with a hardware ladder — and a fluid live transcript** — whisper.cpp via - **Local transcription with a hardware ladder — and a fluid live transcript** — whisper.cpp via
`whisper-rs` on CPU; the **Intel NPU** via ONNX Runtime + OpenVINO; **GPU via Vulkan** (a single `whisper-rs` on CPU; the **Intel NPU** via ONNX Runtime + OpenVINO; **GPU via Vulkan** (a single
binary that runs on NVIDIA, AMD, and Intel). WA detects the hardware, picks the best backend binary that runs on NVIDIA, AMD, and Intel). WA detects the hardware, picks the best backend
(**NPU → NVIDIA → AMD → Intel → CPU**), and shows it in the UI. The **live transcript streams a (**NPU → NVIDIA → AMD → Intel → CPU**), and shows it in the UI. On NPU machines the engine
growing line that refreshes ~once a second and commits at natural pauses** — words appear as **splits Whisper across accelerators — encoder on the NPU, KV-cache decoder on the Intel iGPU —
they're spoken instead of in fixed multi-second blocks, so sentences aren't chopped across so transcription leaves the CPU near-idle**. The **live transcript streams a growing line that
lines. refreshes ~once a second and commits at natural pauses** — words appear as they're spoken
- **Speaker diarization** — `sherpa-onnx` (pyannote segmentation + speaker-embedding clustering), instead of in fixed multi-second blocks, so sentences aren't chopped across lines.
fully offline. Install the two diarization models in **Settings ▸ Hardware** and finished - **Speaker diarization that knows who _you_ are** — `sherpa-onnx` (pyannote segmentation +
recordings are split by speaker (Speaker 1, Speaker 2, …); the microphone speaker is speaker-embedding clustering), fully offline. Install the two diarization models in
auto-labelled from a short voiceprint. **Settings ▸ Hardware** and recordings are split by speaker (You, Speaker 2, …). With the mic
- **Import an existing recording** — add a meeting from a **local audio/video file or a URL** enabled, your voice is recorded on its **own channel**, so your lines are labelled **You** live
(YouTube, a streaming page, or a direct media link). WhispAssist transcribes and diarizes it — from the first pass, not just after the meeting — and attributed by per-channel voice
just like a live recording. Uses **`ffmpeg`** (and **`yt-dlp`** for URLs), which you install activity rather than clustering guesswork. Renaming a speaker updates the transcript **and the
yourself — neither is bundled. generated notes**.
- **Import an existing recording — in the background, with a progress tracker** — add a meeting
from a **local audio/video file or a URL** (YouTube, a streaming page, or a direct media link);
it transcribes quietly in the background behind a four-step tracker (_Transcode → Transcribe →
Identify speakers → Finalize_) while you keep working. Uses **`ffmpeg`** (and **`yt-dlp`** for
URLs), which you install yourself — neither is bundled, and one-click download buttons are in
the Import dialog.
- **Notes & summaries** — Markdown notes; local-LLM summaries via **Ollama** on `localhost` - **Notes & summaries** — Markdown notes; local-LLM summaries via **Ollama** on `localhost`
**or a private LAN endpoint** (RFC-1918), with a full advanced-parameter panel (system prompt, **or a private LAN endpoint** (RFC-1918), with a full advanced-parameter panel (system prompt,
`think`, `keep_alive`, `num_ctx`, sampling/repetition/mirostat, etc.). `think`, `keep_alive`, `num_ctx`, sampling/repetition/mirostat, etc.).
@@ -85,8 +100,10 @@ coding-agent handoff.
recordings are decrypted **in memory on the fly** (nothing plaintext is written to disk). recordings are decrypted **in memory on the fly** (nothing plaintext is written to disk).
Recordings are 16-bit for roughly half the size, and an accidental recording can be **cancelled** Recordings are 16-bit for roughly half the size, and an accidental recording can be **cancelled**
(audio + transcript deleted). (audio + transcript deleted).
- **Notes, summaries & AI tags** — Markdown notes with an **Editor/Preview** toggle; local-LLM - **Notes, summaries & AI tags** — Markdown notes with an **Editor/Preview** toggle, a formatting
summaries and one-click **tag generation** with a chip-based tag editor and tag filtering. toolbar, and **slash commands** (`/todo`, `/h1`, `/quote`, …); **AI-enhance** turns rough notes
into clean structured Markdown via your local LLM (with one-step Undo); local-LLM summaries and
one-click **tag generation** with a chip-based tag editor and tag filtering.
- **At-rest encryption vault** — Argon2id key derivation + XChaCha20-Poly1305; transcripts, - **At-rest encryption vault** — Argon2id key derivation + XChaCha20-Poly1305; transcripts,
notes, summaries, and recordings sealed on disk; startup unlock gate; keys zeroized on lock. notes, summaries, and recordings sealed on disk; startup unlock gate; keys zeroized on lock.
- **Self-hosted sync (optional, off by default)** — WebDAV (Nextcloud, ownCloud, Cloudreve, - **Self-hosted sync (optional, off by default)** — WebDAV (Nextcloud, ownCloud, Cloudreve,
@@ -98,14 +115,27 @@ coding-agent handoff.
- **Outlook `.pst` / calendar context, with auto-record** — import events and attendees from a - **Outlook `.pst` / calendar context, with auto-record** — import events and attendees from a
local Outlook `.pst` backup (read-only, range-limited, de-duplicated, with cleanup), attach local Outlook `.pst` backup (read-only, range-limited, de-duplicated, with cleanup), attach
meetings to events, and — opt-in — **auto-start recording when a calendar event begins** while meetings to events, and — opt-in — **auto-start recording when a calendar event begins** while
the app is open (a one-shot timer, no background polling). the app is open (a one-shot timer, no background polling). The import shells out to `readpst`
(libpst) — **Settings ▸ Calendar** has a one-click download button for it.
- **Close to tray** — closing the window keeps WhispAssist running in the background; reopen it
from the tray icon, quit fully from the tray menu. On by default, toggleable in
**Settings ▸ Recording**.
- **Local MCP server for coding-agent handoff** — hand meeting context to your own coding agents - **Local MCP server for coding-agent handoff** — hand meeting context to your own coding agents
(Claude, Codex, Copilot, OpenCode) over a **loopback-only, token-gated** MCP server that is off (Claude, Codex, Copilot, OpenCode) over a **loopback-only, token-gated** MCP server that is off
by default, scope-limited, audited, and **adds no egress** — data leaves only via the agent's by default, scope-limited, audited, and **adds no egress** — data leaves only via the agent's
own provider, which WA discloses. own provider, which WA discloses.
- **One universal installer** — a single signed MSI and NSIS `-setup.exe` that covers every - **20 interface languages, with RTL** — the whole UI (shell, meetings list, transcript/notes,
machine: Vulkan for all GPUs, the Intel NPU path (+ DirectML fallback), and CPU. The Vulkan summary, and Settings) is available in **20 languages**, chosen under **Settings ▸ Language**:
loader is bundled so it launches even on machines without a GPU driver. Arabic (العربية), Bengali (বাংলা), Simplified Chinese (中文简体), English, Finnish (Suomi),
French — France & Canada (Français), German (Deutsch), Hindi (हिन्दी), Korean (한국어),
Norwegian Bokmål (Norsk bokmål), Portuguese — Brazil & Portugal (Português), Russian (Русский),
Sinhala (සිංහල), Spanish — Spain & Mexico (Español), Swedish (Svenska), Tamil (தமிழ்), and
Urdu (اردو). Selecting Arabic or Urdu flips the interface to **right-to-left**; any untranslated
string falls back to English rather than showing a raw key. Adding a language is a one-file job.
- **One universal installer** — a single MSI and NSIS `-setup.exe` that covers every machine:
Vulkan for all GPUs, the Intel NPU path (+ DirectML fallback), and CPU. The Vulkan loader is
bundled so it launches even on machines without a GPU driver. Every release ships **SHA-256
checksums** in its release notes.
**In progress:** Microsoft Graph calendar (cloud calendar via OAuth) and an optional CUDA **In progress:** Microsoft Graph calendar (cloud calendar via OAuth) and an optional CUDA
(NVIDIA-only) build variant. (NVIDIA-only) build variant.
@@ -119,30 +149,39 @@ during setup).
1. Download the latest **`WhispAssist_<version>_x64_en-US.msi`** from the 1. Download the latest **`WhispAssist_<version>_x64_en-US.msi`** from the
[releases page](https://git.dou.bet/iamdoubz/WhispAssist/releases). [releases page](https://git.dou.bet/iamdoubz/WhispAssist/releases).
2. Double-click the `.msi` and follow the prompts, accepting the UAC prompt to install. The 2. Double-click the `.msi` and follow the prompts, accepting the UAC prompt to install. The
installer is signed; if Windows SmartScreen appears, choose **More info → Run anyway**. installers are currently unsigned, so if Windows SmartScreen appears, choose
**More info → Run anyway** (verify the download against the SHA-256 checksums in the release
notes first).
3. Launch **WhispAssist** from the Start menu. 3. Launch **WhispAssist** from the Start menu.
On first run WA detects your hardware and picks the best transcription backend On first run WA detects your hardware and picks the best transcription backend
(**NPU → NVIDIA → AMD → Intel → CPU**). It works on your **CPU or GPU** (GPU via Vulkan) out of the (**NPU → NVIDIA → AMD → Intel → CPU**). It works on your **CPU or GPU** (GPU via Vulkan) out of the
box; to use an Intel **NPU**, open **Settings ▸ Hardware** and download the one-time NPU box; to use an Intel **NPU**, open **Settings ▸ Hardware** and download the one-time NPU
acceleration package. The app runs without admin rights, never adds itself to startup, and keeps all acceleration package. The app runs without admin rights, does not add itself to startup unless you
data under `%LOCALAPPDATA%\WhispAssist`. opt in (**Settings ▸ Recording ▸ Launch at login**), and keeps all data under
`%LOCALAPPDATA%\WhispAssist`.
Prefer the NSIS installer? Grab **`WhispAssist_<version>_x64-setup.exe`** from the same page. Prefer the NSIS installer? Grab **`WhispAssist_<version>_x64-setup.exe`** from the same page — it
lets you choose a **current-user** (no admin) or **all-users** install.
**Deploying to many machines?** See [`docs/enterprise-deployment.md`](docs/enterprise-deployment.md)
for silent install, custom install location, and presetting defaults with a `wa-defaults.ini` file.
## Optional dependencies ## Optional dependencies
If you do not have these installed, WhispAssist will still work, but some features will be unavailable. If you do not have these installed, WhispAssist will still work, but some features will be
unavailable. The app links to each one where it's needed: `ffmpeg`/`yt-dlp` in the Import dialog,
`libpst` in **Settings ▸ Calendar**.
- [yt-dlp](https://github.com/yt-dlp/yt-dlp/releases) - [yt-dlp](https://github.com/yt-dlp/yt-dlp/releases) — importing from URLs
- [ffmpeg](https://www.ffmpeg.org/download.html#build-windows) - [ffmpeg](https://www.ffmpeg.org/download.html#build-windows) — importing files/URLs
- [libpst](https://sourceforge.net/projects/ezwinports/files/libpst-0.6.63-w32-bin.zip/) - [libpst](https://sourceforge.net/projects/ezwinports/files/libpst-0.6.63-w32-bin.zip/) — Outlook `.pst` calendar import
## Technology ## Technology
WhispAssist is a **Tauri 2** application: a small Rust core with a compiled **Svelte + TypeScript** WhispAssist is a **Tauri 2** application: a small Rust core with a compiled **Svelte + TypeScript**
frontend rendered through the OS WebView2 (no bundled browser → low idle memory). Every decision frontend rendered through the OS WebView2 (no bundled browser → low idle memory). Every decision
and its alternatives are recorded in [`docs/adr/`](docs/adr/) (ADR-0001–0011). and its alternatives are recorded in [`docs/adr/`](docs/adr/) (ADR-0001–0012).
- **Shell / IPC:** Tauri 2 (Rust ⇄ WebView2) - **Shell / IPC:** Tauri 2 (Rust ⇄ WebView2)
- **Audio capture:** WASAPI loopback - **Audio capture:** WASAPI loopback
@@ -163,7 +202,7 @@ and its alternatives are recorded in [`docs/adr/`](docs/adr/) (ADR-0001–0011).
WhispAssist/ WhispAssist/
├── docs/ # The engineering plan (read this first) ├── docs/ # The engineering plan (read this first)
│ ├── 00-overview.md … 07-research-findings.md │ ├── 00-overview.md … 07-research-findings.md
│ └── adr/ Architecture Decision Records (0001–0011) │ └── adr/ Architecture Decision Records (0001–0012)
├── src-tauri/ # Rust core — implemented service modules: ├── src-tauri/ # Rust core — implemented service modules:
│ └── src/{audio,transcription,diarization,storage,llm,calendar, │ └── src/{audio,transcription,diarization,storage,llm,calendar,
│ hardware,notes,sync,mcp,vault} │ hardware,notes,sync,mcp,vault}
-96
View File
@@ -1,96 +0,0 @@
# WhispAssist v0.5.0
**Privacy-first, Windows-native meeting assistant — everything on-device, nothing leaves unless you say so.**
This release is about **living with your recordings**: play back a meeting while following along in
the transcript, manage action items by hand, move recordings between computers, drop a meeting into
your Obsidian vault, and — new in this release — **use WhispAssist in your own language**. Everything
stays off-by-default and local-first.
One universal installer (**MSI** and **NSIS**) covers every machine: **Vulkan** for all GPUs
(NVIDIA/AMD/Intel), the Intel **NPU** (OpenVINO), a **DirectML** fallback, and **CPU**.
---
## ✨ Highlights
### Click the transcript to play that moment
The recording player and the transcript now talk to each other. **Click any transcript line to jump
the audio to that moment** and start playing, and as playback runs the **current line highlights and
scrolls into view** so you never lose your place. If you scroll by hand, auto-scroll steps aside for
a few seconds so it doesn't fight you.
### Interface language selector (i18n)
WhispAssist can now be **fully translated**. A new **Settings ▸ Language** picker switches the
interface language, English ships as the baseline, and **every** user-facing string across the app —
the shell, meetings list, transcript/notes, summary panel, and all of Settings — now flows through a
single translation layer. Adding a new language is as simple as translating **one JSON file**; no
code changes. (This release ships English; the groundwork is in place for community translations.)
### Manage action items yourself
Action items are no longer just whatever the summary extracted. You can now **add, edit, and delete
them by hand** in the summary panel, set an owner and due date, and toggle a local reminder. Your
edits are the source of truth and are reconciled cleanly — deleting an item also cancels its reminder.
### Move recordings between computers (Export / Import)
A new **Export & import** section in **Settings ▸ Storage** writes each meeting as a portable
**bundle folder** (audio, transcript, notes, summary, and a `meeting.json` manifest) and imports them
back on another machine. Imported meetings get a fresh id, so re-importing never overwrites anything.
Export to any folder — a synced drive, a USB stick, or a sync target's local mount — and carry it across.
### Export a meeting to Obsidian
A new **Obsidian** export writes a single self-contained vault note — YAML frontmatter
(title, date, duration, participants, tags) plus notes, summary, action items, and a timestamped
transcript — **without the audio**. Drop it in your vault and the transcript is fully searchable.
---
## 🚀 Also new since v0.4.0
- **Per-segment transcript timestamps** — every line now shows a quiet `m:ss` (or `h:mm:ss`) time
prefix, in both the live and finalized views.
- **Auto-resync on edit** — when sync is enabled, editing a meeting's notes, summary, transcript,
tags, or action items **re-uploads just the changed artifacts** automatically (deduped by hash, so
an unchanged save uploads nothing). Off unless sync is configured.
## 🐛 Fixes & polish
- Transcript scroll-intent handling refined so playback auto-scroll never yanks you back while you're
reading.
- Import preserves each meeting's original date rather than stamping the import time.
---
## 📦 Install
**Requirements:** Windows 10 or 11 (x64). WhispAssist needs **WebView2** (preinstalled on Windows 11;
the installer fetches it on Windows 10). Importing from a file/URL additionally needs **`ffmpeg`**
(and **`yt-dlp`** for URLs) on your PATH — install those separately.
1. Download **`WhispAssist_0.5.0_x64_en-US.msi`** (or the NSIS **`WhispAssist_0.5.0_x64-setup.exe`**).
2. Run it and accept the UAC prompt. If SmartScreen appears, choose **More info → Run anyway**.
3. Launch **WhispAssist** from the Start menu.
On first run WA picks the best transcription backend (**NPU → NVIDIA → AMD → Intel → CPU**). It runs
without admin rights, never adds itself to startup, and keeps all data under
`%LOCALAPPDATA%\WhispAssist`.
## 🔐 Checksums (SHA-256)
```
b423feed1a1171384e46c5e0b5aa63cbf912bfe8f29cc89a5a027bb15358075c WhispAssist_0.5.0_x64_en-US.msi
2dce26b5603a5d02094533c594d814a3d72e05512e1d735ea69e1e309f15b710 WhispAssist_0.5.0_x64-setup.exe
```
Verify after download:
```powershell
Get-FileHash .\WhispAssist_0.5.0_x64_en-US.msi -Algorithm SHA256
```
---
## Privacy, unchanged
Everything optional is **off by default**. With nothing configured, WhispAssist makes **no content
egress at all**. Recording is opt-in; sync/AI credentials live only in the OS credential store; the
MCP server is loopback-only and adds no egress. The reachable-host allowlist is derived from your
settings and enforced in the core.
-73
View File
@@ -1,73 +0,0 @@
# WhispAssist v0.5.1
**Privacy-first, Windows-native meeting assistant — everything on-device, nothing leaves unless you say so.**
v0.5.0 put the **i18n groundwork** in place. v0.5.1 fills it in: WhispAssist now ships in
**20 languages**, so you can run the whole app — the shell, meetings list, transcript/notes, summary
panel, and every corner of Settings — in your own language. Everything stays off-by-default and
local-first; this is a UI-language release with no change to what leaves your device (nothing, by default).
One universal installer (**MSI** and **NSIS**) covers every machine: **Vulkan** for all GPUs
(NVIDIA/AMD/Intel), the Intel **NPU** (OpenVINO), a **DirectML** fallback, and **CPU**.
---
## ✨ Highlights
### 20 interface languages
Pick your language under **Settings ▸ Language**. Alongside English, this release adds full
translations for:
- **Arabic** (العربية) · **Bengali** (বাংলা) · **German** (Deutsch)
- **Spanish** — Spain (Español, España) and **Mexico** (Español, México)
- **Finnish** (Suomi) · **French** — France (Français, France) and **Canada** (Français, Canada)
- **Hindi** (हिन्दी) · **Korean** (한국어) · **Norwegian Bokmål** (Norsk bokmål)
- **Portuguese** — Brazil (Português, Brasil) and **Portugal** (Português, Portugal)
- **Russian** (Русский) · **Sinhala** (සිංහල) · **Swedish** (Svenska)
- **Tamil** (தமிழ்) · **Urdu** (اردو) · **Mandarin Chinese, Simplified** (中文简体)
Your choice persists across launches, and any untranslated string quietly falls back to English rather
than showing a raw key — so partial translations degrade gracefully.
### Right-to-left layout
Selecting **Arabic** or **Urdu** flips the whole interface to **right-to-left**, so those languages
read and lay out correctly rather than being crammed into an LTR shell.
Adding a further language remains a one-file job — drop in a single JSON translation, no code changes.
---
## 📦 Install
**Requirements:** Windows 10 or 11 (x64). WhispAssist needs **WebView2** (preinstalled on Windows 11;
the installer fetches it on Windows 10). Importing from a file/URL additionally needs **`ffmpeg`**
(and **`yt-dlp`** for URLs) on your PATH — install those separately.
1. Download **`WhispAssist_0.5.1_x64_en-US.msi`** (or the NSIS **`WhispAssist_0.5.1_x64-setup.exe`**).
2. Run it and accept the UAC prompt. If SmartScreen appears, choose **More info → Run anyway**.
3. Launch **WhispAssist** from the Start menu.
On first run WA picks the best transcription backend (**NPU → NVIDIA → AMD → Intel → CPU**). It runs
without admin rights, never adds itself to startup, and keeps all data under
`%LOCALAPPDATA%\WhispAssist`.
## 🔐 Checksums (SHA-256)
```
911618e6996a079bad1cbe46731c89ee247f3f5f17c09b60091d7e6a5e7cbef0 WhispAssist_0.5.1_x64_en-US.msi
ffabbd74def981da176931349b2813a3927d7376c0d18e9d191891abfc17c385 WhispAssist_0.5.1_x64-setup.exe
```
Verify after download:
```powershell
Get-FileHash .\WhispAssist_0.5.1_x64_en-US.msi -Algorithm SHA256
```
---
## Privacy, unchanged
Everything optional is **off by default**. With nothing configured, WhispAssist makes **no content
egress at all**. Recording is opt-in; sync/AI credentials live only in the OS credential store; the
MCP server is loopback-only and adds no egress. The reachable-host allowlist is derived from your
settings and enforced in the core.
+78
View File
@@ -0,0 +1,78 @@
# WhispAssist v0.7.0
**Privacy-first, Windows-native meeting assistant — everything on-device, nothing leaves unless you say so.**
This release is about **control over install and startup** — for a single user and for admins rolling
WhispAssist out across a fleet. You can now launch WhispAssist at login, choose a per-user (no-admin)
or all-users install, preset every default with a deployment file, and re-scan your hardware without
restarting. No feature here changes the privacy posture: everything optional stays off-by-default and
local-first.
One universal installer (**MSI** and **NSIS**) covers every machine: **Vulkan** for all GPUs
(NVIDIA/AMD/Intel), the Intel **NPU** (OpenVINO), a **DirectML** fallback, and **CPU**.
---
## ✨ New
### Launch at login (opt-in)
A new **Settings ▸ Recording ▸ Launch WhispAssist at login** toggle starts WhispAssist when you sign
in to Windows. It's **off by default**, needs **no admin** (a per-user startup entry), and does **not**
begin recording on its own.
### Choose a per-user or all-users install
The NSIS setup (`WhispAssist_0.7.0_x64-setup.exe`) now lets you install for **just yourself**
(no admin rights required) or for **all users**. The MSI remains the per-machine, all-users installer.
### Customize deployments with a file (`wa-defaults.ini`)
Admins can preset WhispAssist's defaults for every machine using native Windows tooling (Group Policy,
SCCM, Intune, `msiexec`) — no management console. Drop a `wa-defaults.ini` next to the installer or in
`%PROGRAMDATA%\WhispAssist\`, and each machine's **first launch** seeds its settings from it: record-
by-default, preferred backend, a model to auto-download, retention, storage location, AI provider, and
more. You can also set a **custom install location** with standard `msiexec INSTALLDIR=…` / NSIS `/D=`
flags. **Secrets are never read from this file** — API keys and tokens stay in the OS credential store.
Full key reference and silent-install examples in
[`docs/enterprise-deployment.md`](docs/enterprise-deployment.md).
### Refresh hardware
A **Refresh** button in **Settings ▸ Hardware** re-detects your GPU/NPU on the spot — handy after
installing a driver or plugging in an eGPU — no restart needed.
---
## 📦 Install
**Requirements:** Windows 10 or 11 (x64). WhispAssist needs **WebView2** (preinstalled on Windows 11;
the installer fetches it on Windows 10). Importing from a file/URL additionally needs **`ffmpeg`**
(and **`yt-dlp`** for URLs) on your PATH — install those separately.
1. Download **`WhispAssist_0.7.0_x64_en-US.msi`** (or the NSIS **`WhispAssist_0.7.0_x64-setup.exe`**).
2. Run it and accept the UAC prompt. If SmartScreen appears, choose **More info → Run anyway**.
3. Launch **WhispAssist** from the Start menu.
On first run WA picks the best transcription backend (**NPU → NVIDIA → AMD → Intel → CPU**). It runs
without admin rights, does **not** add itself to startup unless you opt in, and keeps all data under
`%LOCALAPPDATA%\WhispAssist`.
Deploying to many machines? See [`docs/enterprise-deployment.md`](docs/enterprise-deployment.md).
## 🔐 Checksums (SHA-256)
```
730676580dbef5c4bb2c155f46a73a5791722139ab144a3fc0286e3a4449c432 WhispAssist_0.7.0_x64_en-US.msi
6a34c16eb6f876fc92c7a79d37414b4fdd2c09f4f40873008149db36cd0a30aa WhispAssist_0.7.0_x64-setup.exe
```
Verify after download:
```powershell
Get-FileHash .\WhispAssist_0.7.0_x64_en-US.msi -Algorithm SHA256
```
---
## Privacy, unchanged
Everything optional is **off by default**. With nothing configured, WhispAssist makes **no content
egress at all**. Recording is opt-in; sync/AI credentials live only in the OS credential store; the
MCP server is loopback-only and adds no egress; the deployment file never carries secrets. The
reachable-host allowlist is derived from your settings and enforced in the core.
+94
View File
@@ -0,0 +1,94 @@
# WhispAssist v0.7.1
**Privacy-first, Windows-native meeting assistant — everything on-device, nothing leaves unless you say so.**
A quality-of-life release focused on **importing meetings, taking notes, and living in the
background**. Adding a meeting from a file or link now runs without freezing the app and shows you
exactly where it's up to; notes gained AI cleanup and quick formatting; and WhispAssist can now mute
your mic mid-meeting and tuck itself into the system tray. No feature here changes the privacy
posture: everything optional stays off-by-default and local-first.
One universal installer (**MSI** and **NSIS**) covers every machine: **Vulkan** for all GPUs
(NVIDIA/AMD/Intel), the Intel **NPU** (OpenVINO), a **DirectML** fallback, and **CPU**.
---
## ✨ New
### Add a meeting — now in the background, with a progress tracker
Importing a recording (a local audio/video file or a YouTube/streaming/direct URL) no longer blocks
the app while it works. Click **Import** and the meeting appears in your list immediately with a
**four-step progress tracker** — *Transcode → Transcribe → Identify speakers → Finalize* — where the
current step pulses and finished steps show how long they took. A 25-minute video that used to lock
the window for ~13 minutes now transcribes quietly in the background.
- **Pick the transcription model** right in the dialog, and see **"Transcribed with …"** on the
finished meeting so you always know how it was produced.
- **One-click links** to download `ffmpeg` and `yt-dlp` (still external, not bundled).
- A failed import stays in your list marked **error** instead of vanishing.
### Better notes
- **AI-enhance** (✨): turn rough notes into clean, structured notes using your local LLM, with a
one-step **Undo**. Off unless you have a local model configured.
- **Slash commands & a formatting toolbar**: type `/todo`, `/h1`, `/quote`, … or use the toolbar for
headings, lists, checkboxes, quotes, and dividers.
- **Fix:** notes no longer show stale text after re-transcribing a meeting — the pane updates in
place, no restart needed.
### Mute your microphone — press **M**
Mute/unmute the mic mid-meeting with the **M** key or the new mic button by the level meter. The mic
channel goes silent (recording, live transcript, and meter) while system/loopback audio keeps
capturing.
### Close to system tray
Closing the window now **keeps WhispAssist running in the background** instead of quitting. Reopen it
from the tray icon; the tray's **Quit** exits fully. On by default — toggle it in
**Settings ▸ Recording ▸ Close to system tray**. (This release also fixes a bug that showed **two**
WhispAssist icons in the tray — there's now just one.)
### Privacy & hardware odds and ends
- **Vault lock card** in **Settings ▸ Privacy**: lock/unlock the encrypted store and change its
password at a glance.
- **Test your audio devices**: a live level meter for your mic and system audio, plus a test tone.
- **Quick hardware stress test**: benchmark the available backends against your installed models and
apply the fastest real-time combination.
- Tidier recording header.
---
## 📦 Install
**Requirements:** Windows 10 or 11 (x64). WhispAssist needs **WebView2** (preinstalled on Windows 11;
the installer fetches it on Windows 10). Importing from a file/URL additionally needs **`ffmpeg`**
(and **`yt-dlp`** for URLs) on your PATH — the Import dialog now links to both.
1. Download **`WhispAssist_0.7.1_x64_en-US.msi`** (or the NSIS **`WhispAssist_0.7.1_x64-setup.exe`**).
2. Run it and accept the UAC prompt. If SmartScreen appears, choose **More info → Run anyway**.
3. Launch **WhispAssist** from the Start menu.
On first run WA picks the best transcription backend (**NPU → NVIDIA → AMD → Intel → CPU**). It runs
without admin rights, does **not** add itself to startup unless you opt in, and keeps all data under
`%LOCALAPPDATA%\WhispAssist`.
Deploying to many machines? See [`docs/enterprise-deployment.md`](docs/enterprise-deployment.md).
## 🔐 Checksums (SHA-256)
```
bd96a059db3658a9bee81161edc709cfc4741ed99d6c66a12194403b339f9369 WhispAssist_0.7.1_x64_en-US.msi
711fa7df618c8fc3f03c18d543abad2288d56f750b532288204e0ec0f9426595 WhispAssist_0.7.1_x64-setup.exe
```
Verify after download:
```powershell
Get-FileHash .\WhispAssist_0.7.1_x64_en-US.msi -Algorithm SHA256
```
---
## Privacy, unchanged
Everything optional is **off by default**. With nothing configured, WhispAssist makes **no content
egress at all**. Recording is opt-in; sync/AI credentials live only in the OS credential store; the
MCP server is loopback-only and adds no egress; the deployment file never carries secrets. The
reachable-host allowlist is derived from your settings and enforced in the core.
+94
View File
@@ -0,0 +1,94 @@
# WhispAssist v0.7.3
**Privacy-first, Windows-native meeting assistant — everything on-device, nothing leaves unless you say so.**
A performance release: on Intel Core Ultra machines, transcription now leaves the **CPU nearly
idle**. The Whisper encoder keeps running on the **NPU**; the decoder — previously the dominant
CPU cost — now decodes with a **KV cache** and runs on the **Intel iGPU**. Nothing here changes
the privacy posture: the one new download is the upgraded transcription model, fetched from the
same source as before.
One universal installer (**MSI** and **NSIS**) covers every machine: **Vulkan** for all GPUs
(NVIDIA/AMD/Intel), the Intel **NPU** (OpenVINO), a **DirectML** fallback, and **CPU**.
---
## ⚡ Faster, cooler transcription on NPU systems
### KV-cache decoding
The NPU engine previously re-processed the entire token sequence for every new token it
generated. It now uses the upgraded *merged* Whisper decoder and caches attention state between
steps, so each step does a small, constant amount of work. This benefits **every** machine using
the NPU/DirectML engine, whatever device the decoder runs on.
### Decoder on the Intel iGPU
On NPU systems the decoder now runs on the **Intel iGPU** via the same OpenVINO runtime already
installed for the NPU — no new runtime download. Division of labor while transcribing:
- **Encoder** → NPU (the short periodic spikes on Task Manager's NPU graph)
- **Decoder** → Intel iGPU
- **CPU** → audio plumbing only
Measured on a Core Ultra 5 135U (base.en): decoding on the iGPU ran **~4.7× faster** than the
same KV-cache decode on the CPU (1.29 s vs 6.10 s for a 19 s clip), with identical transcripts —
and that work no longer occupies the CPU at all.
**Fallbacks:** no usable Intel iGPU → the decoder automatically drops back to the CPU (still
with the KV-cache speedup), and the log says so. Machines with an NVIDIA/AMD GPU can keep using
the Vulkan backend (Settings ▸ Hardware), which runs the whole model on that GPU. The startup
log names both devices: `transcription engine: ONNX base.en (encoder EP: OpenVINO/NPU, decoder
EP: OpenVINO/GPU)`.
### Model update (automatic, one-time)
The engine's decoder artifact changed to the merged export (`decoder_model_merged.onnx`,
~208 MB, same trusted source). WhispAssist fetches it automatically in the background on first
launch after updating; until it arrives, transcription simply uses the whisper.cpp engine, as on
a fresh install. The superseded decoder file is cleaned up automatically.
---
## ✨ Small addition
**Settings ▸ Calendar** now has a one-click **libpst (readpst)** download button — the same style
as the Import dialog's `ffmpeg`/`yt-dlp` buttons — linking to the ezwinports Windows build.
`.pst` calendar import shells out to `readpst`, which isn't bundled; the button just opens the
download page in your browser.
---
## 📦 Install
**Requirements:** Windows 10 or 11 (x64). WhispAssist needs **WebView2** (preinstalled on Windows 11;
the installer fetches it on Windows 10). Importing from a file/URL additionally needs **`ffmpeg`**
(and **`yt-dlp`** for URLs) on your PATH — the Import dialog links to both.
1. Download **`WhispAssist_0.7.3_x64_en-US.msi`** (or the NSIS **`WhispAssist_0.7.3_x64-setup.exe`**).
2. Run it and accept the UAC prompt. If SmartScreen appears, choose **More info → Run anyway**.
3. Launch **WhispAssist** from the Start menu.
On first run WA picks the best transcription backend (**NPU → NVIDIA → AMD → Intel → CPU**). It runs
without admin rights, does **not** add itself to startup unless you opt in, and keeps all data under
`%LOCALAPPDATA%\WhispAssist`.
Deploying to many machines? See [`docs/enterprise-deployment.md`](docs/enterprise-deployment.md).
## 🔐 Checksums (SHA-256)
```
91a32a17addc31fbc00f38966d52402ca0c5e9e7a581a0431c383ca5a60532bd WhispAssist_0.7.3_x64_en-US.msi
764b758544b075b5aefe9dde37b04e4966df528912fb884ed1be5bd57905ecdb WhispAssist_0.7.3_x64-setup.exe
```
Verify after download:
```powershell
Get-FileHash .\WhispAssist_0.7.3_x64_en-US.msi -Algorithm SHA256
```
---
## Privacy, unchanged
Everything optional is **off by default**. With nothing configured, WhispAssist makes **no content
egress at all**. Recording is opt-in; sync/AI credentials live only in the OS credential store; the
MCP server is loopback-only and adds no egress; the deployment file never carries secrets. The
reachable-host allowlist is derived from your settings and enforced in the core.
Binary file not shown.

After

Width:  |  Height:  |  Size: 728 KiB

+8 -4
View File
@@ -84,10 +84,14 @@ issues from feature briefs.
4. Provisional speaker turns (from cheap segmentation) tag segments live; the user may name a 4. Provisional speaker turns (from cheap segmentation) tag segments live; the user may name a
speaker, applied immediately to past/future segments (FR-SPK-2). speaker, applied immediately to past/future segments (FR-SPK-2).
5. The notes view renders Markdown that updates as segments arrive (FR-NOTE-1, FR-NOTE-6). 5. The notes view renders Markdown that updates as segments arrive (FR-NOTE-1, FR-NOTE-6).
6. On `stop_recording`: finalize the WAV; run full `diarization`, align speaker IDs to segments, 6. On `stop_recording`: finalize the WAV; run `diarization`, align speaker IDs to segments,
apply name mappings; persist transcript JSON + metadata via `storage` (FR-STORE-1). If "Record apply name mappings; persist transcript JSON + metadata via `storage` (FR-STORE-1). When the mic
this meeting" is **off**, delete the working WAV **after** the transcript is finalized; if **on**, is on, `audio.wav` is a **dual-channel split** (left = mic, right = loopback; ADR-0005 Phase 3.5,
keep it as `audio.wav` (ADR-0009, FR-REC-1/4). FR-SPK), so attribution is **per-stream**: sherpa clusters the **right channel only** →
`Speaker N`, and "You" comes from **left-channel** voice activity. Blind whole-signal clustering
+ voiceprint is the mic-off (`summed`) fallback. If "Record this meeting" is **off**, delete the
working WAV **after** the transcript is finalized; if **on**, keep it as `audio.wav` (ADR-0009,
FR-REC-1/4).
7. If an LLM provider is configured: `llm` builds a prompt (transcript + metadata + template), 7. If an LLM provider is configured: `llm` builds a prompt (transcript + metadata + template),
streams a summary/decisions/action-items into the summary panel (FR-LLM-2/4). streams a summary/decisions/action-items into the summary panel (FR-LLM-2/4).
8. Action items are parsed and presented for confirmation; confirmed ones persist and may raise 8. Action items are parsed and presented for confirmation; confirmed ones persist and may raise
+24 -1
View File
@@ -14,7 +14,8 @@ Default root: `%LOCALAPPDATA%\WhispAssist\` (user-configurable, FR-STORE-2).
│ └── spk-eres2net.onnx │ └── spk-eres2net.onnx
└── meetings\ └── meetings\
└── <meeting_id>\ # one folder per meeting (uuid) └── <meeting_id>\ # one folder per meeting (uuid)
├── audio.wav # canonical recording — present ONLY if "Record" was on (ADR-0009) ├── audio.wav # canonical recording — present ONLY if "Record" was on (ADR-0009). Mic on → stereo split (L=mic/"You", R=loopback/"Speaker", `audio_layout=split`); mic off → summed. Playback folds to mono; export folds to dual-mono (ADR-0005 Phase 3.5)
├── voiceprint.wav # mic-only 16kHz sample for "You" re-matching on reprocess of a SUMMED recording (FR-SPK); lives/dies with audio.wav
├── transcript.json # canonical transcript (segments+speakers+timings) ├── transcript.json # canonical transcript (segments+speakers+timings)
├── manual_notes.json # raw user-authored notes captured live during recording ├── manual_notes.json # raw user-authored notes captured live during recording
├── notes.md # the final notes document: manual notes + transcript, merged at finalize ├── notes.md # the final notes document: manual notes + transcript, merged at finalize
@@ -86,6 +87,7 @@ CREATE TABLE meetings (
language TEXT, -- detected/selected language code language TEXT, -- detected/selected language code
backend_used TEXT, -- npu|nvidia|amd|intel|cpu backend_used TEXT, -- npu|nvidia|amd|intel|cpu
model_used TEXT, -- e.g. whisper-base model_used TEXT, -- e.g. whisper-base
audio_layout TEXT, -- split|summed|NULL; how audio.wav's channels are laid out (0009, ADR-0005 Phase 3.5)
calendar_event_id TEXT, -- FK -> calendar_events.id (nullable) calendar_event_id TEXT, -- FK -> calendar_events.id (nullable)
template_id TEXT, -- note-template id (T8.1, FR-NOTE-5); catalog is template_id TEXT, -- note-template id (T8.1, FR-NOTE-5); catalog is
-- a built-in Rust list (notes::templates), not a table -- a built-in Rust list (notes::templates), not a table
@@ -380,6 +382,13 @@ segment (the M1 grounding invariant, asserted by the golden-transcript test).
"expose_recordings": false, // never serve .wav unless explicitly true (FR-MCP-3) "expose_recordings": false, // never serve .wav unless explicitly true (FR-MCP-3)
}, },
"privacy": { "encrypt_at_rest": false }, "privacy": { "encrypt_at_rest": false },
// Launch WhispAssist at login (NFR-RES-4). Opt-in, OFF by default. Toggling this
// via `set_auto_start` also writes a per-user `HKCU\...\Run` entry (no admin);
// startup reconciles the OS entry to this flag (e.g. after a reinstall).
"auto_start": false,
// Closing the window hides WhispAssist to the system tray (keep running in background) instead of
// quitting; ON by default. Tray "Quit" is the real exit. Enforced in the Rust on_window_event handler.
"close_to_tray": true,
// Optional MS Graph calendar source (M4.4, T8.9, FR-CAL-6). Opt-in, explicit consent via OAuth // Optional MS Graph calendar source (M4.4, T8.9, FR-CAL-6). Opt-in, explicit consent via OAuth
// PKCE — OFF by default. `credential_ref` points into the OS credential store; the token itself // PKCE — OFF by default. `credential_ref` points into the OS credential store; the token itself
// is never written here (same invariant as sync credentials, FR-SYNC-6). // is never written here (same invariant as sync credentials, FR-SYNC-6).
@@ -387,6 +396,20 @@ segment (the M1 grounding invariant, asserted by the golden-transcript test).
} }
``` ```
## First-run deploy seeding (`wa-defaults.ini`)
For enterprise mass-deployment, the **first** launch on a machine (before `settings.json` exists)
optionally seeds its defaults from an admin-supplied INI. First file found wins:
1. `%PROGRAMDATA%\WhispAssist\wa-defaults.ini` — machine-wide (GPO / SCCM / Intune file copy).
2. `<install dir>\wa-defaults.ini` — the bundled template (shipped fully commented → no-op).
Keys are flat `key = value` matching `settings.json` field names (bools/ints coerced), plus the
special `auto_download_model = true` which fetches the configured `whisper_model` in the background.
**No secrets** — any key containing `key`/`token`/`secret`/`credential`/`password` is ignored; those
live only in the OS credential store. After first run the file is never read again. See
`docs/enterprise-deployment.md` and `src-tauri/src/deploy.rs`.
## Retention & recovery semantics ## Retention & recovery semantics
- **Retention** (FR-STORE-2): a background job deletes whole meeting folders + rows once a meeting - **Retention** (FR-STORE-2): a background job deletes whole meeting folders + rows once a meeting
+25 -3
View File
@@ -23,6 +23,9 @@ start_recording(input: { meetingTitle?: string; calendarEventId?: string; record
stop_recording(input: { meetingId: MeetingId }): MeetingSummaryRef stop_recording(input: { meetingId: MeetingId }): MeetingSummaryRef
pause_recording(input: { meetingId: MeetingId }): void pause_recording(input: { meetingId: MeetingId }): void
resume_recording(input: { meetingId: MeetingId }): void resume_recording(input: { meetingId: MeetingId }): void
// Mute/unmute the mic mid-meeting (FR-CAP-7): mic channel goes silent (recording + transcript + meter),
// loopback keeps capturing. Returns the new muted state; emits recording://mic. Errs if the mic is off.
toggle_microphone_mute(input: { meetingId: MeetingId }): boolean
set_recording_retention(input: { meetingId: MeetingId; record: boolean }): void // toggle mid-meeting (FR-REC-1) set_recording_retention(input: { meetingId: MeetingId; record: boolean }): void // toggle mid-meeting (FR-REC-1)
acknowledge_recording_consent(): void // one-time (FR-REC-2) acknowledge_recording_consent(): void // one-time (FR-REC-2)
@@ -36,8 +39,14 @@ update_live_notes(input: { meetingId: MeetingId; markdown: string }): void
set_segment_note(input: { meetingId: MeetingId; anchorMs: number; text: string }): void set_segment_note(input: { meetingId: MeetingId; anchorMs: number; text: string }): void
// ---- Hardware ---- // ---- Hardware ----
hardware_status(): { backends: BackendInfo[]; active: BackendId; modelSize: string; estRtf: number } hardware_status(): { backends: BackendInfo[]; active: BackendId; modelSize: string; estRtf: number } // re-detects fresh each call — backs the Settings ▸ Hardware "Refresh" button
set_preferred_backend(input: { backend: BackendId | "auto" }): void set_preferred_backend(input: { backend: BackendId | "auto" }): void
// Launch-at-login (NFR-RES-4). Writes/removes a per-user OS Run entry (no admin) and persists auto_start. Opt-in, off by default.
set_auto_start(input: { enabled: boolean }): void
// Device test: opens a mic ("input") or the render device in loopback ("loopback") for a few seconds and streams device://level (no recording, no retained audio). Refused while recording.
monitor_audio_level(input: { kind: "input" | "loopback"; deviceId?: string; durationMs?: number }): void
// Quick stress test: benchmarks each available backend × installed model (≤3 sizes) on a fixed sample, returns per-pair real-time factor + the most-accurate real-time-capable recommendation. Emits stress://progress. Refused while recording.
stress_test_hardware(): { results: { backend: string; model: string; rtf: number; realtime: boolean }[]; recommended: { backend: string; model: string } | null }
// ---- Transcription / models ---- // ---- Transcription / models ----
// language (T8.7, M4.2): omitted reuses the meeting's current language rather than resetting it. // language (T8.7, M4.2): omitted reuses the meeting's current language rather than resetting it.
@@ -79,6 +88,8 @@ export_meeting(input: { meetingId: MeetingId; dest: string; format: "md" | "pdf"
// they enqueue for finalize-trigger targets and pump in the background. SHA-256 dedup means an // they enqueue for finalize-trigger targets and pump in the background. SHA-256 dedup means an
// edit that didn't alter a file uploads nothing. // edit that didn't alter a file uploads nothing.
update_notes(input: { meetingId: MeetingId; markdown: string }): void update_notes(input: { meetingId: MeetingId; markdown: string }): void
// AI-enhance rough notes into structured Markdown grounded in the transcript (Granola-style), via the configured LlmProvider (no new egress). Takes the live buffer, returns the enhanced text WITHOUT persisting — the UI keeps or undoes it. Refused while recording; errors with no provider.
enhance_notes(input: { meetingId: MeetingId; notes: string }): string
// SearchHit = MeetingListItem fields (id, title, started_at, duration_secs, status, tags) + snippet: string // SearchHit = MeetingListItem fields (id, title, started_at, duration_secs, status, tags) + snippet: string
search(input: { query: string }): SearchHit[] // FTS (FR-SEARCH-1) search(input: { query: string }): SearchHit[] // FTS (FR-SEARCH-1)
set_tags(input: { meetingId: MeetingId; tags: string[] }): void set_tags(input: { meetingId: MeetingId; tags: string[] }): void
@@ -93,6 +104,13 @@ bulk_export_meetings(input: { destDir: string; format: "md" | "pdf" | "docx" | "
// folder of them (from a bulk export). Each is reconstructed under a fresh meeting id (original // folder of them (from a bulk export). Each is reconstructed under a fresh meeting id (original
// title/date/duration/speakers/tags/action items preserved). Returns the count imported. // title/date/duration/speakers/tags/action items preserved). Returns the count imported.
import_meeting_bundle(input: { dir: string }): number import_meeting_bundle(input: { dir: string }): number
// Add a meeting from an existing recording: a local audio/video file path or a URL (YouTube/
// streaming page or direct media URL). Needs ffmpeg (+ yt-dlp for URLs) on PATH; neither bundled.
// `model` overrides the Settings whisper model for this import (recorded as meeting.model_used).
// Returns the new meeting id IMMEDIATELY (status "transcribing"); transcode→transcribe→diarize→
// finalize run in the background, streaming import://progress and ending with transcript://finalized.
// A failed import is left in the list with status "error" (not deleted).
import_media(input: { source: string; title?: string; model?: string }): MeetingId
// ---- LLM / AI provider (ADR-0007/0011) ---- // ---- LLM / AI provider (ADR-0007/0011) ----
// provider ∈ ollama | custom | anthropic | openai | off. Hosted-provider API keys are passed to // provider ∈ ollama | custom | anthropic | openai | off. Hosted-provider API keys are passed to
@@ -183,9 +201,11 @@ privacy_self_check(): {
"recording://state" { meetingId, state: "recording"|"paused"|"stopped"|"cancelled", elapsedMs } "recording://state" { meetingId, state: "recording"|"paused"|"stopped"|"cancelled", elapsedMs }
"recording://level" { meetingId, rms: number, peak: number } // waveform (FR-CAP-5) "recording://level" { meetingId, rms: number, peak: number } // waveform (FR-CAP-5)
"recording://device" { meetingId, recovered: boolean, message: string } // capture device change (FR-CAP-6) "recording://device" { meetingId, recovered: boolean, message: string } // capture device change (FR-CAP-6)
"transcript://segment" { meetingId, segment: TranscriptSegment } // live segments (FR-TRX-2) "recording://mic" { meetingId, muted: boolean } // mic mute toggled (FR-CAP-7)
"transcript://segment" { meetingId, segment: TranscriptSegment } // live segments (FR-TRX-2); may re-emit a committed segment with a refined `speaker` — replace by `segment.id`
"transcript://finalized" { meetingId, segmentCount } "transcript://finalized" { meetingId, segmentCount }
"diarization://updated" { meetingId, speakers: SpeakerInfo[] } // after post-pass (FR-SPK) "import://progress" { meetingId, phase: "prepare"|"transcribe"|"diarize"|"finalize", state: "active"|"done"|"error", elapsedMs: number|null, error: string|null } // background import_media tracker
"diarization://updated" { meetingId, speakers: SpeakerInfo[] } // post-pass AND live 15s provisional passes (FR-SPK); carries "You" once the mic voiceprint matches
"llm://token" { meetingId, text } // streamed summary (FR-LLM-4) "llm://token" { meetingId, text } // streamed summary (FR-LLM-4)
"llm://done" { meetingId, summary: SummaryFile } // full summary.json contents, not just a pointer "llm://done" { meetingId, summary: SummaryFile } // full summary.json contents, not just a pointer
"model://progress" { id, receivedBytes, totalBytes } "model://progress" { id, receivedBytes, totalBytes }
@@ -193,6 +213,8 @@ privacy_self_check(): {
"calendar://linked" { ok: boolean, error?: string } // MS Graph OAuth handshake settled (M4.4) "calendar://linked" { ok: boolean, error?: string } // MS Graph OAuth handshake settled (M4.4)
"calendar://progress" { processed, total } // MS Graph import (M4.4) "calendar://progress" { processed, total } // MS Graph import (M4.4)
"hardware://changed" { active: BackendId, reason: string } // fallback occurred (FR-HW-4) "hardware://changed" { active: BackendId, reason: string } // fallback occurred (FR-HW-4)
"device://level" { kind: "input"|"loopback", rms?, peak?, done?: boolean } // Settings device test meter; done=window ended
"stress://progress" { backend: string, model: string } // quick stress test, per pairing benchmarked
"recording://retention" { meetingId, record: boolean } // retention toggled (FR-REC-1/3) "recording://retention" { meetingId, record: boolean } // retention toggled (FR-REC-1/3)
"sync://job" { jobId, meetingId, targetId, artifact, status, bytesSent, bytesTotal } // FR-SYNC-5 "sync://job" { jobId, meetingId, targetId, artifact, status, bytesSent, bytesTotal } // FR-SYNC-5
"sync://done" { meetingId, targetId, uploaded: number, failed: number } "sync://done" { meetingId, targetId, uploaded: number, failed: number }
+17
View File
@@ -44,3 +44,20 @@ Detection mapping:
## Revisit if ## Revisit if
Windows ML supersedes DirectML for our models, or a vendor SDK gives materially better NPU Windows ML supersedes DirectML for our models, or a vendor SDK gives materially better NPU
throughput than the ONNX/DirectML path. throughput than the ONNX/DirectML path.
## Update (2026-07-16): NPU path uses OpenVINO; decoder split off the CPU
As built (T3.4 + follow-up), the NPU tier runs on **ONNX Runtime + the OpenVINO EP**
(`device_type=NPU`), not DirectML — Intel's prebuilt OpenVINO ORT runtime is downloaded on
demand (DirectML remains the *GPU* fallback EP for AMD/Intel in non-Vulkan builds). The engine
splits Whisper across devices:
- **encoder** (fixed shape): OpenVINO **NPU**;
- **decoder** (autoregressive, KV-cache greedy via Optimum's merged export): OpenVINO **GPU**
(Intel iGPU, same runtime bundle) when available, else the CPU EP —
`WA_ONNX_DECODER_DEVICE=cpu|gpu` overrides.
Rationale: the dynamic-shaped decode loop is a poor NPU fit, and pre-KV-cache it re-ran the full
token prefix every step on the CPU — the dominant CPU cost while transcribing. With the merged
decoder + iGPU placement, sustained CPU during transcription is near-idle on NPU+iGPU systems.
The engine logs both EPs at load (`transcription engine: ONNX …`).
+48
View File
@@ -32,6 +32,54 @@ labels in the post-meeting pass; this keeps latency low while improving final ac
- Speaker IDs (`S1`, `S2`, …) are internal and stable per meeting; name mappings live in the DB - Speaker IDs (`S1`, `S2`, …) are internal and stable per meeting; name mappings live in the DB
and are applied at render/export time, never destructively rewritten onto segments. and are applied at render/export time, never destructively rewritten onto segments.
### Phase 3 refinement — per-stream "You" attribution (FR-SPK, 2026-07-14)
The single-pass-over-the-whole-recording model above blind-clusters a **summed mono** signal
(mic + loopback), then guesses which cluster is the user via a voiceprint match. In practice that
clustering is unreliable in both directions on the summed signal — it over-split a 2-speaker call
into 83 clusters at one threshold and merged two clearly distinct voices into one at another — and
the voiceprint can only *label* a cluster, never *create* the separation.
So when the **microphone is enabled**, WA no longer relies on clustering to find the user:
- A live **mic-activity timeline** (`audio::MicActivity`) records, per 100 ms of `audio.wav` frame
time, when the mic was speech-level — captured in the loopback writer, the one place the mic and
loopback exist separately in the recording's own timebase. Those ranges become **"You"** spans
directly (no clustering, no embedding).
- sherpa clustering is then run over the recording with the "You" ranges **masked out** (zeroed),
so it only ever splits the **far side** into `Speaker N`. The mic can never pollute or merge into
a far-side cluster.
- The timeline is persisted as `mic_activity.json` (retained meetings) so `reprocess_transcript`
re-attributes identically without the live capture.
The original blind-clustering + voiceprint pass **remains the fallback** for mic-off recordings and
imports (no timeline). Naming stays uniform (`build_name_map`: "You", then `Speaker 2…`). Segment
IDs and the names-in-DB rule are unchanged; only the *source* of the spans changes.
### Phase 3.5 refinement — dual-channel capture (supersedes Phase 3's timeline, 2026-07-14)
The Phase 3 timeline above reconstructed the mic/far-side split *after the fact* from a summed-mono
`audio.wav` plus a `mic_activity.json` sidecar, masking the mic ranges before clustering. That was
reliable at stop but **fragile on reprocess** (re-aligning a sidecar against a mono mix — it
collapsed to a single speaker) and carried a parallel persistence path.
**Decision:** when the mic is enabled, record `audio.wav` as **stereo with the streams separated —
left = microphone ("You"), right = system/loopback ("Speaker")** instead of summing them. The
separation is then intrinsic to the recording:
- Diarization runs on the **right channel only** → `Speaker N`; "You" comes from **left-channel**
voice activity (`vad_spans`). The mic is never clustered, by construction.
- Reprocess recomputes both from the file — no sidecar, no masking; stop and reprocess agree.
- Transcription still downmixes (L+R) to the same summed mono; playback folds to mono and bundle
export folds to dual-mono so shared/played audio is normal.
- A `meetings.audio_layout` flag (`summed` | `split`, carried in the bundle manifest) distinguishes
recordings; existing `summed` recordings keep the blind-clustering + voiceprint path. On a stereo
render endpoint this is **size-neutral** (the file was already 2ch with the mic summed into both).
This **retires** `MicActivity` / `mic_activity.json` / masked diarization from Phase 3 (see the
retirement note at `attribute_split` in `commands.rs`); the capture-silence handling,
`diarize_samples`, and `build_name_map` naming are retained. Segment IDs and names-in-DB unchanged.
## Revisit if ## Revisit if
A single model gives joint ASR + diarization with better accuracy, or whisper.cpp gains A single model gives joint ASR + diarization with better accuracy, or whisper.cpp gains
production diarization. production diarization.
@@ -0,0 +1,54 @@
# ADR-0012 — Launch-at-login & enterprise deployment defaults
- **Status:** Accepted
- **Date:** 2026-07-14
- **Context source:** User request (2026-07-14) — auto-start on boot; customize an installation
(install location, per-user/all-users, default settings) with native Windows tooling for
mass-deployment.
## Context
Two related needs. (1) Users want WhispAssist to **launch automatically at login**. NFR-RES-4
forbids adding WA to OS startup without explicit consent, so this must be opt-in. (2) An admin
mass-deploying WA to many machines wants to **customize the deployment** — install location, whether
it installs per-user (no admin) or all-users, and the app's default settings (record-by-default,
preferred backend, a model to pre-download, retention, AI provider) — using **native Windows tools**
(GPO / SCCM / Intune / `msiexec` / silent NSIS), not a bespoke management console.
## Decision
1. **Launch-at-login is opt-in, off by default.** A `set_auto_start` command uses
`tauri-plugin-autostart` to write a **per-user** `HKCU\...\Run` entry (no admin) and persists an
`auto_start` setting. Startup reconciles the OS entry to that flag (restores it after a reinstall).
Nothing runs on a timer — this is a registry entry, not a background process (NFR-RES-1).
2. **Install location & scope are native, no app code.**
- Location: `msiexec INSTALLDIR=…` (MSI) / NSIS `/D=…` (silent).
- Scope: NSIS `installMode: "both"` — the `.exe` setup lets the user choose **current-user
(no admin)** or **all-users (admin)**. The MSI stays per-machine as the enterprise all-users
artifact.
3. **Default settings via a first-run `wa-defaults.ini`.** On a machine's **first** launch (before
`settings.json` exists) WA reads an admin-supplied INI — `%PROGRAMDATA%\WhispAssist\wa-defaults.ini`
first, else the bundled `<install dir>\wa-defaults.ini` — and seeds `settings.json` from it, with an
optional `auto_download_model` to pre-fetch the model. The shipped template is fully commented, so
a normal install is unaffected. This is deployable purely by copying a file with existing Windows
management tooling; no WiX custom actions.
## Consequences
- **Positive:** opt-in startup honors NFR-RES-4 with zero idle cost; install location/scope reuse the
installers' native behavior (no custom code to maintain); one small INI + a first-run guard covers
the whole deployment-customization surface and works for MSI, NSIS, and portable copies alike.
- **Guardrail — no secrets in the deploy file (CLAUDE.md):** the INI must never carry credentials.
`deploy.rs` drops any key containing `key`/`token`/`secret`/`credential`/`password` as defense in
depth; API keys, OAuth tokens and sync passwords remain in the OS credential store only. Seeding a
provider (e.g. `llm_provider=anthropic`) still requires the admin/user to provision its key
separately — no new egress path is created by the file.
- **Negative / care:** the seed runs only when `settings.json` is absent (truly first run); it does
**not** re-apply on upgrade, matching "the user's own settings win thereafter." Array config merges
in Tauri **replace** rather than append, so `wa-defaults.ini` must be listed in both
`tauri.conf.json` and `tauri.vulkan.conf.json` bundle resources (the shipped build uses the latter).
## Revisit if
Admins need per-machine policy that **overrides** user settings on every launch (not just seeds
first-run), or a signed/locked-down enterprise policy channel beyond a plain INI.
+101
View File
@@ -0,0 +1,101 @@
# Enterprise deployment
How to mass-deploy WhispAssist and preset its defaults with native Windows tooling (Group Policy,
SCCM, Intune, `msiexec`, silent NSIS). No management console, no phone-home. See ADR-0012.
WhispAssist ships two bundles:
| Bundle | Scope | Admin? |
|---|---|---|
| `WhispAssist_<ver>_x64_en-US.msi` | Per-machine (all users) | Yes |
| `WhispAssist_<ver>_x64-setup.exe` (NSIS) | Current-user **or** all-users (prompts) | Only for all-users |
## Install location
- **MSI:** `msiexec /i WhispAssist_<ver>_x64_en-US.msi INSTALLDIR="D:\Apps\WhispAssist" /qn`
(`INSTALLDIR` is Tauri's WiX install-dir property; confirm against the generated `.wxs` if a build
changes it.)
- **NSIS:** `WhispAssist_<ver>_x64-setup.exe /S /D=D:\Apps\WhispAssist`
(`/S` = silent, `/D=` = install dir; `/D=` must be **last** and unquoted per NSIS.)
## Install scope (per-user vs all-users)
The NSIS `.exe` shows a "current user / all users" page. **Current user needs no admin** and installs
under the user profile; **all users** requires elevation. Silent all-users:
`WhispAssist_<ver>_x64-setup.exe /S`. The MSI is always per-machine (all-users) and requires admin.
## Auto-start at login
Off by default (NFR-RES-4). Turn it on for the user either in-app (Settings ▸ Recording ▸ *Launch
WhispAssist at login*) or by presetting `auto_start = true` in `wa-defaults.ini` (below). It installs
a **per-user** `HKCU\Software\Microsoft\Windows\CurrentVersion\Run` entry — no admin, and it does
**not** start recording on its own.
## Preset default settings — `wa-defaults.ini`
On a machine's **first** launch (before `settings.json` exists), WhispAssist reads an admin-supplied
INI and seeds that user's `settings.json`. After that the user's own settings win and the file is
ignored. First location found wins:
1. `%PROGRAMDATA%\WhispAssist\wa-defaults.ini` — machine-wide. Deploy with a GPO/SCCM/Intune file copy.
2. `<install dir>\wa-defaults.ini` — the template shipped next to the executable.
The shipped template is fully commented out, so a default install behaves as if it were absent.
Uncomment and edit the keys you want to preset.
### Format
Flat `key = value`, one per line. `;` and `#` comment lines and `[section]` headers are ignored.
`true`/`false` become switches, plain numbers become numbers, everything else is text. Unknown or
misspelled keys are ignored.
> **Never put secrets in this file.** API keys, OAuth tokens and sync passwords live only in the OS
> credential store. Any key containing `key`, `token`, `secret`, `credential` or `password` is
> dropped on read. Presetting `llm_provider = anthropic` still requires the key to be provisioned
> separately — the file adds no egress path.
### Keys
| Key | Values | Meaning |
|---|---|---|
| `default_record` | true/false | Record every meeting by default (consent notice still applies). |
| `preferred_backend` | auto\|npu\|nvidia\|amd\|intel\|cpu | Transcription backend. |
| `whisper_model` | catalog id (e.g. `base.en-q5_1`) | Default transcription model. |
| `auto_download_model` | true/false | Fetch `whisper_model` in the background on first launch. |
| `whisper_language` | auto\|ISO-639-1 | Default language (multilingual model only). |
| `low_overhead` | true/false | CPU + smallest model preset. |
| `storage_root` | path | Where meetings are stored. |
| `retention_max_age_days` | number | Delete meetings older than N days. |
| `retention_max_size_gb` | number | Cap total storage at N GB. |
| `llm_provider` | ollama\|custom\|anthropic\|off | Summary provider (key provisioned separately). |
| `llm_endpoint` | url | LLM endpoint. |
| `llm_model` | text | LLM model name. |
| `microphone_enabled` | true/false | Capture the user's mic into the transcript. |
| `auto_record_calendar` | true/false | Auto-start recording on calendar events (app open only). |
| `theme` | system\|light\|dark | UI theme. |
| `auto_start` | true/false | Launch WhispAssist at login (per-user Run entry). |
| `sync_enabled` | true/false | Sync master switch (targets/credentials configured in-app). |
### Example
```ini
default_record = true
preferred_backend = npu
whisper_model = small.en-q5_1
auto_download_model = true
retention_max_age_days = 90
auto_start = true
```
## Silent end-to-end example
```bat
:: 1. Push machine-wide defaults (as SYSTEM via GPO/SCCM)
mkdir "%ProgramData%\WhispAssist"
copy wa-defaults.ini "%ProgramData%\WhispAssist\wa-defaults.ini"
:: 2. Install per-machine, custom location, no UI
msiexec /i WhispAssist_<ver>_x64_en-US.msi INSTALLDIR="C:\Program Files\WhispAssist" /qn
```
Each user's first launch then seeds their `settings.json` from the machine-wide file.
@@ -0,0 +1,243 @@
# Phase 3 design sketch: per-stream speaker attribution (FR-SPK)
Status: **signed off & in progress**, 2026-07-13. Deepens Phase 3 of
[`2026-07-13-diarization-speaker-accuracy.md`](./2026-07-13-diarization-speaker-accuracy.md).
The §§1–8 below are the original sketch (ADR-0005 review + alternatives); the **finalized
design** the user signed off on is at the top and supersedes the RMS-dominance heuristic.
---
## Finalized design (signed off 2026-07-13)
Simpler than the sketch's dominance heuristic, because the user records on **headphones** so the
mic never hears the far side (no bleed):
> **The microphone stream is "You" by definition. Diarization runs on the loopback (speaker)
> audio only, never the mic. Any loopback speaker is "Speaker N".**
Mechanism, all in **`audio.wav` frame time** (so You-spans and sherpa-spans share one clock):
1. **Mic-activity timeline** — captured in the loopback writer, the one place mic and loopback
exist separately in `audio.wav`'s timebase (`capture_loop`, where `bridge.pull()` is folded
in). Per 100 ms, flag whether the pulled mic chunk was speech-level (reuse `VOICE_ENERGY_FLOOR`).
Held in a shared `MicActivity` (sibling to `VoiceSample`); persisted for reprocess.
2. **At stop / reprocess:**
- Collapse active windows → **"You" spans** (dropping sub-`MIN_SPAN_MS` blips).
- **Mask** the You ranges out of `audio.wav` (zero them) and run sherpa on the remainder → far
side splits into **"Speaker N"** spans. Clustering never sees the mic, so the mic can't
pollute or merge with a far-side cluster.
- Merge span lists; `assign_by_overlap` unchanged.
3. **Voiceprint (Phase 1/2) becomes the fallback** for meetings with no timeline (imports, older
recordings). Live-tick voiceprint "You" is retired once the timeline drives labels.
Timebase note: with the capture-silence fix (FR-CAP-7) `audio.wav` is now continuous, so the
frame-indexed timeline aligns cleanly with what sherpa reads back via `read_wav_mono_16k`.
Implementation status (all landed 2026-07-13/14): **(a)** `MicActivity` struct + capture wiring —
done; **(b)** stop-path attribution + masked diarize (`phase3_attribute`) — done; **(c)** persist
`mic_activity.json` + reprocess uses it (imports fall back, no timeline) — done; **(d)** ADR-0005 +
`docs/02`/`03` amended — done. **Not yet done:** empirical user test; and the *live* 15s tick still
uses the Phase 2 voiceprint, not the timeline (a later polish so live labels match the stop result).
---
_Original sketch (ADR-0005 review, risks, alternatives) follows._
Phases 0–2 shipped: cluster explosion tamed (0.7 + 700 ms floor), reprocess re-diarizes with a
persistent `voiceprint.wav`, and live "You" reaches the UI. Those all still treat diarization as
**blind clustering of a summed mono signal**, then guess which cluster is the mic. Phase 3
removes the guess for the mic side by using information WA throws away today.
---
## 1. ADR-0005 review — what it commits us to
ADR-0005 (Accepted, 2026-06-30) decides: sherpa-onnx offline diarization (pyannote segmentation
+ ERes2Net embedding + clustering), behind the `diarization::Diarizer` trait, run as a
**post-processing pass over recorded audio**, aligned to whisper segments by **timestamp
overlap**. Speaker IDs (`S1`…) are internal/stable; names map in the DB, applied at render time,
**never rewritten onto segments**.
What Phase 3 must respect vs. what it may change:
- **Keeps (contract-level):** the `Diarizer` trait, post-pass-over-audio model, `SpeakerSpan`
→ segment alignment by overlap (`assign_by_overlap`), stable internal labels, names-in-DB.
Phase 3 produces *more* `SpeakerSpan`s from a better source; it does not rewrite segments or
move naming.
- **Bends (needs the ADR noted/updated):** ADR-0005 assumes one clustering pass over "the
recorded audio." Phase 3 introduces a **second, non-clustered source of spans** (the mic
activity timeline) and restricts clustering to the **far-side** audio. That is new enough to
warrant an ADR amendment or a short ADR-0005a, because a future reader will otherwise expect
all spans to come from sherpa.
- **ADR's "Revisit if"** (joint ASR+diarization model, or whisper.cpp diarization) is unrelated
— Phase 3 is orthogonal and does not trigger it.
Conclusion: Phase 3 fits inside the trait and the overlap-alignment contract. The only doc debt
is recording that "mic-dominant spans bypass clustering," which is a genuine deviation from
ADR-0005's single-source assumption. **Recommend: amend ADR-0005** (Consequences section) rather
than a new ADR — same decision, refined.
---
## 2. The asset we currently discard
When the mic is enabled (FR-CAP-7), `start_recording` (`commands.rs:342`) wires two independent
16 kHz-mono streams into `spawn_mixer` (`audio/mod.rs:1020`). The `Mixer` (`audio/mod.rs:979`)
holds `loopback` and `mic` **time-aligned in the same buffer** and sums them sample-for-sample
into the transcript stream — then the per-stream identity is **gone**. Diarization later reads
`audio.wav` (the summed signal) and has to reverse-engineer which cluster was the mic. That
reversal is the entire reason `voiceprint.rs` exists.
Two facts make this cheap to exploit:
1. **Per-stream RMS already exists.** `audio_level(mono, mic)` (`audio/mod.rs:522`) runs on every
WASAPI chunk for *both* directions (`is_loopback` at :564, emitted at :678). We already know,
per ~10 ms chunk, how loud each side is. We just don't persist it against time.
2. **The two streams are already aligned at the mixer**, in the **same timebase the transcript
segments derive from** (the streaming worker consumes the mixer's summed output; segment
timestamps are cumulative-samples-fed / 16). So a timeline indexed by cumulative mixer-output
samples shares the segments' clock — tighter than sherpa spans, which come from `audio.wav`.
---
## 3. Core idea
> The mic stream is a **known** speaker. Don't cluster to find it — record when it's dominant,
> call those spans "You", and run sherpa only on the **far-side** audio to split the *other*
> participants.
Pipeline at stop (mic enabled + retained):
```
mixer ─┬─► loopback samples ──► [far-side WAV] ──► sherpa cluster ──► Speaker 2..N spans
│ │
└─► mic vs loopback RMS per 100 ms ──► mic-dominant ranges ──► "You" spans
│
merge span lists ──► assign_by_overlap ──► segments
```
The mic-dominant spans need no embedding, no clustering, no voiceprint — they are attribution by
construction. Voiceprint (Phases 1–2) stays as the **fallback** for meetings without a timeline
(imports, pre-Phase-3 recordings, mic-disabled meetings).
---
## 4. The two real risks (why this needs sign-off, not just a ticket)
### 4a. Acoustic bleed — the mic is not purely "You"
Without echo cancellation (WA has none — fully local, minimal), a user on **speakers** (not
headphones) has the far side playing into the room and back into the mic. Naive "mic has signal →
You" would attribute the far side to the user whenever the far side is loud.
Mitigation (this is why the parent plan says *mic-**dominant**,* not *mic-active*): a range is
"You" only when **mic RMS meaningfully exceeds loopback RMS** in that window (e.g.
`mic_rms > k · loopback_rms` with `k ≈ 2`, plus an absolute mic-VAD floor so silence isn't
"You"). When both are comparably loud → treat as far-side/ambiguous, let sherpa/overlap decide.
This degrades gracefully: headphone users get near-perfect mic isolation; speaker users get
"You" only on clear self-speech and fall back to the old behavior during overlap. **`k` is a
calibration knob, not a constant to bury** — real rooms differ.
### 4b. Timebase — three clocks, currently reconciled by tolerance
- **Segments** are in *mixer-output* time (streaming worker over the summed stream).
- **sherpa spans** are in *`audio.wav`* time (loopback thread's byte-accurate WAV, mic bridged in
at native rate then resampled to 16 kHz on read).
- The **new mic-timeline** would be in *mixer-output* time.
Today `assign_by_overlap` already bridges segments↔sherpa-spans across the first two clocks and
tolerates the small skew. Phase 3 merges a *third* source. The mic-timeline is actually the
**closest** to segment time (shared origin), so aligning You-spans to segments is *more* reliable
than sherpa's. The residual risk is only that You-spans (mixer time) and Speaker-2..N spans
(`audio.wav` time) coexist in one merged list with a small relative skew at the boundaries. In
practice that costs at most a fraction of a `MIN_SPAN_MS` at each turn boundary — below segment
granularity. **Verification item, not a blocker.** If it proves visible, the clean fix is to run
the far-side sherpa pass over the mixer's *loopback lane* buffered to a WAV rather than over
`audio.wav`, putting everything on one clock; heavier, so deferred until measured.
---
## 5. Concrete design
**Capture (live, near-zero cost).** Add a mic-activity accumulator the mixer writes to. In
`Mixer::drain_ready` (or the `mixer_loop`), for each emitted window compute `mic_rms` and
`loopback_rms` over the aligned overlap (samples are already in hand there) and append a coarse
flag per 100 ms:
```rust
// audio/mod.rs — new, alongside Mixer
struct MicActivity { flags: Vec<bool>, /* one per 100 ms of mixer output */ }
// ponytail: Vec<bool> at 10 Hz = 10 bytes/s ≈ 36 KB/hour. No ring buffer, no config.
```
Expose it via the same `Arc` sharing pattern `VoiceSample` uses so `stop_recording` can read it.
Reuse `audio_level`'s RMS math — do **not** add a second RMS implementation.
**Persist (retained meetings).** At stop, alongside `voiceprint.wav`, write the timeline as
`mic_activity.json` (or pack into an existing artifact) so **reprocess** can reuse it — same
ADR-0009 retention/consent gate as `voiceprint.wav`, same lives-and-dies-with-`audio.wav` rule
(delete it everywhere `audio.wav` is deleted). Tiny, plaintext-or-sealed to match.
**Attribute (stop + reprocess).**
1. Collapse the flag timeline into `You` `SpeakerSpan`s (merge adjacent flags, drop runs
`< MIN_SPAN_MS`, reuse Phase 0's floor).
2. Build a **far-side-only** 16 kHz signal: zero out (or excise) mic-dominant ranges from the
loopback lane, run the existing `SherpaDiarizer::diarize` over it → `Speaker 2..N` spans
(label numbering starts at 2, mirroring `voiceprint::build_name_map`).
3. Merge the two span lists; `assign_by_overlap` unchanged.
4. Fallback chain: no timeline → today's Phase-1/2 voiceprint path; no models → raw `S1`.
**Contract/doc changes.**
- Amend **ADR-0005** Consequences: "mic-dominant spans are attributed directly from a per-stream
activity timeline and bypass clustering; sherpa runs on the far-side residual only."
- `docs/02-architecture.md` data-flow: note the mixer emits a mic-activity side-channel.
- `docs/03-data-model.md`: add `mic_activity.json` to the meeting-dir list (like `voiceprint.wav`).
- No new egress; no new default-on setting; `k` threshold lives in code with a calibration comment.
**Tests.**
- Unit: flag-run → span collapse (boundaries, sub-`MIN_SPAN_MS` drop) — pure, table-driven.
- Unit: dominance classifier (`mic`, `loop` RMS pairs → You / far / ambiguous), incl. the bleed
case (both loud → not You).
- Manual: 2-person call on **headphones** → exactly "You" + 1 speaker, near-zero errors; repeat
on **speakers** → verify bleed doesn't mislabel far-side as You.
---
## 6. Alternatives considered (and why not)
- **Do nothing / stop at Phase 2.** Legitimate. Phases 0–2 already fix the reported bugs
(83-cluster explosion, reprocess collapse, live "You"). Phase 3 is an **accuracy ceiling
raise**, not a bug fix. *If 2-person calls are the dominant case and Phase 2's voiceprint "You"
tests well in the field, Phase 3 may not clear the cost/benefit bar — measure Phase 2 first.*
This is the ponytail-honest recommendation: **verify Phase 2 empirically before building 3.**
- **"Mic active → You" (no dominance test).** Simpler, but wrong under speaker bleed (4a). The
dominance test is the minimum that survives real rooms.
- **Full source separation / AEC.** Much larger, new deps, defeats "minimal + local." The
dominance heuristic gets ~90 % of the benefit for ~5 % of the effort.
- **Better voiceprint only** (bigger sample, re-match every tick — already done in Phase 2).
Still bottlenecked on sherpa's ability to *form* a clean mic cluster in the summed signal;
Phase 3's point is to stop relying on that.
---
## 7. Open questions for sign-off
1. **Build it now, or verify Phase 2 first?** Recommendation: run the empirical 2-person
re-diarization (outstanding for Phases 0–2) and a Phase-2 live "You" check **before**
committing to Phase 3. If Phase 2 accuracy is acceptable, Phase 3 becomes optional.
2. **Dominance threshold `k` and mic-VAD floor** — accept a code-constant default with a
calibration comment (proposed), or expose a hidden setting? Recommendation: constant first.
3. **Far-side signal construction** — zero-out mic-dominant ranges in the loopback lane (simple,
preserves timeline) vs. excise-and-concat (shorter audio, shifts timestamps → needs remap).
Recommendation: **zero-out**, keeps one timebase.
4. **ADR: amend 0005 vs. new ADR-0005a?** Recommendation: amend.
5. **Scope of `mic_activity.json`** — persist for reprocess (proposed) or compute live-only and
accept that reprocess falls back to voiceprint? Persisting is cheap and keeps reprocess at
parity.
## 8. Rough effort
Backend-only until the far-side pass; no frontend change (labels already flow via Phase 2). Est.
~1–1.5 days: mixer side-channel + accumulator (½ day), span collapse + far-side pass + merge
(½ day), persistence + reprocess wiring + docs/tests (½ day). Contained, reversible, behind the
existing mic-enabled + retention gates.
@@ -0,0 +1,137 @@
# Fix plan: speaker diarization accuracy ("You", live labels, 83-speaker explosion)
Status: planned 2026-07-13 (v0.5.2). Investigation confirmed in code; see memanto memories
`a9e5f29e` (diagnosis) and the entry referencing this file (fix order). Execute phases **in
order** — each phase is independently shippable and verifiable, and later phases assume
earlier ones landed.
## Symptoms (user report, all reproduced against code)
1. Live recording shows `S1` for every line; the user's own speech should show "You".
2. Post-stop diarization of a 2-person conversation produced **83** speakers.
3. Re-transcribing (English Medium) collapsed every segment to speaker "You".
## Root causes (verified anchors)
| # | Cause | Anchor |
|---|-------|--------|
| A | Live segments always carry the pre-diarization `"S1"` placeholder | `src-tauri/src/transcription/mod.rs:222`, emitted at `src-tauri/src/commands.rs:480` |
| B | The 15s provisional diarization tick relabels only the backend buffer and emits `diarization://updated`, which **no frontend code listens to** (`src/lib/api.ts` has no listener); live view renders raw labels with no name map (`src/lib/views/TranscriptNotes.svelte:527`) | `src-tauri/src/commands.rs:509-567` |
| C | "You" (mic voiceprint match) runs only in `stop_recording`, never live | `src-tauri/src/commands.rs:666-695`, `src-tauri/src/diarization/voiceprint.rs` |
| D | `reprocess_transcript` never diarizes: fresh segments all default to `"S1"`, then the meeting's **stale** name map (`S1 → "You"` from the original run) is reused, so the whole transcript renders "You" | `src-tauri/src/commands.rs:1646-1742` |
| E | Clustering over the **summed mic+loopback mono** WAV with `num_clusters: -1, threshold: 0.5` over-clusters badly (overlapped speech → mixed embeddings; short pyannote chunks → unstable ERes2Net embeddings) | `src-tauri/src/diarization/mod.rs:68-75` |
---
## Phase 0 — Tame the cluster explosion (do first; everything else is useless at 83 clusters)
Smallest possible change, lands before Phase 1 so re-diarization doesn't reproduce the
83-speaker mess.
1. In `SherpaDiarizer::new` (`src-tauri/src/diarization/mod.rs:73`) raise `threshold` from
`0.5` to `0.7` as a code constant with a comment naming the tuning evidence (83 clusters
for 2 speakers at 0.5 on mixed mono audio). sherpa fast-clustering semantics: larger
threshold → fewer clusters. Do **not** add a settings knob yet.
2. In `segment_to_span`'s caller (`SherpaDiarizer::diarize`), drop spans shorter than
700 ms before returning — sub-second chunks carry unstable embeddings and only cause
label churn in `assign_by_overlap` (which already keeps a segment's prior label when no
span overlaps).
3. Unit test: spans under the minimum are filtered; existing overlap tests still pass.
4. **Verify empirically**: re-run diarization over the user's retained 2-person recording
(the 0.5.2 test meeting) and confirm the cluster count lands near 2–4, not 83. If 0.7
still over-clusters, try 0.8 before considering per-meeting configurability.
Acceptance: 2-person retained recording diarizes to ≤4 speakers.
## Phase 1 — `reprocess_transcript` re-diarizes + persistent mic voiceprint (fixes "everything is You")
1. **Persist the voiceprint at stop.** In `stop_recording`
(`src-tauri/src/commands.rs` after the transcription worker join), when
`session.mic_voice_sample` exists **and** `session.retention` is on, write the mic
sample as 16 kHz mono `voiceprint.wav` next to `audio.wav` in `meeting_dir(&meeting_id)`.
ADR-0009 gate: it is retained audio of the user's voice, so it lives and dies with
`audio.wav` — every code path that deletes `audio.wav` (retention off at finalize,
delete-recording command) must delete `voiceprint.wav` too. Update
`docs/03-data-model.md` (meeting dir file list).
2. **Re-diarize on reprocess.** In `reprocess_transcript` (`src-tauri/src/commands.rs:1646`),
after `transcribe_file`: build the diarizer via `diarizer_from_installed_models`
(inside `spawn_blocking`, same as `stop_recording`), `diarize(&wav_path)`, then
`assign(&mut segments, &spans)`. Missing models → skip gracefully (same degradation as
live).
3. **Rebuild the name map instead of reusing the stale one.** Old labels are meaningless
after re-clustering. If `voiceprint.wav` exists, run
`voiceprint::match_mic_speaker` against the new spans to get a fresh
`You`/`Speaker N` map; otherwise use an empty map (raw `S1…` labels).
Pass `speaker_infos_from_segments(&segments, &new_names)` to `finalize_meeting`
instead of `meeting.speakers` (`src-tauri/src/commands.rs:1713`), and persist the new
names via the store the same way `stop_recording` does. Deliberate policy: user-typed
names from the original run are dropped on reprocess because they key to dead labels —
note this in the command's doc comment.
4. `import_media` reuses this path (`commands.rs:1825` "same batch path"); confirm imported
meetings get diarized speakers too (they have no voiceprint — expect raw labels).
5. Tests: unit test the fresh-name-map policy (stale map not reused); manual: re-transcribe
the 0.5.2 meeting with English Medium → segments show distinct speakers, the user's own
lines show "You".
Acceptance: re-transcription yields per-speaker labels again, with "You" on the mic
speaker when a voiceprint exists; never a single-speaker collapse.
## Phase 2 — Live labels reach the UI, including live "You" (fixes "S1 for everyone")
Backend (`src-tauri/src/commands.rs`, the 15s tick at 509–567):
1. After `diarizer.assign` in the tick, **re-emit every committed segment whose speaker
changed** via the existing `transcript://segment` event (ids are stable; the frontend
store already replaces by id — `src/lib/stores/recording.svelte.ts:49-53`). No new
event needed for relabeling.
2. In the same tick, run `voiceprint::match_mic_speaker` with
`session.mic_voice_sample.samples()` against the fresh spans (clusters re-shuffle every
tick, so match every tick; candidate audio is already capped at 10 s per cluster).
Merge results into `session.speaker_names` **without overwriting user-set names** —
same already-named guard as the post-stop pass (`commands.rs:676-683`). The tick's
existing `diarization://updated` emit then carries the "You" display name.
3. Frontend: add an `onDiarizationUpdated` listener to `src/lib/api.ts` (payload
`{ meetingId, speakers: SpeakerInfo[] }`, already documented in
`docs/04-api-contracts.md:188`). Recording store gains a `speakers` state updated by
it; live rendering at `TranscriptNotes.svelte:527` passes that list to `speakerName`
(the finalized path at :344 already does this).
4. Docs: note in `docs/04-api-contracts.md` that `transcript://segment` may re-emit a
committed segment with an updated `speaker` (replace-by-id contract).
5. Tests: frontend store test — a re-emitted segment with the same id replaces the old
one; manual: during a live 2-person call, own speech flips to "You" within ~15–30 s.
Acceptance: during recording, labels differentiate speakers and the mic speaker shows
"You" while talking (within one tick), not just after stop.
## Phase 3 — Per-stream attribution (accuracy end-game, larger change, needs design sign-off)
Blind clustering of a summed mono signal is the ceiling on accuracy. WA knows which
samples are mic before `spawn_mixer` sums them (`commands.rs:342-371`,
`src-tauri/src/audio/mod.rs` MicBridge/mixer). Design sketch — do NOT start without
reviewing ADR-0005 and getting sign-off, since it changes the diarization contract:
- Record a coarse mic-activity timeline during capture (e.g. per-100 ms mic-RMS-dominant
flags, negligible memory).
- At stop: mic-dominant ranges become "You" spans directly; run sherpa only over the
remaining (far-side) ranges to split the *other* participants; merge span lists before
`assign_by_overlap`.
- Voiceprint match remains as fallback for meetings without the timeline (imports, old
recordings).
- Update ADR-0005 (or add a new ADR) + `docs/02-architecture.md` data flow.
Acceptance: 2-person call yields exactly "You" + 1 speaker with near-zero attribution
errors on non-overlapping speech; overlapping speech attributes to the dominant stream.
---
## Cross-cutting rules for the executing agent
- CLAUDE.md applies in full: `cargo fmt` + `cargo clippy -- -D warnings`, no
`unwrap()`/`expect()` on user-reachable paths, conventional commits referencing
FR-SPK-*, commit each file right after finishing it (one commit per file), docs updated
in the same change when contracts move.
- Memory discipline: `memanto agent activate whispassist` first; `memanto remember` every
decision/tuning result (especially the empirical threshold from Phase 0) with full
metadata; record phase completion so the next agent knows where to resume.
- No new egress, no new settings defaults ON; `voiceprint.wav` is local retained audio
under the existing ADR-0009 consent/retention gate.
@@ -0,0 +1,150 @@
# Plan: dual-channel capture (mic = left, loopback = right) — FR-SPK / FR-CAP
Status: **implemented** 2026-07-14 (all 3 sign-off answers: DB column; retire MicActivity with a
code note; export → dual-mono). Migration 0009 + `write_wav_split` + `attribute_split` +
`fold_wav` + ADR-0005 Phase 3.5. Supersedes the summed-mono + masked-diarize approach
in [`2026-07-13-diarization-phase3-per-stream-attribution.md`](./2026-07-13-diarization-phase3-per-stream-attribution.md).
The proposed **ADR-0005 amendment** is at the bottom — apply it on approval.
## Motivation
Every diarization problem this month traces to one root: **`audio.wav` is a summed mono mix** of
mic + loopback, so the two sources are irrecoverable after the fact. Phase 3 works around that by
(a) recording a separate `mic_activity.json` "You" timeline and (b) diarizing the mix with the You
ranges masked out. It works at stop but is fragile on **reprocess** — the persisted timeline has to
re-align against freshly-transcribed segments over a mono mix, and it currently collapses to a
single "Speaker 2". The timelines are being written correctly (verified: real `you_spans`), so the
failure is the masked-mono re-alignment itself, not the capture.
**Fix the root, not the instance:** keep both sources *physically separate in the recording* by
recording **stereo — left channel = microphone ("You"), right channel = system/loopback
("Speaker")**. The separation then lives in the file forever; reprocess and diarization always have
both streams; no sidecar timeline, no masking, no summed-mono clustering.
## Key facts that make this cheap
- `audio.wav` is **already 2-channel** on a normal (stereo) render endpoint — today we *sum the mic
into both channels*. Writing mic→L and loopback(downmixed)→R is the **same channel count, same
bytes** — no size increase in the common case.
- Transcription already downmixes to mono: `read_wav_mono_16k` averaging L+R of a split file yields
`(mic + loopback)/2` = the same summed signal it gets today, so **transcription is unchanged**.
- The mic↔loopback alignment machinery (`MicBridge`, the drain fix, the capture-silence fix) carries
over unchanged — we just route the aligned samples to two channels instead of summing them.
## Design
### 1. Capture write path (`audio`)
- Force the recording spec to **2ch, 16-bit, native rate** when the mic is enabled ("split"
layout). Loopback-only (mic off) keeps today's behavior.
- New writer (replacing `write_wav_bytes`'s summing for split recordings): per frame,
**L = mic sample** (0 when the mic underran), **R = downmix(loopback channels)**.
- Idle path (loopback silent): **L = drained mic, R = 0** — same silence-synthesis + device-paced
drain we just fixed, now writing to L instead of summing.
### 2. Layout versioning (distinguish old "summed" from new "split")
Content alone can't tell a summed-stereo file from a split-stereo one, so mark it explicitly:
- Add `audio_layout TEXT` to the `meetings` row: `'summed'` (default / NULL for all existing rows
and imports) or `'split'` (new mic-enabled recordings).
- Include it in the **`MeetingBundle` manifest** so exported recordings stay interpretable on
another machine.
- Playback and diarization branch on it; old recordings keep every current code path untouched.
### 3. Read helpers (`audio`)
- `read_wav_mono_16k` — **unchanged** (downmix all channels → mono 16k). Used for transcription of
both layouts; on a split file it returns the summed signal, exactly as wanted.
- New `read_wav_channel_16k(path, channel)` — extract one channel → mono 16k. Diarization reads the
**right** channel (far side); "You" detection reads the **left** (mic).
### 4. Diarization / attribution (`commands`) — *simpler* than Phase 3
For a **split** recording, one path replaces the timeline + masking:
```
far_spans = diarizer.diarize_samples( read_wav_channel_16k(wav, RIGHT) ) // clean far side
you_spans = vad_spans( read_wav_channel_16k(wav, LEFT) ) // energy-windowed mic
merge + assign_by_overlap + build_name_map("You") // You + Speaker N
```
- The mic is **never** in the clustering input — structurally, not procedurally.
- `vad_spans` is the `MicActivity` windowing logic (energy floor + 100 ms windows) applied to the L
channel from the file — so **stop and reprocess share one path** and produce identical results.
- **This deletes `MicActivity`, `mic_activity.json` (persist + read + delete), and `mask_ranges`.**
Net: less code than the current Phase 3.
- Mic-off (`'summed'`) recordings keep the whole-signal pass + voiceprint fallback.
### 5. Playback (`commands::serve_recording`)
- A split file plays mic-in-left / computer-in-right — wrong for a listener. On playback, **downmix
L+R → mono** (dual-mono) so both sources come out both ears.
- `serve_recording` already reads+decrypts the whole file into memory; add a downmix-to-mono step
for `'split'` layout and serve that mono WAV (range/seek computed over the transcoded bytes).
`'summed'` files stream as today.
### 6. Migration & portability
- **Existing recordings** (`'summed'`): every current path unchanged. Their reprocess falls back to
the voiceprint match (acceptable — they predate split). We may retire the `MicActivity`/mask code
once split is default; the handful of recent test recordings lose Phase 3 on reprocess, which is
fine.
- **Bundle export**: export **downmixes the canonical split `audio.wav` to mono** so a shared file
plays normally in any player; the working file stays split for WA's own re-processing.
## What this removes / simplifies
| Removed | Replaced by |
|---|---|
| `MicActivity` struct + capture wiring | L-channel VAD read from the file |
| `mic_activity.json` (write/read/delete) | nothing — recompute from L channel |
| `mask_ranges` + masked diarize | diarize the R channel directly |
| stop vs reprocess divergence | one shared split-attribution path |
The capture-silence fix, the device-paced `drain`, `diarize_samples`, and the `build_name_map`
naming all **stay**.
## Costs / risks (none fatal)
- **New code:** split writer, `audio_layout` column + migration + manifest field, channel read
helper, playback downmix, `vad_spans`, and the split-vs-summed branch. Bigger than a patch, but
it *retires* comparable Phase 3 code.
- **Loopback stereo image lost** (R = loopback downmixed to mono) — irrelevant for speech.
- **Mono render endpoint** (rare): 1ch→2ch doubles that recording's size.
- **Raw file plays split** in a third-party player until exported (export downmixes) — minor.
- **Clock drift** between mic/loopback now shows as tiny L/R desync instead of dropped summed
samples — harmless (diarization tolerates it; playback sums it away).
## Open questions for sign-off
1. Layout marker: **DB column** (proposed) vs. a self-describing WAV chunk? DB column is simpler and
travels via the bundle manifest; recommend it.
2. Retire `MicActivity`/`mask_ranges` outright, or keep them for existing `'summed'` recordings'
reprocess? Recommend **retire** (summed → voiceprint fallback) to avoid two live systems.
3. Export downmix to mono — agree that's the right default for shared/synced copies?
## Implementation order
1. `audio_layout` column + `MeetingBundle` field + read on `get_meeting` (no behavior yet).
2. Split writer + force 2ch when mic on; set `audio_layout='split'`. (`audio` + `start_recording`.)
3. `read_wav_channel_16k` + `vad_spans`; split-attribution path in stop + reprocess.
4. Playback downmix for split; bundle-export downmix.
5. Delete `MicActivity` / `mic_activity.json` / `mask_ranges`; update tests.
6. Apply the ADR-0005 amendment + update `docs/02`/`03`.
---
## Proposed ADR-0005 amendment (apply on approval)
> ### Phase 3.5 refinement — dual-channel capture (FR-SPK/FR-CAP, 2026-07-14)
>
> The Phase 3 per-stream attribution (above) reconstructs the mic/far-side split *after the fact*
> from a summed-mono `audio.wav` plus a `mic_activity.json` timeline, then masks the mic ranges
> before clustering. This is reliable at stop but fragile on reprocess (re-aligning a sidecar
> timeline against a mono mix), and it carries a parallel persistence path.
>
> **Decision:** when the microphone is enabled, record `audio.wav` as **stereo with the streams
> separated — left = microphone ("You"), right = system/loopback ("Speaker")** — instead of summing
> them. The separation is then intrinsic to the recording:
> - Diarization runs on the **right channel only** → `Speaker N`; "You" comes from **left-channel**
> voice activity. The mic is never clustered, by construction.
> - Reprocess recomputes both from the file — no sidecar timeline, no masking.
> - Transcription still downmixes (L+R) to the same summed mono it uses today; playback and bundle
> export downmix to mono so shared/played audio is normal.
> - A `meetings.audio_layout` flag (`summed` | `split`, in the bundle manifest for portability)
> distinguishes recordings; existing `summed` recordings keep the original blind-clustering +
> voiceprint path. On a stereo render endpoint this is **size-neutral** (the file was already 2ch
> with the mic summed into both).
>
> This **supersedes** `MicActivity` / `mic_activity.json` / masked diarization from Phase 3; the
> capture-silence handling, `diarize_samples`, and the `build_name_map` naming are retained. Segment
> IDs and the names-in-DB rule are unchanged.
+1 -1
View File
@@ -1,7 +1,7 @@
{ {
"name": "whispassist", "name": "whispassist",
"private": true, "private": true,
"version": "0.5.1", "version": "0.7.3",
"type": "module", "type": "module",
"description": "Privacy-first, fully local Windows meeting assistant.", "description": "Privacy-first, fully local Windows meeting assistant.",
"license": "MIT OR Apache-2.0", "license": "MIT OR Apache-2.0",
+57 -2
View File
@@ -124,6 +124,17 @@ version = "1.1.2"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1505bd5d3d116872e7271a6d4e16d81d0c8570876c8de68093a09ac269d8aac0" checksum = "1505bd5d3d116872e7271a6d4e16d81d0c8570876c8de68093a09ac269d8aac0"
[[package]]
name = "auto-launch"
version = "0.5.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1f012b8cc0c850f34117ec8252a44418f2e34a2cf501de89e29b241ae5f79471"
dependencies = [
"dirs 4.0.0",
"thiserror 1.0.69",
"winreg 0.10.1",
]
[[package]] [[package]]
name = "autocfg" name = "autocfg"
version = "1.5.1" version = "1.5.1"
@@ -848,6 +859,15 @@ dependencies = [
"subtle", "subtle",
] ]
[[package]]
name = "dirs"
version = "4.0.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ca3aa72a6f96ea37bbc5aa912f6788242832f75369bdfdadcb0e38423f100059"
dependencies = [
"dirs-sys 0.3.7",
]
[[package]] [[package]]
name = "dirs" name = "dirs"
version = "5.0.1" version = "5.0.1"
@@ -866,6 +886,17 @@ dependencies = [
"dirs-sys 0.5.0", "dirs-sys 0.5.0",
] ]
[[package]]
name = "dirs-sys"
version = "0.3.7"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1b1d1d91c932ef41c0f2663aa8b0ca0342d444d842c06914aa0a7e352d0bada6"
dependencies = [
"libc",
"redox_users 0.4.6",
"winapi",
]
[[package]] [[package]]
name = "dirs-sys" name = "dirs-sys"
version = "0.4.1" version = "0.4.1"
@@ -1043,7 +1074,7 @@ dependencies = [
"rustc_version", "rustc_version",
"toml 1.1.2+spec-1.1.0", "toml 1.1.2+spec-1.1.0",
"vswhom", "vswhom",
"winreg", "winreg 0.55.0",
] ]
[[package]] [[package]]
@@ -4935,6 +4966,20 @@ dependencies = [
"walkdir", "walkdir",
] ]
[[package]]
name = "tauri-plugin-autostart"
version = "2.5.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "459383cebc193cdd03d1ba4acc40f2c408a7abce419d64bdcd2d745bc2886f70"
dependencies = [
"auto-launch",
"serde",
"serde_json",
"tauri",
"tauri-plugin",
"thiserror 2.0.18",
]
[[package]] [[package]]
name = "tauri-plugin-dialog" name = "tauri-plugin-dialog"
version = "2.7.1" version = "2.7.1"
@@ -6043,7 +6088,7 @@ dependencies = [
[[package]] [[package]]
name = "whispassist" name = "whispassist"
version = "0.5.1" version = "0.7.3"
dependencies = [ dependencies = [
"argon2", "argon2",
"async-trait", "async-trait",
@@ -6073,6 +6118,7 @@ dependencies = [
"sqlx", "sqlx",
"tauri", "tauri",
"tauri-build", "tauri-build",
"tauri-plugin-autostart",
"tauri-plugin-dialog", "tauri-plugin-dialog",
"thiserror 1.0.69", "thiserror 1.0.69",
"tokio", "tokio",
@@ -6767,6 +6813,15 @@ dependencies = [
"memchr", "memchr",
] ]
[[package]]
name = "winreg"
version = "0.10.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "80d0f4e272c85def139476380b12f9ac60926689dd2e01d4923222f40580869d"
dependencies = [
"winapi",
]
[[package]] [[package]]
name = "winreg" name = "winreg"
version = "0.55.0" version = "0.55.0"
+3 -1
View File
@@ -1,6 +1,6 @@
[package] [package]
name = "whispassist" name = "whispassist"
version = "0.5.1" version = "0.7.3"
description = "Privacy-first, fully local Windows meeting assistant" description = "Privacy-first, fully local Windows meeting assistant"
authors = ["WhispAssist contributors"] authors = ["WhispAssist contributors"]
license = "MIT OR Apache-2.0" license = "MIT OR Apache-2.0"
@@ -81,6 +81,7 @@ ort = { version = "=2.0.0-rc.10", optional = true, default-features = false, fea
rustfft = { version = "6", optional = true } rustfft = { version = "6", optional = true }
sherpa-rs = { version = "0.6", optional = true, default-features = false, features = ["download-binaries"] } # sherpa-onnx bindings (Phase 4, ADR-0005) sherpa-rs = { version = "0.6", optional = true, default-features = false, features = ["download-binaries"] } # sherpa-onnx bindings (Phase 4, ADR-0005)
tauri-plugin-dialog = "2" # native Save/choose-folder (Phase 2 export) tauri-plugin-dialog = "2" # native Save/choose-folder (Phase 2 export)
tauri-plugin-autostart = "2" # opt-in launch-on-login (per-user HKCU\Run, no admin; NFR-RES-4)
# notes export (Phase 8, FR-NOTE-4) — pure-Rust, no external binary/cloud # notes export (Phase 8, FR-NOTE-4) — pure-Rust, no external binary/cloud
# conversion service, consistent with the fully-local invariant. # conversion service, consistent with the fully-local invariant.
@@ -99,6 +100,7 @@ windows = { version = "0.58", features = [
"Win32_Devices_DeviceAndDriverInstallation", # SetupAPI: NPU detection (Phase 3, T3.4) "Win32_Devices_DeviceAndDriverInstallation", # SetupAPI: NPU detection (Phase 3, T3.4)
"Win32_System_Com", "Win32_System_Com",
"Win32_UI_Shell", # SetCurrentProcessExplicitAppUserModelID (Phase 8, T8.6) "Win32_UI_Shell", # SetCurrentProcessExplicitAppUserModelID (Phase 8, T8.6)
"Win32_UI_WindowsAndMessaging", # MessageBoxW: fatal-startup error dialog (see run())
"UI_Notifications", # scheduled toast reminders (Phase 8, T8.6, FR-CAL-5) — the "UI_Notifications", # scheduled toast reminders (Phase 8, T8.6, FR-CAL-5) — the
"Data_Xml_Dom", # OS delivers these itself at the due time, no polling timer "Data_Xml_Dom", # OS delivers these itself at the due time, no polling timer
"Foundation", "Foundation",
File diff suppressed because one or more lines are too long
+42
View File
@@ -176,6 +176,48 @@
"Identifier": { "Identifier": {
"description": "Permission identifier", "description": "Permission identifier",
"oneOf": [ "oneOf": [
{
"description": "This permission set configures if your\napplication can enable or disable auto\nstarting the application on boot.\n\n#### Granted Permissions\n\nIt allows all to check, enable and\ndisable the automatic start on boot.\n\n\n#### This default permission set includes:\n\n- `allow-enable`\n- `allow-disable`\n- `allow-is-enabled`",
"type": "string",
"const": "autostart:default",
"markdownDescription": "This permission set configures if your\napplication can enable or disable auto\nstarting the application on boot.\n\n#### Granted Permissions\n\nIt allows all to check, enable and\ndisable the automatic start on boot.\n\n\n#### This default permission set includes:\n\n- `allow-enable`\n- `allow-disable`\n- `allow-is-enabled`"
},
{
"description": "Enables the disable command without any pre-configured scope.",
"type": "string",
"const": "autostart:allow-disable",
"markdownDescription": "Enables the disable command without any pre-configured scope."
},
{
"description": "Enables the enable command without any pre-configured scope.",
"type": "string",
"const": "autostart:allow-enable",
"markdownDescription": "Enables the enable command without any pre-configured scope."
},
{
"description": "Enables the is_enabled command without any pre-configured scope.",
"type": "string",
"const": "autostart:allow-is-enabled",
"markdownDescription": "Enables the is_enabled command without any pre-configured scope."
},
{
"description": "Denies the disable command without any pre-configured scope.",
"type": "string",
"const": "autostart:deny-disable",
"markdownDescription": "Denies the disable command without any pre-configured scope."
},
{
"description": "Denies the enable command without any pre-configured scope.",
"type": "string",
"const": "autostart:deny-enable",
"markdownDescription": "Denies the enable command without any pre-configured scope."
},
{
"description": "Denies the is_enabled command without any pre-configured scope.",
"type": "string",
"const": "autostart:deny-is-enabled",
"markdownDescription": "Denies the is_enabled command without any pre-configured scope."
},
{ {
"description": "Default core plugins set.\n#### This default permission set includes:\n\n- `core:path:default`\n- `core:event:default`\n- `core:window:default`\n- `core:webview:default`\n- `core:app:default`\n- `core:image:default`\n- `core:resources:default`\n- `core:menu:default`\n- `core:tray:default`", "description": "Default core plugins set.\n#### This default permission set includes:\n\n- `core:path:default`\n- `core:event:default`\n- `core:window:default`\n- `core:webview:default`\n- `core:app:default`\n- `core:image:default`\n- `core:resources:default`\n- `core:menu:default`\n- `core:tray:default`",
"type": "string", "type": "string",
+42
View File
@@ -176,6 +176,48 @@
"Identifier": { "Identifier": {
"description": "Permission identifier", "description": "Permission identifier",
"oneOf": [ "oneOf": [
{
"description": "This permission set configures if your\napplication can enable or disable auto\nstarting the application on boot.\n\n#### Granted Permissions\n\nIt allows all to check, enable and\ndisable the automatic start on boot.\n\n\n#### This default permission set includes:\n\n- `allow-enable`\n- `allow-disable`\n- `allow-is-enabled`",
"type": "string",
"const": "autostart:default",
"markdownDescription": "This permission set configures if your\napplication can enable or disable auto\nstarting the application on boot.\n\n#### Granted Permissions\n\nIt allows all to check, enable and\ndisable the automatic start on boot.\n\n\n#### This default permission set includes:\n\n- `allow-enable`\n- `allow-disable`\n- `allow-is-enabled`"
},
{
"description": "Enables the disable command without any pre-configured scope.",
"type": "string",
"const": "autostart:allow-disable",
"markdownDescription": "Enables the disable command without any pre-configured scope."
},
{
"description": "Enables the enable command without any pre-configured scope.",
"type": "string",
"const": "autostart:allow-enable",
"markdownDescription": "Enables the enable command without any pre-configured scope."
},
{
"description": "Enables the is_enabled command without any pre-configured scope.",
"type": "string",
"const": "autostart:allow-is-enabled",
"markdownDescription": "Enables the is_enabled command without any pre-configured scope."
},
{
"description": "Denies the disable command without any pre-configured scope.",
"type": "string",
"const": "autostart:deny-disable",
"markdownDescription": "Denies the disable command without any pre-configured scope."
},
{
"description": "Denies the enable command without any pre-configured scope.",
"type": "string",
"const": "autostart:deny-enable",
"markdownDescription": "Denies the enable command without any pre-configured scope."
},
{
"description": "Denies the is_enabled command without any pre-configured scope.",
"type": "string",
"const": "autostart:deny-is-enabled",
"markdownDescription": "Denies the is_enabled command without any pre-configured scope."
},
{ {
"description": "Default core plugins set.\n#### This default permission set includes:\n\n- `core:path:default`\n- `core:event:default`\n- `core:window:default`\n- `core:webview:default`\n- `core:app:default`\n- `core:image:default`\n- `core:resources:default`\n- `core:menu:default`\n- `core:tray:default`", "description": "Default core plugins set.\n#### This default permission set includes:\n\n- `core:path:default`\n- `core:event:default`\n- `core:window:default`\n- `core:webview:default`\n- `core:app:default`\n- `core:image:default`\n- `core:resources:default`\n- `core:menu:default`\n- `core:tray:default`",
"type": "string", "type": "string",
@@ -0,0 +1,12 @@
-- WhispAssist dual-channel capture (FR-SPK/FR-CAP). Forward-only migration.
-- Mirrors docs/03-data-model.md.
PRAGMA foreign_keys = ON;
-- How audio.wav stores its channels:
-- NULL / 'summed' — mic summed into every loopback channel (all pre-2026-07-14
-- recordings and every import): one mixed mono-equivalent.
-- 'split' — stereo, left = microphone ("You"), right = system/loopback
-- ("Speaker"), kept separate so diarization runs on the far
-- side alone. New mic-enabled recordings.
-- Diarization, playback (downmix), and bundle export branch on this.
ALTER TABLE meetings ADD COLUMN audio_layout TEXT;
+354 -25
View File
@@ -45,9 +45,26 @@ pub enum AudioError {
pub struct CaptureHandle { pub struct CaptureHandle {
running: Arc<AtomicBool>, running: Arc<AtomicBool>,
paused: Arc<AtomicBool>, paused: Arc<AtomicBool>,
/// Mic-only (FR-CAP-7): when set, the microphone stream emits silence instead
/// of real samples — the recording's mic-left channel and the live transcript
/// go quiet, the meter drops to zero, while loopback keeps recording. Toggled
/// live via `set_muted` (the "press M to mute" control).
muted: Arc<AtomicBool>,
thread: JoinHandle<Result<CaptureSummary, AudioError>>, thread: JoinHandle<Result<CaptureSummary, AudioError>>,
} }
impl CaptureHandle {
/// Mute/unmute this stream live. Only meaningful for the microphone capture.
pub fn set_muted(&self, muted: bool) {
self.muted.store(muted, Ordering::SeqCst);
}
/// Whether this stream is currently muted.
pub fn is_muted(&self) -> bool {
self.muted.load(Ordering::SeqCst)
}
}
/// Where captured frames are delivered for live transcription: mono f32 @ 16kHz, /// Where captured frames are delivered for live transcription: mono f32 @ 16kHz,
/// bounded so a slow/absent consumer can never stall the capture thread. /// bounded so a slow/absent consumer can never stall the capture thread.
pub type FrameSink = SyncSender<Vec<f32>>; pub type FrameSink = SyncSender<Vec<f32>>;
@@ -201,16 +218,37 @@ impl MicBridge {
} }
} }
} }
/// Loopback idle path: take everything buffered at once, device-paced. The
/// idle writer must NOT meter the mic by wall-clock — that lets the buffer
/// back up to `cap` and drop the oldest samples, which sounds like the audio
/// skipping/fast-forwarding late in a long silent stretch. Draining fully
/// keeps the mic flowing at its own clock.
fn drain(&self) -> Vec<f32> {
self.buf
.lock()
.map(|mut buf| buf.drain(..).collect())
.unwrap_or_default()
}
} }
/// Below this per-chunk RMS the mic is treated as silence/room noise and not
/// collected into a voiceprint. Without it the voiceprint was just the first
/// few seconds of the recording regardless of whether the user had spoken —
/// so a user silent for the first 30s got a noise voiceprint that spuriously
/// matched a cluster and mislabeled everyone "You". ponytail: fixed floor for
/// normalized [-1,1] speech; promote to a calibration knob if real mics vary.
const VOICE_ENERGY_FLOOR: f32 = 0.01;
/// A one-shot, bounded capture of raw mic-only audio (16kHz mono, same format /// A one-shot, bounded capture of raw mic-only audio (16kHz mono, same format
/// the transcriber and diarizer both use) taken early in a recording — enough /// the transcriber and diarizer both use) taken from the user's early *speech*
/// to compute a voiceprint that identifies which diarized speaker cluster is /// — enough to compute a voiceprint that identifies which diarized speaker
/// the mic (so it can be labeled "You" instead of a clustered "S1"/"S2"; see /// cluster is the mic (so it can be labeled "You" instead of a clustered
/// `diarization::voiceprint`). Unlike `MicBridge`, this is filled once and /// "S1"/"S2"; see `diarization::voiceprint`). Unlike `MicBridge`, this is filled
/// never drained: the first `cap` samples are kept and everything after is /// once and never drained: the first `cap` samples of speech-level audio are
/// dropped, since a voiceprint only needs a few seconds of real speech, not /// kept and everything after is dropped, since a voiceprint only needs a few
/// the whole meeting. /// seconds of real speech, not the whole meeting. Silence is skipped so the
/// window tracks when the user actually talks, not the wall clock.
pub struct VoiceSample { pub struct VoiceSample {
cap: usize, cap: usize,
buf: Mutex<Vec<f32>>, buf: Mutex<Vec<f32>>,
@@ -225,6 +263,18 @@ impl VoiceSample {
} }
fn push(&self, samples: &[f32]) { fn push(&self, samples: &[f32]) {
// Skip silence/room noise so the voiceprint is the user's voice, not
// whatever the mic picked up before they first spoke (see
// VOICE_ENERGY_FLOOR). RMS inline so VoiceSample stays independent of
// the `audio`-gated level helper.
let rms = if samples.is_empty() {
0.0
} else {
(samples.iter().map(|s| s * s).sum::<f32>() / samples.len() as f32).sqrt()
};
if rms < VOICE_ENERGY_FLOOR {
return;
}
if let Ok(mut buf) = self.buf.lock() { if let Ok(mut buf) = self.buf.lock() {
if buf.len() < self.cap { if buf.len() < self.cap {
buf.extend_from_slice(samples); buf.extend_from_slice(samples);
@@ -240,6 +290,12 @@ impl VoiceSample {
} }
} }
// MicActivity (the live mic-speech timeline) was retired 2026-07-14 with the
// masked-mono diarization it fed — the split (mic-left/loopback-right) layout
// makes the mic/far-side separation intrinsic to audio.wav, so "You" is
// recomputed from the left channel via `vad_spans`. See the retirement note in
// commands.rs (attribute_split).
/// Number of audio frames in a raw WASAPI byte buffer of the given format. /// Number of audio frames in a raw WASAPI byte buffer of the given format.
#[cfg(feature = "audio")] #[cfg(feature = "audio")]
fn frame_count(bytes: &[u8], format: &WaveFormat) -> usize { fn frame_count(bytes: &[u8], format: &WaveFormat) -> usize {
@@ -273,12 +329,15 @@ impl WasapiCapture {
event_sink: EventSink, event_sink: EventSink,
bridge: Option<Arc<MicBridge>>, bridge: Option<Arc<MicBridge>>,
voice_sample: Option<Arc<VoiceSample>>, voice_sample: Option<Arc<VoiceSample>>,
split: bool,
emit_level: bool, emit_level: bool,
) -> Result<CaptureHandle, AudioError> { ) -> Result<CaptureHandle, AudioError> {
let running = Arc::new(AtomicBool::new(true)); let running = Arc::new(AtomicBool::new(true));
let paused = Arc::new(AtomicBool::new(false)); let paused = Arc::new(AtomicBool::new(false));
let muted = Arc::new(AtomicBool::new(false));
let running_th = running.clone(); let running_th = running.clone();
let paused_th = paused.clone(); let paused_th = paused.clone();
let muted_th = muted.clone();
let wav_path = wav_path.map(Path::to_path_buf); let wav_path = wav_path.map(Path::to_path_buf);
let device_id = device_id.map(str::to_string); let device_id = device_id.map(str::to_string);
@@ -293,8 +352,10 @@ impl WasapiCapture {
&event_sink, &event_sink,
&running_th, &running_th,
&paused_th, &paused_th,
&muted_th,
bridge.as_ref(), bridge.as_ref(),
voice_sample.as_ref(), voice_sample.as_ref(),
split,
emit_level, emit_level,
) )
}) })
@@ -303,6 +364,7 @@ impl WasapiCapture {
Ok(CaptureHandle { Ok(CaptureHandle {
running, running,
paused, paused,
muted,
thread, thread,
}) })
} }
@@ -327,7 +389,8 @@ impl WasapiCapture {
event_sink, event_sink,
Some(bridge), Some(bridge),
None, None,
true, true, // split: mic-enabled loopback records mic-left / loopback-right
true, // emit_level
) )
} }
@@ -352,7 +415,8 @@ impl WasapiCapture {
event_sink, event_sink,
Some(bridge), Some(bridge),
voice_sample, voice_sample,
true, false, // split: mic stream has no writer
true, // emit_level
) )
} }
} }
@@ -375,7 +439,8 @@ impl AudioCapture for WasapiCapture {
event_sink, event_sink,
None, None,
None, None,
true, false, // split: loopback-only / mic stream — no split writer
true, // emit_level
) )
} }
@@ -394,7 +459,8 @@ impl AudioCapture for WasapiCapture {
event_sink, event_sink,
None, None,
None, None,
true, false, // split: loopback-only / mic stream — no split writer
true, // emit_level
) )
} }
@@ -540,6 +606,18 @@ fn audio_level(mono: &[f32], mic: bool) -> AudioLevel {
/// without flooding the frontend with an event per ~10ms WASAPI callback. /// without flooding the frontend with an event per ~10ms WASAPI callback.
const LEVEL_EMIT_INTERVAL: Duration = Duration::from_millis(50); const LEVEL_EMIT_INTERVAL: Duration = Duration::from_millis(50);
/// When the loopback (render) endpoint is idle, WASAPI delivers no packets, so
/// the recording would stall and the bridged mic — the user talking over system
/// silence — would be lost as the MicBridge overflows. The loopback loop then
/// synthesizes silence for the elapsed gap and folds the mic in, keeping
/// `audio.wav` continuous and complete (FR-CAP-7). MIN gates on a *sustained*
/// gap so brief inter-packet stalls during active playback don't punch silence
/// into the loopback (audible clicks); it stays well under the MicBridge's
/// ~0.5s cap so the mic can't back up and drop samples between drains. MAX caps
/// the silence padding written in one chunk.
const SILENCE_TICK_MIN: Duration = Duration::from_millis(150);
const SILENCE_TICK_MAX: Duration = Duration::from_millis(500);
/// Runs on a dedicated OS thread for the lifetime of a `CaptureHandle`. Owns the /// Runs on a dedicated OS thread for the lifetime of a `CaptureHandle`. Owns the
/// WASAPI client and the WAV writer; exits (and finalizes the WAV) once `running` /// WASAPI client and the WAV writer; exits (and finalizes the WAV) once `running`
/// is cleared. /// is cleared.
@@ -553,8 +631,15 @@ fn capture_loop(
event_sink: &EventSink, event_sink: &EventSink,
running: &AtomicBool, running: &AtomicBool,
paused: &AtomicBool, paused: &AtomicBool,
// Mic-only live mute (FR-CAP-7): zeroes the decoded mic samples so the
// recording, transcript, and meter all go silent while loopback continues.
muted: &AtomicBool,
bridge: Option<&Arc<MicBridge>>, bridge: Option<&Arc<MicBridge>>,
voice_sample: Option<&Arc<VoiceSample>>, voice_sample: Option<&Arc<VoiceSample>>,
// FR-SPK: when true, the loopback WAV is stereo L=mic / R=loopback (the mic
// is a separate track, not summed in) so diarization can run on the far side
// alone. Only meaningful for the loopback writer.
split: bool,
emit_level: bool, emit_level: bool,
) -> Result<CaptureSummary, AudioError> { ) -> Result<CaptureSummary, AudioError> {
wasapi::initialize_mta() wasapi::initialize_mta()
@@ -566,14 +651,18 @@ fn capture_loop(
// UI can overlay them — but only the loopback stream writes a WAV. // UI can overlay them — but only the loopback stream writes a WAV.
let mut session = open_capture_session(&direction, device_id)?; let mut session = open_capture_session(&direction, device_id)?;
// Loopback publishes its rate so the mic knows what to resample to before // Loopback publishes its rate so the mic knows what to resample to before
// pushing into the shared bridge (mic-into-recording, FR-CAP-7). // pushing into the shared bridge (mic-into-recording, FR-CAP-7), and so the
// Phase 3 mic-activity timeline maps frame positions to windows.
if is_loopback { if is_loopback {
if let Some(b) = bridge { if let Some(b) = bridge {
b.rate b.rate
.store(session.format.get_samplespersec(), Ordering::Relaxed); .store(session.format.get_samplespersec(), Ordering::Relaxed);
} }
} }
let spec = wav_spec_for(&session.format)?; let mut spec = wav_spec_for(&session.format)?;
if split && is_loopback {
spec.channels = 2; // L = mic, R = loopback (downmixed)
}
let mut writer = match wav_path { let mut writer = match wav_path {
Some(path) => Some(WavWriter::create(path, spec).map_err(|e| { Some(path) => Some(WavWriter::create(path, spec).map_err(|e| {
AudioError::Capture(format!("could not create {}: {e}", path.display())) AudioError::Capture(format!("could not create {}: {e}", path.display()))
@@ -593,6 +682,9 @@ fn capture_loop(
let mut queue: std::collections::VecDeque<u8> = std::collections::VecDeque::new(); let mut queue: std::collections::VecDeque<u8> = std::collections::VecDeque::new();
let mut frames_written: u64 = 0; let mut frames_written: u64 = 0;
let mut last_level_emit = Instant::now() - LEVEL_EMIT_INTERVAL; let mut last_level_emit = Instant::now() - LEVEL_EMIT_INTERVAL;
// Wall-clock of the last WAV write (real or synthesized), so the loopback
// loop can fill idle gaps with silence+mic instead of stalling.
let mut last_write = Instant::now();
while running.load(Ordering::Relaxed) { while running.load(Ordering::Relaxed) {
// Short timeout so we periodically re-check `running` even with no data. // Short timeout so we periodically re-check `running` even with no data.
@@ -636,8 +728,42 @@ fn capture_loop(
continue; continue;
} }
if queue.is_empty() { if queue.is_empty() {
// Loopback idle (nothing playing): keep audio.wav advancing so the
// bridged mic (user talking over system silence) is preserved
// instead of lost to MicBridge overflow. Only after a *sustained*
// gap, so brief inter-packet stalls during active playback don't
// punch silence into the loopback (clicks). Loopback + mic-enabled
// only; the mic capture stream has no writer.
if is_loopback && !paused.load(Ordering::Relaxed) {
if let (Some(w), Some(b)) = (writer.as_mut(), bridge) {
let elapsed = last_write.elapsed();
if elapsed >= SILENCE_TICK_MIN {
// Drain the whole mic buffer (device-paced). Metering the
// pull by wall-clock let the buffer back up to its cap and
// drop the oldest samples — the audio skipped/fast-
// forwarded late in a long gap. Draining fully avoids that.
let mic = b.drain();
let rate = session.format.get_samplespersec() as usize;
// Advance the timeline by the wall-clock gap (capped), but
// never fewer frames than the mic we must write.
let gap_frames =
(elapsed.min(SILENCE_TICK_MAX).as_micros() as usize * rate / 1_000_000)
.max(mic.len());
let bpf = session.format.get_nchannels() as usize
* (session.format.get_bitspersample() as usize / 8);
let silence = vec![0u8; gap_frames * bpf];
frames_written += if split {
write_wav_split(w, &silence, &session.format, &mic)?
} else {
write_wav_bytes(w, &silence, &session.format, &mic)?
};
last_write = Instant::now();
}
}
}
continue; continue;
} }
last_write = Instant::now();
let bytes: Vec<u8> = queue.drain(..).collect(); let bytes: Vec<u8> = queue.drain(..).collect();
// Must keep pulling WASAPI buffers even while paused (required to avoid // Must keep pulling WASAPI buffers even while paused (required to avoid
@@ -653,9 +779,20 @@ fn capture_loop(
Some(b) if is_loopback => b.pull(frame_count(&bytes, &session.format)), Some(b) if is_loopback => b.pull(frame_count(&bytes, &session.format)),
_ => Vec::new(), _ => Vec::new(),
}; };
frames_written += write_wav_bytes(w, &bytes, &session.format, &mic)?; frames_written += if split {
write_wav_split(w, &bytes, &session.format, &mic)?
} else {
write_wav_bytes(w, &bytes, &session.format, &mic)?
};
}
let mut mono = decode_mono_f32(&bytes, &session.format)?;
// Mic muted: replace the decoded samples with silence before anything
// downstream sees them — the recording's mic channel, the bridge, the
// transcript feed, the meter, and the voiceprint sample all go quiet.
// Loopback (`is_loopback`) is never muted this way.
if !is_loopback && muted.load(Ordering::Relaxed) {
mono.iter_mut().for_each(|s| *s = 0.0);
} }
let mono = decode_mono_f32(&bytes, &session.format)?;
// Mic: feed the shared bridge (resampled to the loopback's rate) so the // Mic: feed the shared bridge (resampled to the loopback's rate) so the
// loopback thread can fold it into the recording. // loopback thread can fold it into the recording.
@@ -783,6 +920,57 @@ fn f32_to_i16(v: f32) -> i16 {
(v.clamp(-1.0, 1.0) * i16::MAX as f32) as i16 (v.clamp(-1.0, 1.0) * i16::MAX as f32) as i16
} }
/// Split layout (FR-SPK): write frames as **stereo, L = mic, R = loopback**
/// (downmixed to mono), so the two sources stay separate in `audio.wav` and
/// diarization can run on the far side alone. `loopback_bytes` is the raw WASAPI
/// mix buffer (all-zero for a synthesized-silence chunk); `mic` may be shorter
/// than the frame count (rest silent) or empty (mic underran). Returns frames.
fn write_wav_split(
writer: &mut WavWriter<BufWriter<File>>,
loopback_bytes: &[u8],
format: &WaveFormat,
mic: &[f32],
) -> Result<u64, AudioError> {
let sample_type = format
.get_subformat()
.map_err(|e| AudioError::Device(format!("unrecognized mix format: {e}")))?;
let channels = format.get_nchannels() as usize;
if channels == 0 {
return Ok(0);
}
let (bytes_per_sample, decode): (usize, fn(&[u8]) -> f32) =
match (sample_type, format.get_bitspersample()) {
(SampleType::Float, 32) => (4, |c| f32::from_le_bytes(c.try_into().unwrap())),
(SampleType::Int, 16) => (2, |c| {
i16::from_le_bytes(c.try_into().unwrap()) as f32 / i16::MAX as f32
}),
(st, bits) => {
return Err(AudioError::Device(format!(
"unsupported capture format: {st} {bits}-bit"
)))
}
};
let mut frames = 0u64;
for frame in loopback_bytes.chunks_exact(bytes_per_sample * channels) {
// R = loopback downmixed to mono.
let loop_mono = frame
.chunks_exact(bytes_per_sample)
.map(decode)
.sum::<f32>()
/ channels as f32;
// L = mic for this frame (silent if the mic underran).
let mic_s = mic.get(frames as usize).copied().unwrap_or(0.0);
writer
.write_sample(f32_to_i16(mic_s))
.map_err(|e| AudioError::Capture(format!("wav write: {e}")))?;
writer
.write_sample(f32_to_i16(loop_mono))
.map_err(|e| AudioError::Capture(format!("wav write: {e}")))?;
frames += 1;
}
Ok(frames)
}
/// Downmix raw WASAPI capture bytes to mono `f32` in `[-1.0, 1.0]`, at the /// Downmix raw WASAPI capture bytes to mono `f32` in `[-1.0, 1.0]`, at the
/// device's native sample rate (resampling to 16kHz happens separately). /// device's native sample rate (resampling to 16kHz happens separately).
fn decode_mono_f32(bytes: &[u8], format: &WaveFormat) -> Result<Vec<f32>, AudioError> { fn decode_mono_f32(bytes: &[u8], format: &WaveFormat) -> Result<Vec<f32>, AudioError> {
@@ -897,6 +1085,22 @@ impl Resampler {
/// through hound's size-aware iterator; hound is only used to parse the /// through hound's size-aware iterator; hound is only used to parse the
/// format/`fmt ` chunk. /// format/`fmt ` chunk.
pub fn read_wav_mono_16k(path: &Path) -> Result<Vec<f32>, AudioError> { pub fn read_wav_mono_16k(path: &Path) -> Result<Vec<f32>, AudioError> {
read_wav_16k(path, None)
}
/// Like [`read_wav_mono_16k`] but extracts a single channel instead of
/// downmixing — used by the Phase 3.5 split path to diarize the right
/// (loopback) channel and VAD the left (mic) channel (FR-SPK). `channel` is
/// clamped to the file's last channel, so it's safe on a mono/summed file.
pub fn read_wav_channel_16k(path: &Path, channel: usize) -> Result<Vec<f32>, AudioError> {
read_wav_16k(path, Some(channel))
}
/// Shared reader: decrypt (T8.8), reduce each frame to one f32 — averaging all
/// channels (`channel = None`) or picking one (`channel = Some`) — resample to
/// 16kHz. Reads the `data` payload as raw bytes (an unfinalized WAV's size
/// fields read back as 0), so it works on a mid-write / crash-left file (FR-REL-1).
fn read_wav_16k(path: &Path, channel: Option<usize>) -> Result<Vec<f32>, AudioError> {
let raw = std::fs::read(path).map_err(|e| AudioError::Capture(format!("read wav: {e}")))?; let raw = std::fs::read(path).map_err(|e| AudioError::Capture(format!("read wav: {e}")))?;
// Decrypt if the recording was sealed at rest (T8.8); plaintext passes through. // Decrypt if the recording was sealed at rest (T8.8); plaintext passes through.
let file = let file =
@@ -906,6 +1110,8 @@ pub fn read_wav_mono_16k(path: &Path) -> Result<Vec<f32>, AudioError> {
.map_err(|e| AudioError::Capture(format!("open wav: {e}")))? .map_err(|e| AudioError::Capture(format!("open wav: {e}")))?
.spec(); .spec();
let channels = spec.channels.max(1) as usize; let channels = spec.channels.max(1) as usize;
// Which channel index to pick (clamped), or None to downmix.
let pick = channel.map(|c| c.min(channels - 1));
let data_marker = file let data_marker = file
.windows(4) .windows(4)
@@ -918,21 +1124,17 @@ pub fn read_wav_mono_16k(path: &Path) -> Result<Vec<f32>, AudioError> {
(SampleFormat::Float, 32) => bytes (SampleFormat::Float, 32) => bytes
.chunks_exact(4 * channels) .chunks_exact(4 * channels)
.map(|frame| { .map(|frame| {
frame reduce_frame(frame, 4, pick, channels, |c| {
.chunks_exact(4) f32::from_le_bytes(c.try_into().unwrap())
.map(|c| f32::from_le_bytes(c.try_into().unwrap())) })
.sum::<f32>()
/ channels as f32
}) })
.collect(), .collect(),
(SampleFormat::Int, 16) => bytes (SampleFormat::Int, 16) => bytes
.chunks_exact(2 * channels) .chunks_exact(2 * channels)
.map(|frame| { .map(|frame| {
frame reduce_frame(frame, 2, pick, channels, |c| {
.chunks_exact(2) i16::from_le_bytes(c.try_into().unwrap()) as f32 / i16::MAX as f32
.map(|c| i16::from_le_bytes(c.try_into().unwrap()) as f32 / i16::MAX as f32) })
.sum::<f32>()
/ channels as f32
}) })
.collect(), .collect(),
(fmt, bits) => { (fmt, bits) => {
@@ -946,6 +1148,78 @@ pub fn read_wav_mono_16k(path: &Path) -> Result<Vec<f32>, AudioError> {
Ok(resampler.process(&mono)) Ok(resampler.process(&mono))
} }
/// Voice-activity "You" spans (FR-SPK): 100ms windows of 16kHz mono audio above
/// [`VOICE_ENERGY_FLOOR`], merged into `(start_ms, end_ms)` ranges. Run on the
/// left (mic) channel of a split recording to attribute segments to "You" —
/// replaces the old live `MicActivity` timeline (recomputed from the file, so
/// stop and reprocess agree).
pub fn vad_spans(samples: &[f32]) -> Vec<(u64, u64)> {
const WINDOW_MS: u64 = 100;
let win = (TARGET_SAMPLE_RATE as u64 * WINDOW_MS / 1000) as usize; // 1600 @16kHz
if win == 0 {
return Vec::new();
}
let mut spans = Vec::new();
let mut start: Option<u64> = None;
let mut w = 0u64;
let mut i = 0usize;
while i < samples.len() {
let chunk = &samples[i..(i + win).min(samples.len())];
let rms = (chunk.iter().map(|s| s * s).sum::<f32>() / chunk.len().max(1) as f32).sqrt();
match (rms >= VOICE_ENERGY_FLOOR, start) {
(true, None) => start = Some(w * WINDOW_MS),
(false, Some(s)) => {
spans.push((s, w * WINDOW_MS));
start = None;
}
_ => {}
}
i += win;
w += 1;
}
if let Some(s) = start {
spans.push((s, w * WINDOW_MS));
}
spans
}
/// Reduce one interleaved frame to a single f32: pick channel `pick` or average
/// all `channels`. `width` is bytes-per-sample; `decode` turns those bytes → f32.
fn reduce_frame(
frame: &[u8],
width: usize,
pick: Option<usize>,
channels: usize,
decode: impl Fn(&[u8]) -> f32,
) -> f32 {
match pick {
Some(c) => decode(&frame[c * width..c * width + width]),
None => frame.chunks_exact(width).map(decode).sum::<f32>() / channels as f32,
}
}
/// Write mono 16kHz `f32` samples as a Float32 WAV — the format
/// [`read_wav_mono_16k`] reads back. Used to persist the mic voiceprint sample
/// next to a retained recording so it can be re-matched on reprocess (FR-SPK).
pub fn write_wav_mono_16k(path: &Path, samples: &[f32]) -> Result<(), AudioError> {
let spec = WavSpec {
channels: 1,
sample_rate: 16_000,
bits_per_sample: 32,
sample_format: SampleFormat::Float,
};
let mut writer = WavWriter::create(path, spec)
.map_err(|e| AudioError::Capture(format!("write wav: {e}")))?;
for &s in samples {
writer
.write_sample(s)
.map_err(|e| AudioError::Capture(format!("write wav: {e}")))?;
}
writer
.finalize()
.map_err(|e| AudioError::Capture(format!("write wav: {e}")))
}
/// Ceiling on how far the still-flowing stream may run ahead of a stalled /// Ceiling on how far the still-flowing stream may run ahead of a stalled
/// partner before the mixer forwards it alone — so a muted/dead microphone (or a /// partner before the mixer forwards it alone — so a muted/dead microphone (or a
/// silent system output) can't hold up the live transcript. 0.5s @ 16kHz. /// silent system output) can't hold up the live transcript. 0.5s @ 16kHz.
@@ -1051,6 +1325,34 @@ fn mixer_loop(
mod tests { mod tests {
use super::*; use super::*;
#[test]
fn vad_spans_ignores_silence() {
// 200ms of 16kHz silence → no "You" spans.
assert!(vad_spans(&[0.0; 3_200]).is_empty());
}
#[test]
fn vad_spans_collapses_speech_into_spans_with_gaps() {
// 16kHz: 1600 samples per 100ms window. Speech in windows 0,1 (0-200ms)
// and 4,5 (400-600ms), silence between.
let mut samples = vec![0.0f32; 9_600]; // 600ms
samples[0..3_200].fill(0.2); // windows 0,1
samples[6_400..9_600].fill(0.2); // windows 4,5
assert_eq!(vad_spans(&samples), vec![(0, 200), (400, 600)]);
}
#[test]
fn voice_sample_skips_silence_and_keeps_speech() {
let vs = VoiceSample::new(16_000);
vs.push(&[0.0; 4_000]); // silence: below the energy floor
assert!(
vs.samples().is_empty(),
"silence must not fill the voiceprint"
);
vs.push(&[0.2; 4_000]); // speech-level audio
assert_eq!(vs.samples().len(), 4_000, "speech-level audio is collected");
}
#[test] #[test]
fn mixer_sums_aligned_streams_and_keeps_the_unmatched_tail() { fn mixer_sums_aligned_streams_and_keeps_the_unmatched_tail() {
let mut m = Mixer::new(); let mut m = Mixer::new();
@@ -1127,6 +1429,33 @@ mod tests {
let _ = std::fs::remove_dir_all(&dir); let _ = std::fs::remove_dir_all(&dir);
} }
#[test]
fn write_wav_split_puts_mic_left_and_loopback_right() {
let dir = std::env::temp_dir().join(format!("wa-test-{}", uuid::Uuid::new_v4()));
std::fs::create_dir_all(&dir).unwrap();
let path = dir.join("audio.wav");
// 2ch float loopback input; split output is always 2ch 16-bit.
let format = WaveFormat::new(32, 32, &SampleType::Float, 48_000, 2, None);
let spec = WavSpec {
channels: 2,
sample_rate: 48_000,
bits_per_sample: 16,
sample_format: SampleFormat::Int,
};
let mut writer = WavWriter::create(&path, spec).unwrap();
// One 2ch loopback frame [0.4, 0.6] → R = downmix 0.5; mic 0.2 → L.
let loop_bytes: Vec<u8> = [0.4f32, 0.6].iter().flat_map(|s| s.to_le_bytes()).collect();
let n = write_wav_split(&mut writer, &loop_bytes, &format, &[0.2]).unwrap();
writer.finalize().unwrap();
assert_eq!(n, 1);
let mut reader = hound::WavReader::open(&path).unwrap();
let s: Vec<i16> = reader.samples::<i16>().map(|x| x.unwrap()).collect();
assert!((s[0] as f32 / i16::MAX as f32 - 0.2).abs() < 0.001); // L = mic
assert!((s[1] as f32 / i16::MAX as f32 - 0.5).abs() < 0.001); // R = loopback downmix
let _ = std::fs::remove_dir_all(&dir);
}
#[test] #[test]
fn f32_to_i16_clamps_and_scales() { fn f32_to_i16_clamps_and_scales() {
assert_eq!(f32_to_i16(0.0), 0); assert_eq!(f32_to_i16(0.0), 0);
+1094 -122
View File
File diff suppressed because it is too large Load Diff
+183
View File
@@ -0,0 +1,183 @@
//! Enterprise deployment: seed default settings from an admin-supplied `.ini`
//! on **first run only** (before any `settings.json` exists).
//!
//! An admin mass-deploying WhispAssist (GPO / SCCM / Intune) drops a
//! `wa-defaults.ini` and every fresh install picks it up once, seeding
//! `settings.json` with their chosen defaults (record-by-default, preferred
//! backend, retention, model to auto-download, …) — all via native Windows file
//! deployment, no WiX custom actions. See `docs/enterprise-deployment.md`.
//!
//! **Guardrail (CLAUDE.md):** the file must never carry secrets. Keys that look
//! like credentials are ignored here as defense in depth — API keys / OAuth
//! tokens live only in the OS credential store.
use crate::models::Settings;
use serde_json::{Map, Value};
use std::path::PathBuf;
/// Special (non-`Settings`) INI key: when truthy, the first-run seed also fetches
/// the configured `whisper_model` in the background so the machine is ready
/// offline. Stripped before the settings merge.
const AUTO_DOWNLOAD_KEY: &str = "auto_download_model";
/// Candidate locations, first found wins:
/// 1. `%PROGRAMDATA%\WhispAssist\wa-defaults.ini` — machine-wide enterprise path.
/// 2. `<exe dir>\wa-defaults.ini` — the bundled template / per-install override.
fn candidate_paths() -> Vec<PathBuf> {
let mut paths = Vec::new();
if let Ok(program_data) = std::env::var("ProgramData") {
paths.push(
PathBuf::from(program_data)
.join("WhispAssist")
.join("wa-defaults.ini"),
);
}
if let Ok(exe) = std::env::current_exe() {
if let Some(dir) = exe.parent() {
paths.push(dir.join("wa-defaults.ini"));
}
}
paths
}
/// Reads the first existing defaults file and produces the seeded settings plus
/// the whisper model id to auto-download (if `auto_download_model` was set).
/// `None` when no file exists or it contains no overrides (the shipped template
/// is fully commented, so normal installs get exactly today's behavior).
pub fn seed_settings_from_defaults() -> Option<(Settings, Option<String>)> {
let text = candidate_paths()
.into_iter()
.find_map(|p| std::fs::read_to_string(p).ok())?;
seed_from_ini(&text)
}
/// Testable core: parse INI text → merge onto the built-in defaults.
fn seed_from_ini(text: &str) -> Option<(Settings, Option<String>)> {
let mut overrides = parse_ini(text);
if overrides.is_empty() {
return None;
}
// Pull the non-Settings auto-download flag out before the merge.
let auto_download = overrides
.remove(AUTO_DOWNLOAD_KEY)
.map(|v| truthy(&v))
.unwrap_or(false);
// Merge overrides onto the default settings' JSON form, then deserialize.
// Unknown keys (typos) are ignored — `Settings` has no deny_unknown_fields.
let mut base = match serde_json::to_value(crate::commands::default_settings()) {
Ok(Value::Object(map)) => map,
_ => return None,
};
for (k, v) in overrides {
base.insert(k, v);
}
let settings: Settings = serde_json::from_value(Value::Object(base)).ok()?;
let model = if auto_download {
Some(settings.whisper_model.clone())
} else {
None
};
Some((settings, model))
}
/// Minimal INI reader: skips blanks, `;`/`#` comments and `[section]` headers;
/// splits each `key = value` on the first `=`; coerces values to bool / integer /
/// string so serde lands them on the typed `Settings` fields. Silently drops any
/// key that looks like a secret (guardrail — no credentials in the deploy file).
fn parse_ini(text: &str) -> Map<String, Value> {
let mut map = Map::new();
for line in text.lines() {
let line = line.trim();
if line.is_empty()
|| line.starts_with(';')
|| line.starts_with('#')
|| line.starts_with('[')
{
continue;
}
let Some((key, value)) = line.split_once('=') else {
continue;
};
let key = key.trim().to_string();
let value = value.trim();
if key.is_empty() || looks_like_secret(&key) {
continue;
}
map.insert(key, coerce(value));
}
map
}
/// `true`/`false` → bool, all-integer → number, everything else → string.
fn coerce(value: &str) -> Value {
match value.to_ascii_lowercase().as_str() {
"true" => return Value::Bool(true),
"false" => return Value::Bool(false),
_ => {}
}
if let Ok(n) = value.parse::<i64>() {
return Value::Number(n.into());
}
Value::String(value.to_string())
}
fn truthy(v: &Value) -> bool {
matches!(v, Value::Bool(true))
|| matches!(v, Value::String(s) if s.eq_ignore_ascii_case("true"))
}
/// Defense in depth: never seed anything that smells like a credential.
fn looks_like_secret(key: &str) -> bool {
let k = key.to_ascii_lowercase();
["key", "token", "secret", "credential", "password"]
.iter()
.any(|needle| k.contains(needle))
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn fully_commented_file_is_a_noop() {
let ini = "; default_record = true\n# preferred_backend = cpu\n[general]\n\n";
assert!(seed_from_ini(ini).is_none());
}
#[test]
fn coerces_bool_int_and_string_fields() {
let ini = "default_record = true\nretention_max_age_days = 90\npreferred_backend = cpu\n";
let (settings, model) = seed_from_ini(ini).expect("overrides present");
assert!(settings.default_record);
assert_eq!(settings.retention_max_age_days, Some(90));
assert_eq!(settings.preferred_backend, "cpu");
assert!(model.is_none());
}
#[test]
fn auto_download_returns_the_configured_model() {
let ini = "whisper_model = base.en-q5_1\nauto_download_model = true\n";
let (_settings, model) = seed_from_ini(ini).expect("overrides present");
assert_eq!(model.as_deref(), Some("base.en-q5_1"));
}
#[test]
fn unset_fields_keep_their_defaults() {
let ini = "default_record = true\n";
let (settings, _) = seed_from_ini(ini).unwrap();
// microphone stays on, auto_start stays off — only the named key changed.
assert!(settings.microphone_enabled);
assert!(!settings.auto_start);
}
#[test]
fn secret_keys_are_ignored() {
let ini = "anthropic_api_key = sk-should-be-dropped\ndefault_record = true\n";
let map = parse_ini(ini);
assert!(!map.contains_key("anthropic_api_key"));
assert!(map.contains_key("default_record"));
}
}
+174 -2
View File
@@ -23,6 +23,11 @@ pub enum DiarError {
pub trait Diarizer: Send + Sync { pub trait Diarizer: Send + Sync {
/// Partition audio into speaker spans. /// Partition audio into speaker spans.
fn diarize(&self, wav: &Path) -> Result<Vec<SpeakerSpan>, DiarError>; fn diarize(&self, wav: &Path) -> Result<Vec<SpeakerSpan>, DiarError>;
/// Partition already-loaded 16kHz mono samples into speaker spans. Phase 3
/// (FR-SPK) diarizes a *masked* far-side signal (the mic's "You" ranges
/// zeroed), so the caller loads and masks the samples itself rather than
/// pointing at the raw WAV.
fn diarize_samples(&self, samples: Vec<f32>) -> Result<Vec<SpeakerSpan>, DiarError>;
/// Assign speaker labels to transcript segments by overlap with spans. /// Assign speaker labels to transcript segments by overlap with spans.
fn assign(&self, segments: &mut [TranscriptSegment], spans: &[SpeakerSpan]); fn assign(&self, segments: &mut [TranscriptSegment], spans: &[SpeakerSpan]);
} }
@@ -53,6 +58,60 @@ pub fn assign_by_overlap(segments: &mut [TranscriptSegment], spans: &[SpeakerSpa
} }
} }
/// Split-layout attribution (FR-SPK): decides per segment between "You" (mic
/// channel voice activity) and the far side's diarized speakers by comparing
/// the *total* voiced overlap on each channel, not by picking the single
/// longest span — a long far-side diarizer span could otherwise swallow a
/// segment the user spoke most of, showing their words under "Speaker N".
/// The mic channel is physically the user's voice alone, so channel evidence
/// outranks cluster evidence; ties go to "You" (mislabeling the user's own
/// words as someone else is the worse failure). A segment with no voiced
/// overlap on either channel keeps its prior label rather than guessing.
// ponytail: whole-segment labels — a segment genuinely containing both sides
// still gets one speaker; the upgrade path is transcribing each channel
// separately so segments can never mix voices.
pub fn assign_split(
segments: &mut [TranscriptSegment],
you_spans: &[(u64, u64)],
far_vad: &[(u64, u64)],
far_spans: &[SpeakerSpan],
) {
fn overlap(a0: u64, a1: u64, b0: u64, b1: u64) -> u64 {
a1.min(b1).saturating_sub(a0.max(b0))
}
for seg in segments.iter_mut() {
let mic_ms: u64 = you_spans
.iter()
.map(|&(s, e)| overlap(seg.start_ms, seg.end_ms, s, e))
.sum();
let far_ms: u64 = far_vad
.iter()
.map(|&(s, e)| overlap(seg.start_ms, seg.end_ms, s, e))
.sum();
if mic_ms == 0 && far_ms == 0 {
continue;
}
if mic_ms >= far_ms {
seg.speaker = "You".to_string();
} else if let Some(span) = far_spans
.iter()
.map(|sp| {
(
overlap(seg.start_ms, seg.end_ms, sp.start_ms, sp.end_ms),
sp,
)
})
.filter(|(o, _)| *o > 0)
.max_by_key(|(o, _)| *o)
.map(|(_, sp)| sp)
{
seg.speaker = span.speaker.clone();
}
// Far side voiced but no diarizer span overlaps (e.g. a sub-700ms span
// was filtered): keep the prior label rather than guess.
}
}
/// sherpa-onnx-backed diarizer: pyannote segmentation + speaker-embedding + /// sherpa-onnx-backed diarizer: pyannote segmentation + speaker-embedding +
/// fast clustering (ADR-0005, T4.1). `Diarize::compute` needs `&mut self`; it's /// fast clustering (ADR-0005, T4.1). `Diarize::compute` needs `&mut self`; it's
/// wrapped in a `Mutex` to satisfy `Diarizer: Sync` — diarization is a /// wrapped in a `Mutex` to satisfy `Diarizer: Sync` — diarization is a
@@ -70,7 +129,13 @@ impl SherpaDiarizer {
// sherpa-onnx to pick the cluster count itself from `threshold` // sherpa-onnx to pick the cluster count itself from `threshold`
// instead of forcing a fixed number of speakers. // instead of forcing a fixed number of speakers.
num_clusters: Some(-1), num_clusters: Some(-1),
threshold: Some(0.5), // Tuning evidence: 0.5 over the summed mic+loopback mono WAV
// exploded a 2-speaker conversation into 83 clusters (overlapped
// speech + short pyannote chunks yield unstable embeddings). sherpa
// fast-clustering: larger threshold → fewer clusters. 0.7 lands a
// 2-person meeting near 2–4. ponytail: code constant, no settings
// knob until 0.7 proves insufficient in the field.
threshold: Some(0.7),
..Default::default() ..Default::default()
}; };
let engine = sherpa_rs::diarize::Diarize::new(segmentation_model, embedding_model, config) let engine = sherpa_rs::diarize::Diarize::new(segmentation_model, embedding_model, config)
@@ -86,11 +151,28 @@ impl Diarizer for SherpaDiarizer {
fn diarize(&self, wav: &Path) -> Result<Vec<SpeakerSpan>, DiarError> { fn diarize(&self, wav: &Path) -> Result<Vec<SpeakerSpan>, DiarError> {
let samples = let samples =
crate::audio::read_wav_mono_16k(wav).map_err(|e| DiarError::Run(e.to_string()))?; crate::audio::read_wav_mono_16k(wav).map_err(|e| DiarError::Run(e.to_string()))?;
self.diarize_samples(samples)
}
fn diarize_samples(&self, samples: Vec<f32>) -> Result<Vec<SpeakerSpan>, DiarError> {
// Guard the native FFI against degenerate input (a fully-masked far side
// is all zeros; a tiny buffer has no usable window). sherpa can crash
// rather than error on these, so skip cleanly instead.
if samples.len() < MIN_DIARIZE_SAMPLES {
return Ok(Vec::new());
}
let mut engine = self.engine.lock().unwrap_or_else(|e| e.into_inner()); let mut engine = self.engine.lock().unwrap_or_else(|e| e.into_inner());
let segments = engine let segments = engine
.compute(samples, None) .compute(samples, None)
.map_err(|e| DiarError::Run(e.to_string()))?; .map_err(|e| DiarError::Run(e.to_string()))?;
Ok(segments.into_iter().map(segment_to_span).collect()) // Drop sub-700ms spans: their embeddings are unstable and only cause
// label churn in assign_by_overlap (which keeps the prior label when no
// span overlaps). See MIN_SPAN_MS.
Ok(segments
.into_iter()
.map(segment_to_span)
.filter(span_long_enough)
.collect())
} }
fn assign(&self, segments: &mut [TranscriptSegment], spans: &[SpeakerSpan]) { fn assign(&self, segments: &mut [TranscriptSegment], spans: &[SpeakerSpan]) {
@@ -98,6 +180,23 @@ impl Diarizer for SherpaDiarizer {
} }
} }
/// Minimum span duration kept from a diarization pass. Sub-second pyannote
/// chunks carry unstable ERes2Net embeddings and only churn labels.
#[cfg(feature = "diarization")]
const MIN_SPAN_MS: u64 = 700;
/// Below this many 16kHz samples (~1s) the diarizer skips the sherpa FFI: a
/// fully-masked far side is all zeros and a tiny buffer has no usable
/// segmentation window, either of which can crash the native code.
#[cfg(feature = "diarization")]
const MIN_DIARIZE_SAMPLES: usize = 16_000;
/// A span is kept only if it lasts at least [`MIN_SPAN_MS`].
#[cfg(feature = "diarization")]
fn span_long_enough(span: &SpeakerSpan) -> bool {
span.end_ms.saturating_sub(span.start_ms) >= MIN_SPAN_MS
}
/// sherpa-onnx speaker indices are 0-based; WA's internal labels are 1-based ("S1"…). /// sherpa-onnx speaker indices are 0-based; WA's internal labels are 1-based ("S1"…).
#[cfg(feature = "diarization")] #[cfg(feature = "diarization")]
fn segment_to_span(seg: sherpa_rs::diarize::Segment) -> SpeakerSpan { fn segment_to_span(seg: sherpa_rs::diarize::Segment) -> SpeakerSpan {
@@ -165,6 +264,63 @@ mod overlap_tests {
assign_by_overlap(&mut segments, &[]); assign_by_overlap(&mut segments, &[]);
assert_eq!(segments[0].speaker, "S1"); assert_eq!(segments[0].speaker, "S1");
} }
#[test]
fn split_labels_a_mic_dominant_segment_you_even_against_a_longer_far_span() {
// The user spoke 0-4000ms; the far side 4000-6000ms — but the far
// cluster span covers the whole window, so the old merged max-overlap
// pick handed the entire segment (the user's words included) to the
// far speaker. Channel totals must side with the mic instead.
let mut segments = vec![segment(0, 6000)];
let you = vec![(0u64, 4000u64)];
let far_vad = vec![(4000u64, 6000u64)];
let far_spans = vec![span(0, 6000, "S1")]; // long far cluster span
assign_split(&mut segments, &you, &far_vad, &far_spans);
assert_eq!(segments[0].speaker, "You");
}
#[test]
fn split_ties_go_to_you() {
let mut segments = vec![segment(0, 2000)];
let you = vec![(0u64, 1000u64)];
let far_vad = vec![(1000u64, 2000u64)];
let far_spans = vec![span(1000, 2000, "S1")];
assign_split(&mut segments, &you, &far_vad, &far_spans);
assert_eq!(segments[0].speaker, "You");
}
#[test]
fn split_assigns_the_best_far_span_when_the_far_side_dominates() {
let mut segments = vec![segment(0, 3000)];
let you = vec![(0u64, 500u64)];
let far_vad = vec![(500u64, 3000u64)];
let far_spans = vec![span(500, 1000, "S1"), span(1000, 3000, "S2")];
assign_split(&mut segments, &you, &far_vad, &far_spans);
assert_eq!(segments[0].speaker, "S2");
}
#[test]
fn split_keeps_the_prior_label_when_both_channels_are_silent() {
let mut segments = vec![segment(5000, 6000)];
segments[0].speaker = "S9".to_string();
assign_split(
&mut segments,
&[(0, 1000)],
&[(0, 1000)],
&[span(0, 1000, "S1")],
);
assert_eq!(segments[0].speaker, "S9");
}
#[test]
fn split_keeps_the_prior_label_when_far_is_voiced_but_no_far_span_overlaps() {
// Far VAD hears speech but every diarizer span was filtered (sub-700ms):
// don't guess a label.
let mut segments = vec![segment(0, 1000)];
segments[0].speaker = "S3".to_string();
assign_split(&mut segments, &[], &[(0, 1000)], &[span(2000, 3000, "S1")]);
assert_eq!(segments[0].speaker, "S3");
}
} }
#[cfg(all(test, feature = "diarization"))] #[cfg(all(test, feature = "diarization"))]
@@ -184,6 +340,22 @@ mod tests {
assert_eq!(span.speaker, "S1"); assert_eq!(span.speaker, "S1");
} }
#[test]
fn span_long_enough_filters_sub_minimum_spans() {
let short = SpeakerSpan {
start_ms: 0,
end_ms: MIN_SPAN_MS - 1,
speaker: "S1".to_string(),
};
let exact = SpeakerSpan {
start_ms: 0,
end_ms: MIN_SPAN_MS,
speaker: "S1".to_string(),
};
assert!(!span_long_enough(&short));
assert!(span_long_enough(&exact));
}
#[test] #[test]
fn new_surfaces_a_load_error_for_missing_models_instead_of_panicking() { fn new_surfaces_a_load_error_for_missing_models_instead_of_panicking() {
let result = let result =
+9 -4
View File
@@ -110,8 +110,9 @@ pub fn match_mic_speaker(
} }
/// Distinct speaker labels in first-appearance order — spans come back from /// Distinct speaker labels in first-appearance order — spans come back from
/// the diarizer already sorted by start time. /// the diarizer already sorted by start time. `pub(crate)` so the Phase 3
fn first_appearance_order(spans: &[SpeakerSpan]) -> Vec<String> { /// per-stream path can reuse it to name its merged You + far-side spans.
pub(crate) fn first_appearance_order(spans: &[SpeakerSpan]) -> Vec<String> {
let mut seen = std::collections::HashSet::new(); let mut seen = std::collections::HashSet::new();
spans spans
.iter() .iter()
@@ -152,8 +153,12 @@ fn cosine_similarity(a: &[f32], b: &[f32]) -> f32 {
/// `mic_label` -> "You"; every other label, in first-appearance order -> /// `mic_label` -> "You"; every other label, in first-appearance order ->
/// "Speaker 2", "Speaker 3", … (numbering starts at 2 — "You" stands in for /// "Speaker 2", "Speaker 3", … (numbering starts at 2 — "You" stands in for
/// "Speaker 1" without ever being called that). /// "Speaker 1" without ever being called that). `pub(crate)` so Phase 3 reuses
fn build_name_map(labels_in_order: &[String], mic_label: &str) -> HashMap<String, String> { /// it for its merged You + far-side spans, keeping naming uniform app-wide.
pub(crate) fn build_name_map(
labels_in_order: &[String],
mic_label: &str,
) -> HashMap<String, String> {
let mut names = HashMap::new(); let mut names = HashMap::new();
let mut next_speaker_number = 2; let mut next_speaker_number = 2;
for label in labels_in_order { for label in labels_in_order {
+166 -6
View File
@@ -9,6 +9,7 @@ pub mod audio;
pub mod briefs; pub mod briefs;
pub mod calendar; pub mod calendar;
pub mod commands; pub mod commands;
pub mod deploy;
pub mod diarization; pub mod diarization;
pub mod error; pub mod error;
pub mod hardware; pub mod hardware;
@@ -27,7 +28,8 @@ pub mod vault;
use std::path::PathBuf; use std::path::PathBuf;
use std::sync::{Arc, Mutex as StdMutex}; use std::sync::{Arc, Mutex as StdMutex};
use std::thread::JoinHandle; use std::thread::JoinHandle;
use tauri::tray::TrayIcon; use tauri::menu::{Menu, MenuItem};
use tauri::tray::{MouseButton, MouseButtonState, TrayIcon, TrayIconBuilder, TrayIconEvent};
use tauri::Manager; use tauri::Manager;
use tokio::sync::Mutex; use tokio::sync::Mutex;
@@ -85,6 +87,10 @@ pub struct RecordingSession {
/// auto-labeled "You" instead of a clustered "S1"/"S2". `None` when the /// auto-labeled "You" instead of a clustered "S1"/"S2". `None` when the
/// mic is disabled (same conditions as `mic_capture`). /// mic is disabled (same conditions as `mic_capture`).
pub mic_voice_sample: Option<Arc<audio::VoiceSample>>, pub mic_voice_sample: Option<Arc<audio::VoiceSample>>,
/// `audio.wav` channel layout for this recording (FR-SPK): `"split"` (mic
/// enabled → stereo mic-left/loopback-right) or `"summed"` (mic off). Passed
/// to `finalize_meeting` so playback/diarization/export can branch on it.
pub audio_layout: &'static str,
/// Live notes redesign: raw user-authored notes accumulated *during* the /// Live notes redesign: raw user-authored notes accumulated *during* the
/// recording (freeform text + per-moment annotations) — see /// recording (freeform text + per-moment annotations) — see
/// `models::ManualNotes`. Mutated by `update_live_notes`/`set_segment_note` /// `models::ManualNotes`. Mutated by `update_live_notes`/`set_segment_note`
@@ -98,18 +104,71 @@ pub struct RecordingSession {
/// tooltip on recording state changes (FR-CAP-4). /// tooltip on recording state changes (FR-CAP-4).
pub struct TrayHandle(pub TrayIcon); pub struct TrayHandle(pub TrayIcon);
/// Show a blocking native error dialog and exit non-zero. Used for failures
/// that happen before the Tauri app (and its window) exist, where a silent
/// `panic=abort` fast-fail would otherwise close with no explanation. Exits
/// cleanly (code 1) rather than aborting so the OS reports no crash.
fn fatal_startup_error(body: &str) -> ! {
tracing::error!("fatal startup error: {body}");
#[cfg(windows)]
{
use windows::core::PCWSTR;
use windows::Win32::UI::WindowsAndMessaging::{
MessageBoxW, MB_ICONERROR, MB_OK, MB_SYSTEMMODAL,
};
// Null-terminated UTF-16 for the Win32 wide API.
let to_wide = |s: &str| {
s.encode_utf16()
.chain(std::iter::once(0))
.collect::<Vec<u16>>()
};
let text = to_wide(body);
let caption = to_wide("WhispAssist can't start");
// SAFETY: both buffers are null-terminated and outlive the call.
unsafe {
MessageBoxW(
None,
PCWSTR(text.as_ptr()),
PCWSTR(caption.as_ptr()),
MB_OK | MB_ICONERROR | MB_SYSTEMMODAL,
);
}
}
#[cfg(not(windows))]
eprintln!("WhispAssist can't start: {body}");
std::process::exit(1);
}
/// Build state, register commands/events, and run the app. /// Build state, register commands/events, and run the app.
pub fn run() { pub fn run() {
tracing_subscriber::fmt().with_env_filter("info").init(); tracing_subscriber::fmt().with_env_filter("info").init();
let store: Arc<dyn storage::Store> = Arc::new( let store: Arc<dyn storage::Store> =
tauri::async_runtime::block_on(storage::SqliteStore::connect()) match tauri::async_runtime::block_on(storage::SqliteStore::connect()) {
.expect("failed to initialize storage (wa.db)"), Ok(s) => Arc::new(s),
); // A migration checksum mismatch (an edited-after-shipping migration)
// or a locked/corrupt file lands here. Tell the user instead of
// fast-failing into a silent no-launch.
Err(e) => fatal_startup_error(&format!(
"WhispAssist couldn't open its database (wa.db):\n\n{e}\n\n\
Your recordings and notes are not deleted. This usually means the \
database is from a newer version of WhispAssist, or a data migration \
is incompatible. Reinstalling the matching version, or moving wa.db \
aside from %LOCALAPPDATA%\\WhispAssist, will let the app start."
)),
};
let store_for_setup = store.clone(); let store_for_setup = store.clone();
tauri::Builder::default() tauri::Builder::default()
.plugin(tauri_plugin_dialog::init()) .plugin(tauri_plugin_dialog::init())
// Opt-in launch-at-login (NFR-RES-4). The macOS launcher arg is required
// by the signature but unused on Windows, where enable/disable writes a
// per-user HKCU\...\Run entry (no admin). Off until the user (or an
// enterprise deploy file) turns `auto_start` on.
.plugin(tauri_plugin_autostart::init(
tauri_plugin_autostart::MacosLauncher::LaunchAgent,
None,
))
// In-memory streaming of recordings for the player (FR-REC-5): decrypts // In-memory streaming of recordings for the player (FR-REC-5): decrypts
// on the fly so no plaintext audio is ever written to disk. // on the fly so no plaintext audio is ever written to disk.
.register_uri_scheme_protocol("waaudio", |_ctx, request| { .register_uri_scheme_protocol("waaudio", |_ctx, request| {
@@ -120,13 +179,88 @@ pub fn run() {
session: Mutex::new(None), session: Mutex::new(None),
}) })
.setup(move |app| { .setup(move |app| {
// Single tray icon (the `trayIcon` in tauri.conf.json was removed so
// this is the only one). It carries a Show/Quit menu and, on
// left-click, restores the window — the always-available way back
// from "close to tray".
let icon = tauri::image::Image::from_bytes(include_bytes!("../icons/tray.png"))?; let icon = tauri::image::Image::from_bytes(include_bytes!("../icons/tray.png"))?;
let tray = tauri::tray::TrayIconBuilder::new() let show_item = MenuItem::with_id(app, "show", "Show WhispAssist", true, None::<&str>)?;
let quit_item = MenuItem::with_id(app, "quit", "Quit", true, None::<&str>)?;
let menu = Menu::with_items(app, &[&show_item, &quit_item])?;
let tray = TrayIconBuilder::new()
.icon(icon) .icon(icon)
.tooltip("WhispAssist — idle") .tooltip("WhispAssist — idle")
.menu(&menu)
.show_menu_on_left_click(false)
.on_menu_event(|app, event| match event.id.as_ref() {
"show" => show_main_window(app),
"quit" => app.exit(0),
_ => {}
})
.on_tray_icon_event(|tray, event| {
if let TrayIconEvent::Click {
button: MouseButton::Left,
button_state: MouseButtonState::Up,
..
} = event
{
show_main_window(tray.app_handle());
}
})
.build(app)?; .build(app)?;
app.manage(TrayHandle(tray)); app.manage(TrayHandle(tray));
// First-run enterprise deploy seeding (deploy.rs): if no settings.json
// exists yet and an admin dropped a wa-defaults.ini, seed settings once
// and optionally fetch the configured model in the background. One-shot
// — guarded by the settings file's absence, so it never re-runs and adds
// nothing to idle cost (NFR-RES-1).
if !crate::paths::settings_path().exists() {
if let Some((seeded, model_to_download)) = deploy::seed_settings_from_defaults() {
match commands::save_settings(&seeded) {
Ok(()) => {
tracing::info!("seeded settings.json from wa-defaults.ini");
if let Some(id) = model_to_download {
let app_handle = app.handle().clone();
tauri::async_runtime::spawn(async move {
if let Err(e) = commands::download_model(
app_handle,
commands::DownloadModelArgs {
kind: "whisper".into(),
id,
},
)
.await
{
tracing::warn!(
"deploy auto-download of model failed: {e:?}"
);
}
});
}
}
Err(e) => {
tracing::error!(
"first-run deploy seeding failed to write settings: {e:?}"
)
}
}
}
}
// Reconcile launch-at-login with the persisted preference (NFR-RES-4):
// if the user opted in but the OS entry is missing (e.g. after a
// reinstall or a deploy file that set auto_start), restore it. One-shot.
{
use tauri_plugin_autostart::ManagerExt;
let manager = app.autolaunch();
if commands::load_settings().auto_start && !manager.is_enabled().unwrap_or(false) {
if let Err(e) = manager.enable() {
tracing::warn!("failed to restore auto-start entry: {e}");
}
}
}
// Reminders (Phase 8, T8.6, FR-CAL-5) are Windows-scheduled toasts, not // Reminders (Phase 8, T8.6, FR-CAL-5) are Windows-scheduled toasts, not
// an app-side timer — Windows itself is what's "polling", so this stays // an app-side timer — Windows itself is what's "polling", so this stays
// within NFR-RES-1. init() just registers the AppUserModelID. // within NFR-RES-1. init() just registers the AppUserModelID.
@@ -220,6 +354,17 @@ pub fn run() {
}); });
Ok(()) Ok(())
}) })
// Close to tray (keep running in background): when the setting is on,
// the window X hides instead of quitting; the tray "Quit" is the real
// exit. Off → default behavior (closing the window quits the app).
.on_window_event(|window, event| {
if let tauri::WindowEvent::CloseRequested { api, .. } = event {
if commands::load_settings().close_to_tray {
api.prevent_close();
let _ = window.hide();
}
}
})
.invoke_handler(tauri::generate_handler![ .invoke_handler(tauri::generate_handler![
commands::start_recording, commands::start_recording,
commands::stop_recording, commands::stop_recording,
@@ -227,6 +372,7 @@ pub fn run() {
commands::recording_playback_path, commands::recording_playback_path,
commands::pause_recording, commands::pause_recording,
commands::resume_recording, commands::resume_recording,
commands::toggle_microphone_mute,
commands::set_recording_retention, commands::set_recording_retention,
commands::acknowledge_recording_consent, commands::acknowledge_recording_consent,
commands::update_live_notes, commands::update_live_notes,
@@ -238,6 +384,9 @@ pub fn run() {
commands::list_audio_devices, commands::list_audio_devices,
commands::list_input_devices, commands::list_input_devices,
commands::set_preferred_backend, commands::set_preferred_backend,
commands::set_auto_start,
commands::monitor_audio_level,
commands::stress_test_hardware,
commands::list_models, commands::list_models,
commands::list_whisper_languages, commands::list_whisper_languages,
commands::download_npu_package, commands::download_npu_package,
@@ -266,6 +415,7 @@ pub fn run() {
commands::generate_summary, commands::generate_summary,
commands::confirm_action_items, commands::confirm_action_items,
commands::generate_tags, commands::generate_tags,
commands::enhance_notes,
commands::llm_setup_suggestions, commands::llm_setup_suggestions,
commands::pull_ollama_model, commands::pull_ollama_model,
commands::import_pst, commands::import_pst,
@@ -310,6 +460,16 @@ pub fn run() {
.expect("error while running WhispAssist"); .expect("error while running WhispAssist");
} }
/// Restore the main window from the tray (show + unminimize + focus). Shared by
/// the tray left-click and the "Show WhispAssist" menu item.
fn show_main_window(app: &tauri::AppHandle) {
if let Some(w) = app.get_webview_window("main") {
let _ = w.show();
let _ = w.unminimize();
let _ = w.set_focus();
}
}
/// Used by `commands.rs` to keep the tray tooltip honest about capture state (FR-CAP-4). /// Used by `commands.rs` to keep the tray tooltip honest about capture state (FR-CAP-4).
pub(crate) fn update_tray_tooltip(app: &tauri::AppHandle, text: &str) { pub(crate) fn update_tray_tooltip(app: &tauri::AppHandle, text: &str) {
if let Some(tray) = app.try_state::<TrayHandle>() { if let Some(tray) = app.try_state::<TrayHandle>() {
+7 -11
View File
@@ -21,10 +21,7 @@ pub enum MediaError {
#[error("{0} isn't installed or on PATH — install it and try again")] #[error("{0} isn't installed or on PATH — install it and try again")]
ToolMissing(&'static str), ToolMissing(&'static str),
#[error("{tool} failed: {message}")] #[error("{tool} failed: {message}")]
Failed { Failed { tool: &'static str, message: String },
tool: &'static str,
message: String,
},
#[error("io error: {0}")] #[error("io error: {0}")]
Io(#[from] std::io::Error), Io(#[from] std::io::Error),
} }
@@ -60,7 +57,11 @@ fn run(program: &'static str, cmd: &mut Command) -> Result<(), MediaError> {
if !output.status.success() { if !output.status.success() {
let stderr = String::from_utf8_lossy(&output.stderr); let stderr = String::from_utf8_lossy(&output.stderr);
let stdout = String::from_utf8_lossy(&output.stdout); let stdout = String::from_utf8_lossy(&output.stdout);
let text = if stderr.trim().is_empty() { stdout } else { stderr }; let text = if stderr.trim().is_empty() {
stdout
} else {
stderr
};
let tail: Vec<&str> = text.lines().filter(|l| !l.trim().is_empty()).collect(); let tail: Vec<&str> = text.lines().filter(|l| !l.trim().is_empty()).collect();
let start = tail.len().saturating_sub(6); let start = tail.len().saturating_sub(6);
return Err(MediaError::Failed { return Err(MediaError::Failed {
@@ -145,12 +146,7 @@ mod tests {
fn import_reports_a_missing_local_file_without_touching_a_tool() { fn import_reports_a_missing_local_file_without_touching_a_tool() {
let dir = std::env::temp_dir().join(format!("wa-media-{}", uuid::Uuid::new_v4())); let dir = std::env::temp_dir().join(format!("wa-media-{}", uuid::Uuid::new_v4()));
std::fs::create_dir_all(&dir).unwrap(); std::fs::create_dir_all(&dir).unwrap();
let err = import_to_wav( let err = import_to_wav("no-such-file.mp4", &dir.join("out.wav"), &dir).unwrap_err();
"no-such-file.mp4",
&dir.join("out.wav"),
&dir,
)
.unwrap_err();
assert!(matches!(err, MediaError::Failed { tool: "import", .. })); assert!(matches!(err, MediaError::Failed { tool: "import", .. }));
let _ = std::fs::remove_dir_all(&dir); let _ = std::fs::remove_dir_all(&dir);
} }
+16
View File
@@ -214,6 +214,11 @@ pub struct MeetingBundle {
pub tags: Vec<String>, pub tags: Vec<String>,
pub speakers: Vec<SpeakerInfo>, pub speakers: Vec<SpeakerInfo>,
pub action_items: Vec<ActionItem>, pub action_items: Vec<ActionItem>,
/// `audio.wav` channel layout (FR-SPK/FR-CAP): `"split"` or `"summed"`.
/// `default` so bundles exported before this field deserialize as `None`
/// (treated as `"summed"`).
#[serde(default)]
pub audio_layout: Option<String>,
} }
#[derive(Debug, Clone, Serialize, Deserialize)] #[derive(Debug, Clone, Serialize, Deserialize)]
@@ -349,6 +354,17 @@ pub struct Settings {
pub mcp_expose: String, // none|selected|all pub mcp_expose: String, // none|selected|all
#[serde(default)] #[serde(default)]
pub mcp_expose_recordings: bool, pub mcp_expose_recordings: bool,
/// Launch WhispAssist automatically at login (opt-in, NFR-RES-4). OFF by
/// default; toggled via `set_auto_start`, which writes a per-user
/// `HKCU\...\Run` entry through `tauri-plugin-autostart` (no admin). An
/// enterprise deploy file may set this to `true` (see `deploy.rs`).
#[serde(default)]
pub auto_start: bool,
/// Closing the window hides WhispAssist to the system tray instead of
/// quitting, so it keeps running in the background (tray "Quit" really
/// exits). ON by default; the tray icon is the always-available way back.
#[serde(default = "default_true")]
pub close_to_tray: bool,
} }
fn default_mcp_transport() -> String { fn default_mcp_transport() -> String {
+54 -9
View File
@@ -74,6 +74,10 @@ pub struct FinalizeMeeting {
pub language: Option<String>, pub language: Option<String>,
pub backend_used: Option<String>, pub backend_used: Option<String>,
pub model_used: Option<String>, pub model_used: Option<String>,
/// How `audio.wav`'s channels are laid out (FR-SPK/FR-CAP): `"split"` =
/// stereo mic-left/loopback-right, `"summed"`/`None` = mic mixed into every
/// channel. `None` leaves the stored value unchanged (e.g. on reprocess).
pub audio_layout: Option<String>,
} }
/// Full meeting detail: DB row + transcript + speakers + notes (`get_meeting`'s /// Full meeting detail: DB row + transcript + speakers + notes (`get_meeting`'s
@@ -91,6 +95,10 @@ pub struct Meeting {
pub language: Option<String>, pub language: Option<String>,
pub backend_used: Option<String>, pub backend_used: Option<String>,
pub model_used: Option<String>, pub model_used: Option<String>,
/// `audio.wav` channel layout (FR-SPK/FR-CAP): `"split"` (mic-left /
/// loopback-right) or `"summed"`/`None` (mic mixed into every channel).
/// Diarization, playback downmix, and bundle export branch on it.
pub audio_layout: Option<String>,
pub segments: Vec<TranscriptSegment>, pub segments: Vec<TranscriptSegment>,
pub speakers: Vec<SpeakerInfo>, pub speakers: Vec<SpeakerInfo>,
pub notes_markdown: String, pub notes_markdown: String,
@@ -268,6 +276,10 @@ pub trait Store: Send + Sync {
) -> Result<Vec<MeetingListItem>, StoreError>; ) -> Result<Vec<MeetingListItem>, StoreError>;
async fn get_meeting(&self, id: &MeetingId) -> Result<Meeting, StoreError>; async fn get_meeting(&self, id: &MeetingId) -> Result<Meeting, StoreError>;
async fn delete_meeting(&self, id: &MeetingId) -> Result<(), StoreError>; async fn delete_meeting(&self, id: &MeetingId) -> Result<(), StoreError>;
/// Overwrite a meeting's lifecycle `status` (e.g. mark a background import
/// `transcribing` while it runs, or `error` if it fails). `finalize_meeting`
/// is still the only path to `ready`.
async fn set_meeting_status(&self, id: &MeetingId, status: &str) -> Result<(), StoreError>;
async fn update_notes(&self, id: &MeetingId, markdown: &str) -> Result<(), StoreError>; async fn update_notes(&self, id: &MeetingId, markdown: &str) -> Result<(), StoreError>;
/// (Re)builds this meeting's FTS index row from the current title and /// (Re)builds this meeting's FTS index row from the current title and
/// whatever's on disk/in the DB for transcript/notes/summary/tags (Phase /// whatever's on disk/in the DB for transcript/notes/summary/tags (Phase
@@ -284,6 +296,14 @@ pub trait Store: Send + Sync {
label: &str, label: &str,
name: &str, name: &str,
) -> Result<(), StoreError>; ) -> Result<(), StoreError>;
/// Delete every `speakers` row for a meeting (labels, names, participant
/// links, merges). Used before a full re-diarization rebuild
/// (`reprocess_transcript`, FR-SPK): the old labels key to the previous
/// clustering and are meaningless once the audio is re-clustered, so the
/// caller re-inserts the fresh set via `finalize_meeting`. Without this,
/// stale labels from an over-split run linger in the DB (and the
/// Participants pane) even though the transcript no longer references them.
async fn clear_speakers(&self, id: &MeetingId) -> Result<(), StoreError>;
/// Fold over-split speaker labels into one canonical label (T4.5, /// Fold over-split speaker labels into one canonical label (T4.5,
/// FR-SPK-3). Segment speaker IDs in storage are never rewritten /// FR-SPK-3). Segment speaker IDs in storage are never rewritten
/// (FR-SPK-5) — `get_meeting` resolves `from` labels to `into` when it /// (FR-SPK-5) — `get_meeting` resolves `from` labels to `into` when it
@@ -734,7 +754,8 @@ impl Store for SqliteStore {
let now = now_unix(); let now = now_unix();
sqlx::query( sqlx::query(
"UPDATE meetings SET status = 'ready', ended_at = ?, duration_secs = ?, recorded = ?, "UPDATE meetings SET status = 'ready', ended_at = ?, duration_secs = ?, recorded = ?,
language = ?, backend_used = ?, model_used = ?, updated_at = ? WHERE id = ?", language = ?, backend_used = ?, model_used = ?,
audio_layout = COALESCE(?, audio_layout), updated_at = ? WHERE id = ?",
) )
.bind(now) .bind(now)
.bind(s.duration_secs) .bind(s.duration_secs)
@@ -742,6 +763,7 @@ impl Store for SqliteStore {
.bind(&s.language) .bind(&s.language)
.bind(&s.backend_used) .bind(&s.backend_used)
.bind(&s.model_used) .bind(&s.model_used)
.bind(&s.audio_layout)
.bind(now) .bind(now)
.bind(id) .bind(id)
.execute(&self.pool) .execute(&self.pool)
@@ -857,7 +879,7 @@ impl Store for SqliteStore {
async fn get_meeting(&self, id: &MeetingId) -> Result<Meeting, StoreError> { async fn get_meeting(&self, id: &MeetingId) -> Result<Meeting, StoreError> {
let row = sqlx::query( let row = sqlx::query(
"SELECT id, title, started_at, ended_at, duration_secs, status, recorded, language, backend_used, model_used, calendar_event_id, template_id "SELECT id, title, started_at, ended_at, duration_secs, status, recorded, language, backend_used, model_used, calendar_event_id, template_id, audio_layout
FROM meetings WHERE id = ?", FROM meetings WHERE id = ?",
) )
.bind(id) .bind(id)
@@ -928,6 +950,7 @@ impl Store for SqliteStore {
language: row.get("language"), language: row.get("language"),
backend_used: row.get("backend_used"), backend_used: row.get("backend_used"),
model_used: row.get("model_used"), model_used: row.get("model_used"),
audio_layout: row.get("audio_layout"),
segments, segments,
speakers, speakers,
notes_markdown, notes_markdown,
@@ -957,6 +980,16 @@ impl Store for SqliteStore {
Ok(()) Ok(())
} }
async fn set_meeting_status(&self, id: &MeetingId, status: &str) -> Result<(), StoreError> {
sqlx::query("UPDATE meetings SET status = ?, updated_at = ? WHERE id = ?")
.bind(status)
.bind(now_unix())
.bind(id)
.execute(&self.pool)
.await?;
Ok(())
}
async fn update_notes(&self, id: &MeetingId, markdown: &str) -> Result<(), StoreError> { async fn update_notes(&self, id: &MeetingId, markdown: &str) -> Result<(), StoreError> {
write_artifact( write_artifact(
&paths::meeting_dir(id).join("notes.md"), &paths::meeting_dir(id).join("notes.md"),
@@ -1227,13 +1260,15 @@ impl Store for SqliteStore {
started_at: i64, started_at: i64,
ended_at: Option<i64>, ended_at: Option<i64>,
) -> Result<(), StoreError> { ) -> Result<(), StoreError> {
sqlx::query("UPDATE meetings SET started_at = ?, ended_at = ?, updated_at = ? WHERE id = ?") sqlx::query(
.bind(started_at) "UPDATE meetings SET started_at = ?, ended_at = ?, updated_at = ? WHERE id = ?",
.bind(ended_at) )
.bind(now_unix()) .bind(started_at)
.bind(meeting_id) .bind(ended_at)
.execute(&self.pool) .bind(now_unix())
.await?; .bind(meeting_id)
.execute(&self.pool)
.await?;
Ok(()) Ok(())
} }
@@ -1252,6 +1287,14 @@ impl Store for SqliteStore {
.await .await
} }
async fn clear_speakers(&self, id: &MeetingId) -> Result<(), StoreError> {
sqlx::query("DELETE FROM speakers WHERE meeting_id = ?")
.bind(id)
.execute(&self.pool)
.await?;
Ok(())
}
async fn merge_speakers( async fn merge_speakers(
&self, &self,
id: &MeetingId, id: &MeetingId,
@@ -2015,6 +2058,7 @@ mod tests {
language: Some("fr".to_string()), // what auto-detect resolved to language: Some("fr".to_string()), // what auto-detect resolved to
backend_used: Some("cpu".to_string()), backend_used: Some("cpu".to_string()),
model_used: Some("small-q5_1".to_string()), model_used: Some("small-q5_1".to_string()),
audio_layout: None,
}, },
) )
.await .await
@@ -2058,6 +2102,7 @@ mod tests {
language: Some("en".to_string()), language: Some("en".to_string()),
backend_used: Some("cpu".to_string()), backend_used: Some("cpu".to_string()),
model_used: Some("small-q5_1".to_string()), model_used: Some("small-q5_1".to_string()),
audio_layout: None,
}, },
) )
.await .await
+8 -2
View File
@@ -588,7 +588,10 @@ mod tests {
feed_into(&mut s, &step, &d2, &mut out); feed_into(&mut s, &step, &d2, &mut out);
assert!(out[3].interim && out[3].text == "next sentence"); assert!(out[3].interim && out[3].text == "next sentence");
assert_ne!(out[3].id, out[2].id); assert_ne!(out[3].id, out[2].id);
assert!(out[3].start_ms >= 3000, "starts after the 3 committed steps"); assert!(
out[3].start_ms >= 3000,
"starts after the 3 committed steps"
);
} }
#[test] #[test]
@@ -612,7 +615,10 @@ mod tests {
assert!(out.last().unwrap().interim, "still growing before flush"); assert!(out.last().unwrap().interim, "still growing before flush");
flush_into(&mut s, &decode, &mut out); flush_into(&mut s, &decode, &mut out);
let last = out.last().unwrap(); let last = out.last().unwrap();
assert!(!last.interim && last.text == "partial", "flush finalizes it"); assert!(
!last.interim && last.text == "partial",
"flush finalizes it"
);
} }
#[test] #[test]
+226 -23
View File
@@ -7,21 +7,24 @@
//! //!
//! Split of work, validated by the T3.4 spike (encoder ~3.6× faster on NPU): //! Split of work, validated by the T3.4 spike (encoder ~3.6× faster on NPU):
//! - **encoder** (fixed `[1,80,3000]` shape) runs on the **accelerator** (NPU or //! - **encoder** (fixed `[1,80,3000]` shape) runs on the **accelerator** (NPU or
//! GPU) — the expensive graph, and the fixed shape accelerators want; //! DirectML GPU) — the expensive graph, and the fixed shape accelerators want;
//! - **decoder** (dynamic, autoregressive) runs greedy on the **CPU** EP — a //! - **decoder** (dynamic, autoregressive) decodes greedily **with a KV cache**
//! dynamic KV loop is a poor accelerator fit and the cheap half anyway. //! (Optimum's merged export: `use_cache_branch` + `past_key_values`), so each
//! //! step feeds one token instead of re-running the whole prefix. On the NPU
//! ponytail: greedy, no-KV-cache decode (re-feeds the full token prefix each //! path the decoder session itself goes to the **OpenVINO GPU EP** (Intel
//! step). Correct and simple; windows are short so the token count is small. //! iGPU, same runtime bundle) so sustained CPU stays near-idle; it falls back
//! Add `decoder_with_past` only if the decoder shows up in a profile. //! to the CPU EP when no iGPU is usable, and `WA_ONNX_DECODER_DEVICE=cpu|gpu`
//! overrides the choice. DirectML backends keep the CPU decoder — the DML EP
//! handles per-step-growing KV shapes poorly.
use super::{AudioWindow, SegmentSink, Transcriber, TrxError}; use super::{AudioWindow, SegmentSink, Transcriber, TrxError};
use crate::models::{BackendId, TranscriptSegment}; use crate::models::{BackendId, TranscriptSegment};
use ort::execution_providers::{ use ort::execution_providers::{
DirectMLExecutionProvider, ExecutionProviderDispatch, OpenVINOExecutionProvider, DirectMLExecutionProvider, ExecutionProviderDispatch, OpenVINOExecutionProvider,
}; };
use ort::session::Session; use ort::session::{Session, SessionInputValue};
use ort::value::Tensor; use ort::value::{Tensor, ValueType};
use std::borrow::Cow;
use std::collections::HashMap; use std::collections::HashMap;
use std::path::Path; use std::path::Path;
use std::sync::atomic::{AtomicU64, Ordering}; use std::sync::atomic::{AtomicU64, Ordering};
@@ -30,6 +33,36 @@ use std::sync::Mutex;
/// Safety cap on generated tokens per 30 s window (Whisper's own max is 448). /// Safety cap on generated tokens per 30 s window (Whisper's own max is 448).
const MAX_NEW_TOKENS: usize = 224; const MAX_NEW_TOKENS: usize = 224;
/// One KV-cache tensor pair of the merged decoder, discovered from the graph
/// at load: the `past_key_values.*` input, its matching `present.*` output,
/// whether it's decoder self-attention (refreshed every step) or encoder
/// cross-attention (computed once on the first step, then passed through
/// untouched by the cache branch), and the dummy shape to feed before any
/// past exists (see [`dummy_past_dims`]).
struct KvSlot {
past: String,
present: String,
self_attention: bool,
dummy_dims: Vec<usize>,
}
/// `"past_key_values.3.decoder.key"` → `"present.3.decoder.key"` — Optimum's
/// fixed naming convention for the merged decoder's cache I/O.
fn present_name_for(past: &str) -> String {
format!("present.{}", past.trim_start_matches("past_key_values."))
}
/// The "no past yet" shape for a KV input: batch → 1, any other dynamic (−1)
/// dim → **1** (a dummy length-1 past — ORT's raw-data tensor creation rejects
/// zero-length dims, and the no-cache branch never reads the values anyway;
/// same dummy Optimum's own runner feeds), static dims kept.
fn dummy_past_dims(dims: &[i64]) -> Vec<usize> {
dims.iter()
.enumerate()
.map(|(i, &d)| if i == 0 || d < 0 { 1 } else { d as usize })
.collect()
}
pub struct OnnxTranscriber { pub struct OnnxTranscriber {
encoder: Mutex<Session>, encoder: Mutex<Session>,
decoder: Mutex<Session>, decoder: Mutex<Session>,
@@ -38,6 +71,13 @@ pub struct OnnxTranscriber {
dec_ids: String, dec_ids: String,
dec_hidden: String, dec_hidden: String,
dec_logits: String, dec_logits: String,
/// The merged decoder's KV-cache plumbing (see [`KvSlot`]) plus its
/// branch selector input.
kv_slots: Vec<KvSlot>,
dec_use_cache: String,
/// Which EP the decoder session actually landed on ("OpenVINO/GPU" or
/// "CPU") — surfaced so `load_transcriber`'s engine log tells the truth.
decoder_ep: &'static str,
detok: Detok, detok: Detok,
/// `[decoder_start_token_id, ...forced_decoder_ids]` — the fixed prompt /// `[decoder_start_token_id, ...forced_decoder_ids]` — the fixed prompt
/// prefix before generation begins. /// prefix before generation begins.
@@ -120,29 +160,76 @@ impl OnnxTranscriber {
let hidden = Tensor::from_array((hidden_shape, hidden_data)) let hidden = Tensor::from_array((hidden_shape, hidden_data))
.map_err(|e| TrxError::Inference(e.to_string()))?; .map_err(|e| TrxError::Inference(e.to_string()))?;
// Greedy decode on the CPU. // Greedy KV-cache decode. Step 1 runs the merged graph's no-cache
// branch over the full prompt and yields every present KV; later steps
// feed a single token plus the cache, refreshing only the decoder
// self-attention slots — the cache branch passes encoder
// cross-attention KVs through untouched, so the step-1 tensors stay
// authoritative (same contract as Optimum's own runner).
// ponytail: KVs round-trip host memory each step; wire ort IoBinding
// to pin them on-device if GPU-decoder profiling shows the copies matter.
let mut kv_cache: Vec<Tensor<f32>> = self
.kv_slots
.iter()
.map(|slot| {
let len = slot.dummy_dims.iter().product::<usize>();
Tensor::from_array((slot.dummy_dims.clone(), vec![0.0f32; len]))
})
.collect::<Result<_, _>>()
.map_err(|e| TrxError::Inference(e.to_string()))?;
let mut tokens = self.initial_tokens.clone(); let mut tokens = self.initial_tokens.clone();
let mut cached = false; // flips true once step 1 has filled kv_cache
for _ in 0..MAX_NEW_TOKENS { for _ in 0..MAX_NEW_TOKENS {
let ids: Vec<i64> = tokens.clone(); let ids: Vec<i64> = if cached {
vec![*tokens.last().unwrap_or(&self.eot)]
} else {
tokens.clone()
};
let ids_tensor = Tensor::from_array(([1usize, ids.len()], ids)) let ids_tensor = Tensor::from_array(([1usize, ids.len()], ids))
.map_err(|e| TrxError::Inference(e.to_string()))?; .map_err(|e| TrxError::Inference(e.to_string()))?;
let branch = Tensor::from_array(([1usize], vec![cached]))
.map_err(|e| TrxError::Inference(e.to_string()))?;
let next = { // Presents are extracted (copied) inside the outputs' borrow scope,
// then written back into kv_cache after it ends.
let (next, fresh_kvs) = {
let mut dec = self let mut dec = self
.decoder .decoder
.lock() .lock()
.map_err(|_| TrxError::Inference("decoder mutex poisoned".into()))?; .map_err(|_| TrxError::Inference("decoder mutex poisoned".into()))?;
let mut inputs: Vec<(Cow<'_, str>, SessionInputValue<'_>)> = vec![
(Cow::from(self.dec_ids.as_str()), ids_tensor.view().into()),
(Cow::from(self.dec_hidden.as_str()), hidden.view().into()),
(Cow::from(self.dec_use_cache.as_str()), branch.view().into()),
];
for (slot, kv) in self.kv_slots.iter().zip(kv_cache.iter()) {
inputs.push((Cow::from(slot.past.as_str()), kv.view().into()));
}
let outputs = dec let outputs = dec
.run(ort::inputs![ .run(inputs)
self.dec_ids.as_str() => ids_tensor.view(),
self.dec_hidden.as_str() => hidden.view(),
])
.map_err(|e| TrxError::Inference(e.to_string()))?; .map_err(|e| TrxError::Inference(e.to_string()))?;
let (shape, data) = outputs[self.dec_logits.as_str()] let (shape, data) = outputs[self.dec_logits.as_str()]
.try_extract_tensor::<f32>() .try_extract_tensor::<f32>()
.map_err(|e| TrxError::Inference(e.to_string()))?; .map_err(|e| TrxError::Inference(e.to_string()))?;
self.argmax_last(shape, data) let next = self.argmax_last(shape, data);
let mut fresh: Vec<(usize, Vec<usize>, Vec<f32>)> = Vec::new();
for (i, slot) in self.kv_slots.iter().enumerate() {
if slot.self_attention || !cached {
let (s, d) = outputs[slot.present.as_str()]
.try_extract_tensor::<f32>()
.map_err(|e| TrxError::Inference(e.to_string()))?;
fresh.push((i, s.iter().map(|d| *d as usize).collect(), d.to_vec()));
}
}
(next, fresh)
}; };
for (i, dims, data) in fresh_kvs {
kv_cache[i] = Tensor::from_array((dims, data))
.map_err(|e| TrxError::Inference(e.to_string()))?;
}
cached = true;
if next == self.eot { if next == self.eot {
break; break;
@@ -180,6 +267,11 @@ impl OnnxTranscriber {
best best
} }
/// Which EP the decoder session landed on — "OpenVINO/GPU" or "CPU".
pub fn decoder_ep(&self) -> &'static str {
self.decoder_ep
}
fn segment(&self, text: String, start_ms: u64, end_ms: u64) -> Option<TranscriptSegment> { fn segment(&self, text: String, start_ms: u64, end_ms: u64) -> Option<TranscriptSegment> {
if text.is_empty() { if text.is_empty() {
return None; return None;
@@ -252,7 +344,7 @@ impl Transcriber for OnnxTranscriber {
Self::ensure_runtime_env(runtime_dll); Self::ensure_runtime_env(runtime_dll);
let encoder_path = model.join("encoder_model.onnx"); let encoder_path = model.join("encoder_model.onnx");
let decoder_path = model.join("decoder_model.onnx"); let decoder_path = model.join("decoder_model_merged.onnx");
let tokenizer_path = model.join("tokenizer.json"); let tokenizer_path = model.join("tokenizer.json");
let gen_config_path = model.join("generation_config.json"); let gen_config_path = model.join("generation_config.json");
@@ -263,11 +355,46 @@ impl Transcriber for OnnxTranscriber {
.commit_from_file(&encoder_path) .commit_from_file(&encoder_path)
.map_err(|e| TrxError::Load(e.to_string()))?; .map_err(|e| TrxError::Load(e.to_string()))?;
// Decoder on the default CPU EP (dynamic shapes; poor NPU fit). // Decoder session. The KV-cache loop is dynamic-shaped and
let decoder = Session::builder() // autoregressive — a poor NPU fit — but it needn't burn CPU either: on
.map_err(|e| TrxError::Load(e.to_string()))? // the NPU path, try the same OpenVINO runtime's GPU plugin (Intel
.commit_from_file(&decoder_path) // iGPU) first and fall back to the default CPU EP if no GPU is usable.
.map_err(|e| TrxError::Load(e.to_string()))?; // `WA_ONNX_DECODER_DEVICE=cpu|gpu` overrides (debug escape hatch, like
// WA_NPU_RUNTIME_URL). DirectML backends keep the CPU decoder — the
// DML EP handles per-step-growing KV shapes poorly.
let want_gpu = match std::env::var("WA_ONNX_DECODER_DEVICE").as_deref() {
Ok("cpu") => false,
Ok("gpu") => true,
_ => matches!(backend, BackendId::Npu),
};
let mut decoder_ep = "CPU";
let mut decoder = None;
if want_gpu {
let gpu = Session::builder()
.and_then(|b| {
b.with_execution_providers([OpenVINOExecutionProvider::default()
.with_device_type("GPU")
.build()
.error_on_failure()])
})
.and_then(|b| b.commit_from_file(&decoder_path));
match gpu {
Ok(s) => {
decoder_ep = "OpenVINO/GPU";
decoder = Some(s);
}
Err(e) => tracing::warn!(
"OpenVINO GPU EP unavailable for the decoder ({e}); using the CPU EP"
),
}
}
let decoder = match decoder {
Some(s) => s,
None => Session::builder()
.map_err(|e| TrxError::Load(e.to_string()))?
.commit_from_file(&decoder_path)
.map_err(|e| TrxError::Load(e.to_string()))?,
};
let enc_input = Self::find_input(&encoder, "input_features")?; let enc_input = Self::find_input(&encoder, "input_features")?;
let enc_output = Self::find_output(&encoder, "last_hidden_state") let enc_output = Self::find_output(&encoder, "last_hidden_state")
@@ -275,6 +402,53 @@ impl Transcriber for OnnxTranscriber {
let dec_ids = Self::find_input(&decoder, "input_ids")?; let dec_ids = Self::find_input(&decoder, "input_ids")?;
let dec_hidden = Self::find_input(&decoder, "encoder_hidden")?; let dec_hidden = Self::find_input(&decoder, "encoder_hidden")?;
let dec_logits = Self::find_output(&decoder, "logits")?; let dec_logits = Self::find_output(&decoder, "logits")?;
let dec_use_cache = Self::find_input(&decoder, "use_cache_branch")?;
// KV-cache plumbing, discovered rather than hardcoded so the layer
// count comes from the graph (whisper-base: 6 layers → 24 slots).
let mut kv_slots: Vec<KvSlot> = Vec::new();
for input in &decoder.inputs {
if !input.name.starts_with("past_key_values.") {
continue;
}
let dims: Vec<i64> = match &input.input_type {
ValueType::Tensor { shape, .. } => shape.to_vec(),
other => {
return Err(TrxError::Load(format!(
"KV input '{}' has non-tensor type {other:?}",
input.name
)))
}
};
let present = present_name_for(&input.name);
if !decoder.outputs.iter().any(|o| o.name == present) {
return Err(TrxError::Load(format!(
"decoder missing present output '{present}'"
)));
}
kv_slots.push(KvSlot {
self_attention: input.name.contains(".decoder."),
dummy_dims: dummy_past_dims(&dims),
past: input.name.clone(),
present,
});
}
if kv_slots.is_empty() {
return Err(TrxError::Load(
"decoder has no past_key_values inputs — expected the merged (KV-cache) export"
.into(),
));
}
// A pre-KV-cache install leaves the old plain decoder behind — 208 MB
// of dead weight once the merged graph loads. Best-effort removal.
let legacy = model.join("decoder_model.onnx");
if legacy.exists() {
match std::fs::remove_file(&legacy) {
Ok(()) => tracing::info!("removed superseded decoder_model.onnx"),
Err(e) => tracing::warn!("couldn't remove superseded decoder_model.onnx: {e}"),
}
}
let detok = Detok::from_tokenizer_json(&tokenizer_path)?; let detok = Detok::from_tokenizer_json(&tokenizer_path)?;
let gen = GenConfig::from_file(&gen_config_path)?; let gen = GenConfig::from_file(&gen_config_path)?;
@@ -290,6 +464,9 @@ impl Transcriber for OnnxTranscriber {
dec_ids, dec_ids,
dec_hidden, dec_hidden,
dec_logits, dec_logits,
kv_slots,
dec_use_cache,
decoder_ep,
detok, detok,
initial_tokens, initial_tokens,
special_floor: gen.decoder_start, special_floor: gen.decoder_start,
@@ -457,6 +634,28 @@ mod tests {
assert_eq!(d.get(&'A'), Some(&b'A')); assert_eq!(d.get(&'A'), Some(&b'A'));
} }
#[test]
fn present_name_follows_optimum_convention() {
assert_eq!(
present_name_for("past_key_values.3.decoder.key"),
"present.3.decoder.key"
);
assert_eq!(
present_name_for("past_key_values.0.encoder.value"),
"present.0.encoder.value"
);
}
#[test]
fn dummy_past_dims_pins_dynamic_axes_to_one_and_keeps_static_ones() {
// whisper-base decoder self-attention past: [batch, 8, past_seq, 64]
// with batch/past_seq dynamic — both pin to 1 (ORT rejects 0-length
// dims for raw-data tensors; the no-cache branch ignores the values).
assert_eq!(dummy_past_dims(&[-1, 8, -1, 64]), vec![1, 8, 1, 64]);
// A static trailing shape is preserved (batch still pins to 1).
assert_eq!(dummy_past_dims(&[-1, 8, 1500, 64]), vec![1, 8, 1500, 64]);
}
#[test] #[test]
fn byte_decoder_covers_all_256_bytes() { fn byte_decoder_covers_all_256_bytes() {
let d = byte_decoder(); let d = byte_decoder();
@@ -481,10 +680,14 @@ mod tests {
return; return;
}; };
let wav = std::env::var("WA_NPU_TEST_WAV").expect("set WA_NPU_TEST_WAV"); let wav = std::env::var("WA_NPU_TEST_WAV").expect("set WA_NPU_TEST_WAV");
// Surface the engine's own EP-selection logs (GPU-decoder fallback
// warnings etc.) — the test harness has no subscriber otherwise.
let _ = tracing_subscriber::fmt().with_env_filter("info").try_init();
let t0 = std::time::Instant::now(); let t0 = std::time::Instant::now();
let t = OnnxTranscriber::load(Path::new(&model_dir), BackendId::Npu, None) let t = OnnxTranscriber::load(Path::new(&model_dir), BackendId::Npu, None)
.expect("load NPU transcriber"); .expect("load NPU transcriber");
let load_ms = t0.elapsed().as_millis(); let load_ms = t0.elapsed().as_millis();
eprintln!("[spike] decoder EP: {}", t.decoder_ep());
let t1 = std::time::Instant::now(); let t1 = std::time::Instant::now();
let segs = t.transcribe_file(Path::new(&wav)).expect("transcribe"); let segs = t.transcribe_file(Path::new(&wav)).expect("transcribe");
let infer_ms = t1.elapsed().as_millis(); let infer_ms = t1.elapsed().as_millis();
+13 -3
View File
@@ -15,9 +15,19 @@ const REPO: &str = "onnx-community/whisper-base.en";
pub const DEFAULT_ONNX_MODEL: &str = "base.en"; pub const DEFAULT_ONNX_MODEL: &str = "base.en";
/// The four artifacts the `OnnxTranscriber` needs, relative to the repo root. /// The four artifacts the `OnnxTranscriber` needs, relative to the repo root.
/// The decoder is Optimum's *merged* export — both the no-past and with-past
/// branches in one graph, selected by a `use_cache_branch` input — which is
/// what enables the KV-cache decode loop (the plain `decoder_model.onnx`
/// re-ran the whole prefix every step). Existing installs migrate for free:
/// swapping this entry flips `is_installed` to false, the startup background
/// fetch re-runs `download`, and `download` skips the three files already on
/// disk, fetching only the merged decoder.
const FILES: &[(&str, &str)] = &[ const FILES: &[(&str, &str)] = &[
("encoder_model.onnx", "onnx/encoder_model.onnx"), ("encoder_model.onnx", "onnx/encoder_model.onnx"),
("decoder_model.onnx", "onnx/decoder_model.onnx"), (
"decoder_model_merged.onnx",
"onnx/decoder_model_merged.onnx",
),
("tokenizer.json", "tokenizer.json"), ("tokenizer.json", "tokenizer.json"),
("generation_config.json", "generation_config.json"), ("generation_config.json", "generation_config.json"),
]; ];
@@ -45,7 +55,7 @@ pub fn model_files(id: &str) -> OnnxModelFiles {
let dir = onnx_dir(id); let dir = onnx_dir(id);
OnnxModelFiles { OnnxModelFiles {
encoder: dir.join("encoder_model.onnx"), encoder: dir.join("encoder_model.onnx"),
decoder: dir.join("decoder_model.onnx"), decoder: dir.join("decoder_model_merged.onnx"),
tokenizer: dir.join("tokenizer.json"), tokenizer: dir.join("tokenizer.json"),
generation_config: dir.join("generation_config.json"), generation_config: dir.join("generation_config.json"),
} }
@@ -125,6 +135,6 @@ mod tests {
fn catalog_lists_the_four_required_artifacts() { fn catalog_lists_the_four_required_artifacts() {
assert_eq!(FILES.len(), 4); assert_eq!(FILES.len(), 4);
assert!(FILES.iter().any(|(n, _)| *n == "encoder_model.onnx")); assert!(FILES.iter().any(|(n, _)| *n == "encoder_model.onnx"));
assert!(FILES.iter().any(|(n, _)| *n == "decoder_model.onnx")); assert!(FILES.iter().any(|(n, _)| *n == "decoder_model_merged.onnx"));
} }
} }
+6 -6
View File
@@ -1,7 +1,7 @@
{ {
"$schema": "https://schema.tauri.app/config/2", "$schema": "https://schema.tauri.app/config/2",
"productName": "WhispAssist", "productName": "WhispAssist",
"version": "0.5.1", "version": "0.7.3",
"identifier": "bet.dou.whispassist", "identifier": "bet.dou.whispassist",
"build": { "build": {
"frontendDist": "../dist", "frontendDist": "../dist",
@@ -22,18 +22,18 @@
], ],
"security": { "security": {
"csp": "default-src 'self'; connect-src 'self' http://localhost:* http://127.0.0.1:*; img-src 'self' data:; media-src 'self' http://waaudio.localhost; style-src 'self' 'unsafe-inline'" "csp": "default-src 'self'; connect-src 'self' http://localhost:* http://127.0.0.1:*; img-src 'self' data:; media-src 'self' http://waaudio.localhost; style-src 'self' 'unsafe-inline'"
},
"trayIcon": {
"iconPath": "icons/tray.png",
"tooltip": "WhispAssist"
} }
}, },
"bundle": { "bundle": {
"active": true, "active": true,
"targets": ["msi", "nsis"], "targets": ["msi", "nsis"],
"icon": ["icons/icon.ico"], "icon": ["icons/icon.ico"],
"resources": ["wa-defaults.ini"],
"windows": { "windows": {
"webviewInstallMode": { "type": "downloadBootstrapper" } "webviewInstallMode": { "type": "downloadBootstrapper" },
"nsis": {
"installMode": "both"
}
} }
} }
} }
+1 -1
View File
@@ -1,6 +1,6 @@
{ {
"$schema": "gen/schemas/desktop-schema.json", "$schema": "gen/schemas/desktop-schema.json",
"bundle": { "bundle": {
"resources": ["vulkan-1.dll"] "resources": ["vulkan-1.dll", "wa-defaults.ini"]
} }
} }
+60
View File
@@ -0,0 +1,60 @@
; ============================================================================
; WhispAssist enterprise deployment defaults (wa-defaults.ini)
; ============================================================================
; Read ONCE, on a machine's FIRST launch (before settings.json exists), to
; seed the app's default settings. After that the user's own settings.json
; wins and this file is ignored. Deploy it with native Windows tooling
; (Group Policy / SCCM / Intune file copy) to either location — the first
; one found wins:
;
; 1. %PROGRAMDATA%\WhispAssist\wa-defaults.ini (machine-wide)
; 2. <install dir>\wa-defaults.ini (this bundled template)
;
; As shipped, every setting below is COMMENTED OUT, so a normal install
; behaves exactly as if this file were absent. Uncomment and edit the lines
; you want to preset. Format is flat "key = value" — no sections required;
; [section] headers, ';' and '#' comment lines are ignored.
;
; Values: true / false for switches; a plain number for numeric fields;
; text otherwise. Unknown / misspelled keys are ignored.
;
; SECURITY: never put secrets here. API keys, OAuth tokens and sync
; passwords live only in the OS credential store; any key containing
; "key", "token", "secret", "credential" or "password" is dropped on read.
; ----------------------------------------------------------------------------
; ---- Recording (ADR-0009) --------------------------------------------------
; Record every meeting to disk by default (consent notice still applies).
; default_record = false
; ---- Transcription backend & model -----------------------------------------
; preferred_backend = auto ; auto | npu | nvidia | amd | intel | cpu
; whisper_model = base.en-q5_1 ; a catalog model id
; whisper_language = auto ; auto | ISO-639-1 code (multilingual model only)
; low_overhead = false
; Download whisper_model in the background on first launch so the machine is
; ready offline. Requires network at first run.
; auto_download_model = false
; ---- Storage & retention (FR-STORE-2) --------------------------------------
; storage_root = C:\ProgramData\WhispAssist\data
; retention_max_age_days = 90
; retention_max_size_gb = 20
; ---- Local LLM / summaries (ADR-0007) --------------------------------------
; llm_provider = ollama ; ollama | custom | anthropic | off
; llm_endpoint = http://localhost:11434
; llm_model = llama3
; ---- Capture & UX ----------------------------------------------------------
; microphone_enabled = true
; auto_record_calendar = false
; theme = system ; system | light | dark
; ---- Startup (NFR-RES-4) ---------------------------------------------------
; Launch WhispAssist automatically at login for the user.
; auto_start = false
; ---- Sync master switch (ADR-0010; targets/creds configured in-app) --------
; sync_enabled = false
+74 -24
View File
@@ -12,6 +12,7 @@
import { recording } from "./lib/stores/recording.svelte"; import { recording } from "./lib/stores/recording.svelte";
import { settings } from "./lib/stores/settings.svelte"; import { settings } from "./lib/stores/settings.svelte";
import { meetings } from "./lib/stores/meetings.svelte"; import { meetings } from "./lib/stores/meetings.svelte";
import { imports } from "./lib/stores/imports.svelte";
import { calendar } from "./lib/stores/calendar.svelte"; import { calendar } from "./lib/stores/calendar.svelte";
import { api, type NoteTemplate, type CalendarEvent } from "./lib/api"; import { api, type NoteTemplate, type CalendarEvent } from "./lib/api";
import { onMount } from "svelte"; import { onMount } from "svelte";
@@ -24,6 +25,8 @@
Square, Square,
Trash2, Trash2,
FilePlus, FilePlus,
Mic,
MicOff,
Settings as SettingsIcon, Settings as SettingsIcon,
AlertTriangle, AlertTriangle,
PanelLeftClose, PanelLeftClose,
@@ -131,6 +134,7 @@
recording.init(); recording.init();
settings.load(); settings.load();
meetings.init(); meetings.init();
imports.init(); // live background-import progress for the tracker
calendar.load(); // events power the auto-record timer above calendar.load(); // events power the auto-record timer above
checkVault(); checkVault();
api api
@@ -207,6 +211,18 @@
} else if (e.ctrlKey && e.key === ",") { } else if (e.ctrlKey && e.key === ",") {
e.preventDefault(); e.preventDefault();
showSettings = !showSettings; showSettings = !showSettings;
} else if (
// Press "M" to mute/unmute the mic mid-meeting (FR-CAP-7). Bare key (no
// modifiers) and only while recording with the mic on.
e.key.toLowerCase() === "m" &&
!e.ctrlKey &&
!e.metaKey &&
!e.altKey &&
recording.state !== "idle" &&
settings.settings.microphone_enabled
) {
e.preventDefault();
recording.toggleMute();
} }
} }
@@ -232,10 +248,16 @@
<div class="app" data-theme={resolvedTheme}> <div class="app" data-theme={resolvedTheme}>
<div class="sr-only" role="status" aria-live="polite">{recordingAnnouncement}</div> <div class="sr-only" role="status" aria-live="polite">{recordingAnnouncement}</div>
<header class="bar"> <header class="bar">
<strong>WhispAssist</strong>
<span class="muted">{t("app.tagline")}</span>
<div class="spacer"></div>
{#if recording.state === "idle"} {#if recording.state === "idle"}
<button
class="record-btn"
onclick={startRecording}
title={t("app.record_title")}
aria-keyshortcuts="Control+Shift+R"
>
<Circle size={11} fill="currentColor" aria-hidden="true" />
{t("app.record")}
</button>
<select <select
class="theme-select" class="theme-select"
bind:value={selectedTemplateId} bind:value={selectedTemplateId}
@@ -247,23 +269,6 @@
<option value={tpl.id}>{tpl.name}</option> <option value={tpl.id}>{tpl.name}</option>
{/each} {/each}
</select> </select>
<button
class="import-btn"
onclick={() => (showImport = true)}
title={t("app.add_meeting_title")}
>
<FilePlus size={13} aria-hidden="true" />
{t("app.add_meeting")}
</button>
<button
class="record-btn"
onclick={startRecording}
title={t("app.record_title")}
aria-keyshortcuts="Control+Shift+R"
>
<Circle size={11} fill="currentColor" aria-hidden="true" />
{t("app.record")}
</button>
{:else} {:else}
<button <button
class="stop-btn" class="stop-btn"
@@ -278,6 +283,18 @@
<Trash2 size={12} aria-hidden="true" /> <Trash2 size={12} aria-hidden="true" />
{t("app.cancel")} {t("app.cancel")}
</button> </button>
{/if}
<div class="spacer"></div>
{#if recording.state === "idle"}
<button
class="import-btn"
onclick={() => (showImport = true)}
title={t("app.add_meeting_title")}
>
<FilePlus size={13} aria-hidden="true" />
{t("app.add_meeting")}
</button>
{:else}
<span class="rec"> <span class="rec">
<span class="rec-dot" aria-hidden="true"></span> <span class="rec-dot" aria-hidden="true"></span>
{t("app.recording")} {t("app.recording")}
@@ -289,6 +306,23 @@
micPeak={recording.levelPeakMic} micPeak={recording.levelPeakMic}
showMic={settings.settings.microphone_enabled} showMic={settings.settings.microphone_enabled}
/> />
{#if settings.settings.microphone_enabled}
<button
class="mute-btn"
class:muted={recording.micMuted}
onclick={() => recording.toggleMute()}
aria-pressed={recording.micMuted}
aria-keyshortcuts="M"
title={recording.micMuted ? t("app.unmute_title") : t("app.mute_title")}
>
{#if recording.micMuted}
<MicOff size={14} aria-hidden="true" />
{:else}
<Mic size={14} aria-hidden="true" />
{/if}
<span class="sr-only">{recording.micMuted ? t("app.unmute") : t("app.mute")}</span>
</button>
{/if}
{#if settings.hardware} {#if settings.hardware}
<span class="backend" title={t("app.backend_title")}>{settings.hardware.active}</span> <span class="backend" title={t("app.backend_title")}>{settings.hardware.active}</span>
{/if} {/if}
@@ -588,10 +622,6 @@
border-bottom: 1px solid var(--border); border-bottom: 1px solid var(--border);
background: var(--bg-elevated); background: var(--bg-elevated);
} }
.bar strong {
font-size: 0.95rem;
letter-spacing: -0.01em;
}
.spacer { .spacer {
flex: 1; flex: 1;
} }
@@ -718,6 +748,26 @@
border-radius: var(--radius-full); border-radius: var(--radius-full);
padding: 0.15rem 0.5rem; padding: 0.15rem 0.5rem;
} }
.mute-btn {
display: inline-flex;
align-items: center;
justify-content: center;
width: 30px;
height: 30px;
color: var(--fg);
background: var(--bg);
border: 1px solid var(--border);
border-radius: var(--radius-full);
cursor: pointer;
}
.mute-btn:hover {
background: var(--bg-hover);
}
.mute-btn.muted {
color: var(--danger, #d33);
border-color: var(--danger, #d33);
background: color-mix(in srgb, var(--danger, #d33) 12%, transparent);
}
.retention { .retention {
display: flex; display: flex;
align-items: center; align-items: center;
+72 -4
View File
@@ -56,6 +56,19 @@ export interface AudioDeviceInfo {
name: string; name: string;
} }
// Quick hardware stress test (Settings ▸ Hardware): per-(backend, model)
// real-time factor, plus the recommended real-time-capable pairing.
export interface StressResult {
backend: string;
model: string;
rtf: number;
realtime: boolean;
}
export interface StressTestResult {
results: StressResult[];
recommended: { backend: string; model: string } | null;
}
export interface LlmStatus { export interface LlmStatus {
provider: string; // ollama|custom|anthropic|off (ADR-0011; "openai" not yet wired) provider: string; // ollama|custom|anthropic|off (ADR-0011; "openai" not yet wired)
reachable: boolean; reachable: boolean;
@@ -83,6 +96,19 @@ export interface LanguageOption {
export type MeetingStatus = "recording" | "transcribing" | "ready" | "recovering" | "error"; export type MeetingStatus = "recording" | "transcribing" | "ready" | "recovering" | "error";
// The four ordered phases of a background media import (import://progress).
export type ImportPhase = "prepare" | "transcribe" | "diarize" | "finalize";
// One `import://progress` tick. `state` is active (running), done (finished,
// `elapsedMs` set) or error (`error` message set) for the given `phase`.
export interface ImportProgress {
meetingId: MeetingId;
phase: ImportPhase;
state: "active" | "done" | "error";
elapsedMs: number | null;
error: string | null;
}
export interface MeetingListItem { export interface MeetingListItem {
id: MeetingId; id: MeetingId;
title: string; title: string;
@@ -336,6 +362,12 @@ export interface AppSettings {
audio_output_device: string | null; audio_output_device: string | null;
microphone_enabled: boolean; microphone_enabled: boolean;
audio_input_device: string | null; audio_input_device: string | null;
/** Launch WhispAssist at login (opt-in, off by default; NFR-RES-4). Toggled
* via setAutoStart, which writes a per-user Run entry (no admin). */
auto_start: boolean;
/** Closing the window hides to the tray (keep running in background) instead
* of quitting; on by default. Tray "Quit" is the real exit. */
close_to_tray: boolean;
} }
// Feature brief — agent-ready spec distilled from a meeting (ADR-0011). // Feature brief — agent-ready spec distilled from a meeting (ADR-0011).
@@ -398,6 +430,10 @@ export const api = {
invoke<string>("recording_playback_path", { meetingId }), invoke<string>("recording_playback_path", { meetingId }),
pauseRecording: (meetingId: MeetingId) => invoke<void>("pause_recording", { meetingId }), pauseRecording: (meetingId: MeetingId) => invoke<void>("pause_recording", { meetingId }),
resumeRecording: (meetingId: MeetingId) => invoke<void>("resume_recording", { meetingId }), resumeRecording: (meetingId: MeetingId) => invoke<void>("resume_recording", { meetingId }),
// Toggle mic mute for the active recording (FR-CAP-7); returns the new muted
// state. Errors if the meeting was started with the mic off.
toggleMicrophoneMute: (meetingId: MeetingId) =>
invoke<boolean>("toggle_microphone_mute", { meetingId }),
setRecordingRetention: (meetingId: MeetingId, record: boolean) => setRecordingRetention: (meetingId: MeetingId, record: boolean) =>
invoke<void>("set_recording_retention", { meetingId, record }), invoke<void>("set_recording_retention", { meetingId, record }),
acknowledgeRecordingConsent: () => invoke<void>("acknowledge_recording_consent"), acknowledgeRecordingConsent: () => invoke<void>("acknowledge_recording_consent"),
@@ -417,6 +453,11 @@ export const api = {
listInputDevices: () => invoke<AudioDeviceInfo[]>("list_input_devices"), listInputDevices: () => invoke<AudioDeviceInfo[]>("list_input_devices"),
setPreferredBackend: (backend: BackendId | "auto") => setPreferredBackend: (backend: BackendId | "auto") =>
invoke<void>("set_preferred_backend", { args: { backend } }), invoke<void>("set_preferred_backend", { args: { backend } }),
setAutoStart: (enabled: boolean) => invoke<void>("set_auto_start", { enabled }),
// Test a device: stream device://level for a few seconds. Resolves when done.
monitorAudioLevel: (kind: "input" | "loopback", deviceId: string | null, durationMs = 6000) =>
invoke<void>("monitor_audio_level", { kind, deviceId, durationMs }),
stressTestHardware: () => invoke<StressTestResult>("stress_test_hardware"),
downloadNpuPackage: () => invoke<void>("download_npu_package"), downloadNpuPackage: () => invoke<void>("download_npu_package"),
downloadDirectmlPackage: () => invoke<void>("download_directml_package"), downloadDirectmlPackage: () => invoke<void>("download_directml_package"),
listModels: () => invoke<ModelInfo[]>("list_models"), listModels: () => invoke<ModelInfo[]>("list_models"),
@@ -437,10 +478,12 @@ export const api = {
invoke<void>("reprocess_transcript", { meetingId, model, language }), invoke<void>("reprocess_transcript", { meetingId, model, language }),
// Manually add a meeting from an existing recording — a local audio/video // Manually add a meeting from an existing recording — a local audio/video
// file path or a URL (YouTube/streaming page or direct media URL). Requires // file path or a URL (YouTube/streaming page or direct media URL). Requires
// ffmpeg (and yt-dlp for URLs) on PATH; neither is bundled. Returns the new // ffmpeg (and yt-dlp for URLs) on PATH; neither is bundled. `model` overrides
// meeting's id once transcription + diarization have finished. // the Settings whisper model for this one import. Returns the new meeting's id
importMedia: (source: string, title?: string) => // *immediately*; transcode/transcribe/diarize run in the background and stream
invoke<MeetingId>("import_media", { source, title }), // `import://progress` ticks, finishing with `transcript://finalized`.
importMedia: (source: string, title?: string, model?: string) =>
invoke<MeetingId>("import_media", { source, title, model }),
resumeTranscription: (meetingId: MeetingId) => resumeTranscription: (meetingId: MeetingId) =>
invoke<void>("resume_transcription", { meetingId }), invoke<void>("resume_transcription", { meetingId }),
listMeetings: (filter?: MeetingFilter) => listMeetings: (filter?: MeetingFilter) =>
@@ -460,6 +503,10 @@ export const api = {
deleteMeeting: (meetingId: MeetingId) => invoke<void>("delete_meeting", { meetingId }), deleteMeeting: (meetingId: MeetingId) => invoke<void>("delete_meeting", { meetingId }),
updateNotes: (meetingId: MeetingId, markdown: string) => updateNotes: (meetingId: MeetingId, markdown: string) =>
invoke<void>("update_notes", { meetingId, markdown }), invoke<void>("update_notes", { meetingId, markdown }),
// AI-enhance rough notes into structured Markdown grounded in the transcript
// (Granola-style). Returns the enhanced text; the caller decides to keep it.
enhanceNotes: (meetingId: MeetingId, notes: string) =>
invoke<string>("enhance_notes", { meetingId, notes }),
// dest is a file path for md/pdf/docx/obsidian, a folder for bundle. // dest is a file path for md/pdf/docx/obsidian, a folder for bundle.
// "obsidian" writes one self-contained vault note (no audio) — FR-STORE-4. // "obsidian" writes one self-contained vault note (no audio) — FR-STORE-4.
exportMeeting: ( exportMeeting: (
@@ -588,12 +635,24 @@ export const events = {
onDeviceChanged: ( onDeviceChanged: (
cb: (p: { meetingId: string; recovered: boolean; message: string }) => void, cb: (p: { meetingId: string; recovered: boolean; message: string }) => void,
): Promise<UnlistenFn> => listen("recording://device", (e) => cb(e.payload as never)), ): Promise<UnlistenFn> => listen("recording://device", (e) => cb(e.payload as never)),
// Mic mute toggled for the active recording (FR-CAP-7).
onMicMuted: (
cb: (p: { meetingId: string; muted: boolean }) => void,
): Promise<UnlistenFn> => listen("recording://mic", (e) => cb(e.payload as never)),
onSegment: ( onSegment: (
cb: (p: { meetingId: string; segment: TranscriptSegment }) => void, cb: (p: { meetingId: string; segment: TranscriptSegment }) => void,
): Promise<UnlistenFn> => listen("transcript://segment", (e) => cb(e.payload as never)), ): Promise<UnlistenFn> => listen("transcript://segment", (e) => cb(e.payload as never)),
onFinalized: ( onFinalized: (
cb: (p: { meetingId: string; segmentCount: number }) => void, cb: (p: { meetingId: string; segmentCount: number }) => void,
): Promise<UnlistenFn> => listen("transcript://finalized", (e) => cb(e.payload as never)), ): Promise<UnlistenFn> => listen("transcript://finalized", (e) => cb(e.payload as never)),
// Per-phase progress of a background media import (feeds the import tracker).
onImportProgress: (cb: (p: ImportProgress) => void): Promise<UnlistenFn> =>
listen("import://progress", (e) => cb(e.payload as never)),
// Live diarization refined the speaker list mid-recording (FR-SPK): updated
// labels/display names, including the mic speaker resolved to "You".
onDiarizationUpdated: (
cb: (p: { meetingId: string; speakers: SpeakerInfo[] }) => void,
): Promise<UnlistenFn> => listen("diarization://updated", (e) => cb(e.payload as never)),
onLlmToken: (cb: (p: { meetingId: string; text: string }) => void): Promise<UnlistenFn> => onLlmToken: (cb: (p: { meetingId: string; text: string }) => void): Promise<UnlistenFn> =>
listen("llm://token", (e) => cb(e.payload as never)), listen("llm://token", (e) => cb(e.payload as never)),
onLlmDone: (cb: (p: { meetingId: string; summary: SummaryFile }) => void): Promise<UnlistenFn> => onLlmDone: (cb: (p: { meetingId: string; summary: SummaryFile }) => void): Promise<UnlistenFn> =>
@@ -604,6 +663,15 @@ export const events = {
onHardwareChanged: ( onHardwareChanged: (
cb: (p: { active: BackendId; reason: string }) => void, cb: (p: { active: BackendId; reason: string }) => void,
): Promise<UnlistenFn> => listen("hardware://changed", (e) => cb(e.payload as never)), ): Promise<UnlistenFn> => listen("hardware://changed", (e) => cb(e.payload as never)),
// Live level meter for a device test (Settings ▸ Hardware). `done` marks the
// end of the monitor window.
onDeviceLevel: (
cb: (p: { kind: string; rms?: number; peak?: number; done?: boolean }) => void,
): Promise<UnlistenFn> => listen("device://level", (e) => cb(e.payload as never)),
// Per-(backend, model) progress ticks during the quick stress test.
onStressProgress: (
cb: (p: { backend: string; model: string }) => void,
): Promise<UnlistenFn> => listen("stress://progress", (e) => cb(e.payload as never)),
onNpuDownload: ( onNpuDownload: (
cb: (p: { cb: (p: {
stage: "model" | "runtime" | "done"; stage: "model" | "runtime" | "done";
+102 -21
View File
@@ -1,22 +1,43 @@
<script lang="ts"> <script lang="ts">
// Manually add a meeting from an existing recording (feature: "add a meeting // Manually add a meeting from an existing recording (feature: "add a meeting
// + upload a video URL or audio file"). Transcoding is done by the backend // + upload a video URL or audio file"). Transcoding is done by the backend
// via ffmpeg (+ yt-dlp for URLs) — both external, not bundled — so this is // via ffmpeg (+ yt-dlp for URLs) — both external, not bundled — so this is a
// just a small form: pick a local file or paste a URL, optional title, go. // small form: pick a file or URL, choose the transcription model, go. Import
import { api, errorMessage } from "../api"; // runs in the background (import_media returns as soon as the meeting row
// exists), so this closes immediately and the meetings list shows progress.
import { api, errorMessage, type ModelInfo } from "../api";
import { open } from "@tauri-apps/plugin-dialog"; import { open } from "@tauri-apps/plugin-dialog";
import { onMount } from "svelte";
import { trapFocus } from "../actions/trapFocus"; import { trapFocus } from "../actions/trapFocus";
import { t } from "../i18n/index.svelte"; import { t } from "../i18n/index.svelte";
import { X, FileUp, Link as LinkIcon } from "@lucide/svelte"; import { X, FileUp, Download } from "@lucide/svelte";
let { onClose, onImported }: { onClose: () => void; onImported: (id: string) => void } = $props(); let { onClose, onImported }: { onClose: () => void; onImported: (id: string) => void } = $props();
// External download pages for the two tools this feature shells out to.
const FFMPEG_URL = "https://github.com/BtbN/FFmpeg-Builds/releases/latest";
const YTDLP_URL = "https://github.com/yt-dlp/yt-dlp/releases/latest";
// `source` is either a local file path (set via Browse) or a URL (typed). // `source` is either a local file path (set via Browse) or a URL (typed).
let source = $state(""); let source = $state("");
let title = $state(""); let title = $state("");
let model = $state("");
let models = $state<ModelInfo[]>([]);
let busy = $state(false); let busy = $state(false);
let error = $state<string | null>(null); let error = $state<string | null>(null);
// Only installed whisper models are selectable; default to the active one so
// the pick matches the user's Settings default unless they change it here.
onMount(async () => {
try {
const all = await api.listModels();
models = all.filter((m) => m.installed);
model = models.find((m) => m.active)?.id ?? models[0]?.id ?? "";
} catch {
models = [];
}
});
async function browse() { async function browse() {
const path = await open({ const path = await open({
multiple: false, multiple: false,
@@ -51,7 +72,7 @@
busy = true; busy = true;
error = null; error = null;
try { try {
const id = await api.importMedia(source.trim(), title.trim() || undefined); const id = await api.importMedia(source.trim(), title.trim() || undefined, model || undefined);
onImported(id); onImported(id);
onClose(); onClose();
} catch (e) { } catch (e) {
@@ -101,17 +122,32 @@
</div> </div>
</label> </label>
<label class="wide"> <div class="grid">
{t("import.title_label")} <em>({t("import.optional")})</em> <label>
<input bind:value={title} placeholder={t("import.title_placeholder")} disabled={busy} /> {t("import.model_label")}
</label> <select bind:value={model} disabled={busy || models.length === 0}>
{#each models as m (m.id)}
<option value={m.id}>{m.label}</option>
{/each}
</select>
<span class="hint">{t("import.model_hint")}</span>
</label>
<p class="muted small"> <label>
<LinkIcon size={12} aria-hidden="true" /> {t("import.title_label")} <em>({t("import.optional")})</em>
{t("import.requires_1")} <code>ffmpeg</code> <input bind:value={title} placeholder={t("import.title_placeholder")} disabled={busy} />
{t("import.requires_2")} <code>yt-dlp</code> </label>
{t("import.requires_3")} </div>
</p>
<div class="tools">
<span class="muted small">{t("import.requires")}</span>
<button class="tool" type="button" onclick={() => api.openUrl(FFMPEG_URL)}>
<Download size={12} aria-hidden="true" /> ffmpeg
</button>
<button class="tool" type="button" onclick={() => api.openUrl(YTDLP_URL)}>
<Download size={12} aria-hidden="true" /> yt-dlp
</button>
</div>
{#if error} {#if error}
<p class="error">{error}</p> <p class="error">{error}</p>
@@ -121,6 +157,7 @@
<button class="primary" onclick={doImport} disabled={!source.trim() || busy}> <button class="primary" onclick={doImport} disabled={!source.trim() || busy}>
{busy ? t("import.importing") : t("import.import")} {busy ? t("import.importing") : t("import.import")}
</button> </button>
<span class="muted small note">{t("import.background_note")}</span>
<button class="link" onclick={onClose} disabled={busy}>{t("import.cancel")}</button> <button class="link" onclick={onClose} disabled={busy}>{t("import.cancel")}</button>
</div> </div>
</div> </div>
@@ -143,7 +180,7 @@
border: 1px solid var(--border); border: 1px solid var(--border);
border-radius: var(--radius-lg); border-radius: var(--radius-lg);
padding: 1.25rem; padding: 1.25rem;
width: min(520px, 100%); width: min(540px, 100%);
max-height: 90vh; max-height: 90vh;
overflow: auto; overflow: auto;
box-shadow: 0 12px 40px rgba(0, 0, 0, 0.3); box-shadow: 0 12px 40px rgba(0, 0, 0, 0.3);
@@ -179,7 +216,8 @@
font-weight: 400; font-weight: 400;
color: var(--muted); color: var(--muted);
} }
input { input,
select {
width: 100%; width: 100%;
box-sizing: border-box; box-sizing: border-box;
padding: 0.4rem 0.55rem; padding: 0.4rem 0.55rem;
@@ -189,10 +227,18 @@
color: var(--fg); color: var(--fg);
font: inherit; font: inherit;
} }
input:focus-visible { input:focus-visible,
select:focus-visible {
border-color: var(--accent); border-color: var(--accent);
outline: none; outline: none;
} }
.hint {
display: block;
margin-top: 0.25rem;
font-size: 0.75rem;
font-weight: 400;
color: var(--muted);
}
.row { .row {
display: flex; display: flex;
gap: 0.4rem; gap: 0.4rem;
@@ -213,18 +259,49 @@
padding: 0.4rem 0.6rem; padding: 0.4rem 0.6rem;
cursor: pointer; cursor: pointer;
} }
/* Model + title side by side on wide panels, stacked when cramped. */
.grid {
display: grid;
grid-template-columns: 1fr 1fr;
gap: 0 0.75rem;
}
@media (max-width: 460px) {
.grid {
grid-template-columns: 1fr;
}
}
.tools {
display: flex;
align-items: center;
flex-wrap: wrap;
gap: 0.4rem;
margin-top: 0.9rem;
}
.tool {
display: inline-flex;
align-items: center;
gap: 0.25rem;
background: var(--bg-hover, transparent);
color: var(--accent);
border: 1px solid var(--border);
border-radius: var(--radius-sm);
padding: 0.2rem 0.5rem;
font-size: 0.78rem;
cursor: pointer;
}
.tool:hover {
border-color: var(--accent);
}
.muted { .muted {
color: var(--muted); color: var(--muted);
} }
.small { .small {
font-size: 0.8rem; font-size: 0.8rem;
display: flex;
align-items: center;
gap: 0.3rem;
} }
.error { .error {
color: var(--danger, #d33); color: var(--danger, #d33);
font-size: 0.85rem; font-size: 0.85rem;
margin-top: 0.75rem;
} }
.actions { .actions {
display: flex; display: flex;
@@ -232,6 +309,10 @@
gap: 0.6rem; gap: 0.6rem;
margin-top: 1rem; margin-top: 1rem;
} }
.actions .note {
flex: 1;
line-height: 1.2;
}
.actions .primary { .actions .primary {
background: var(--accent); background: var(--accent);
color: var(--accent-fg, #fff); color: var(--accent-fg, #fff);
+214
View File
@@ -0,0 +1,214 @@
<script lang="ts">
// Domino's-pizza-tracker-style progress for a background media import: four
// ordered steps, the running one pulses, finished ones show how long they
// took. Fed by the `imports` store (import://progress events). Renders nothing
// until the first tick arrives. Design per ui-ux-pro-max: color is never the
// only signal (icon + label + time), tabular figures for the timers, and the
// pulse is dropped under prefers-reduced-motion.
import { imports } from "../stores/imports.svelte";
import { t } from "../i18n/index.svelte";
import { AudioLines, Captions, Users, FileCheck2, Check, X } from "@lucide/svelte";
let { meetingId }: { meetingId: string } = $props();
const run = $derived(imports.get(meetingId));
const ICONS = {
prepare: AudioLines,
transcribe: Captions,
diarize: Users,
finalize: FileCheck2,
} as const;
// ms → compact, human duration for a finished step ("820 ms", "4.3s", "2m 05s").
function fmtDur(ms: number | null): string {
if (ms == null) return "";
if (ms < 1000) return `${ms} ms`;
const s = ms / 1000;
if (s < 60) return `${s.toFixed(1)}s`;
const m = Math.floor(s / 60);
const rem = Math.round(s % 60);
return `${m}m ${String(rem).padStart(2, "0")}s`;
}
</script>
{#if run}
<section class="tracker" aria-label={t("import.tracker.label")}>
<header>
{#if run.error}
<span class="head err">{t("import.tracker.failed")}</span>
{:else if run.done}
<span class="head ok">{t("import.tracker.done")}</span>
{:else}
<span class="head">{t("import.tracker.running")}</span>
{/if}
</header>
<ol class="steps" aria-live="polite">
{#each run.phases as p (p.phase)}
{@const Icon = ICONS[p.phase]}
<li class="step {p.state}">
<div class="node">
{#if p.state === "done"}
<Check size={18} aria-hidden="true" />
{:else if p.state === "error"}
<X size={18} aria-hidden="true" />
{:else}
<Icon size={18} aria-hidden="true" />
{/if}
</div>
<div class="meta">
<span class="name">{t(`import.phase.${p.phase}`)}</span>
<span class="time">
{#if p.state === "done"}{fmtDur(p.elapsedMs)}
{:else if p.state === "active"}{t("import.tracker.working")}
{:else if p.state === "error"}{t("import.tracker.stopped")}
{/if}
</span>
</div>
</li>
{/each}
</ol>
{#if run.error}
<p class="msg">{run.error}</p>
{/if}
</section>
{/if}
<style>
.tracker {
border: 1px solid var(--border);
border-radius: var(--radius-sm);
background: var(--panel, var(--bg-elevated));
padding: 0.85rem 1rem 1rem;
}
header {
margin-bottom: 0.9rem;
}
.head {
font-size: 0.85rem;
font-weight: 600;
color: var(--fg);
}
.head.ok {
color: var(--success);
}
.head.err {
color: var(--danger);
}
.steps {
display: flex;
list-style: none;
margin: 0;
padding: 0;
}
.step {
flex: 1;
position: relative;
text-align: center;
min-width: 0;
}
/* Connector from the previous node's center to this one's (each step is the
same width, so -50%→+50% spans center to center), sitting behind the node. */
.step::before {
content: "";
position: absolute;
top: 17px;
left: -50%;
width: 100%;
height: 2px;
background: var(--border);
z-index: 0;
}
.step:first-child::before {
display: none;
}
.step.done::before,
.step.active::before,
.step.error::before {
background: var(--accent);
}
.node {
position: relative;
z-index: 1;
width: 36px;
height: 36px;
margin: 0 auto 0.45rem;
display: grid;
place-items: center;
border-radius: var(--radius-full);
border: 2px solid var(--border);
background: var(--bg);
color: var(--muted);
}
.step.active .node {
border-color: var(--accent);
background: var(--accent-soft, transparent);
color: var(--accent);
animation: pulse 1.4s ease-out infinite;
}
.step.done .node {
border-color: var(--success);
background: var(--success);
color: #fff;
}
.step.error .node {
border-color: var(--danger);
background: var(--danger);
color: #fff;
}
.meta {
display: flex;
flex-direction: column;
gap: 0.1rem;
padding: 0 0.2rem;
}
.name {
font-size: 0.78rem;
font-weight: 500;
color: var(--muted);
line-height: 1.2;
}
.step.active .name,
.step.done .name {
color: var(--fg);
}
.time {
font-size: 0.72rem;
color: var(--muted);
font-variant-numeric: tabular-nums;
min-height: 1em;
}
.step.active .time {
color: var(--accent);
}
.msg {
margin: 0.85rem 0 0;
font-size: 0.8rem;
color: var(--danger);
word-break: break-word;
}
@keyframes pulse {
0% {
box-shadow: 0 0 0 0 color-mix(in srgb, var(--accent) 45%, transparent);
}
70% {
box-shadow: 0 0 0 8px color-mix(in srgb, var(--accent) 0%, transparent);
}
100% {
box-shadow: 0 0 0 0 color-mix(in srgb, var(--accent) 0%, transparent);
}
}
@media (prefers-reduced-motion: reduce) {
.step.active .node {
animation: none;
box-shadow: 0 0 0 3px var(--accent-soft, transparent);
}
}
</style>
+55 -5
View File
@@ -24,6 +24,25 @@
"settings.transcription.switch_multilingual": "Switch to a multilingual model above to choose a language.", "settings.transcription.switch_multilingual": "Switch to a multilingual model above to choose a language.",
"settings.hardware.title": "Hardware", "settings.hardware.title": "Hardware",
"settings.hardware.refresh": "Refresh",
"settings.hardware.test_output": "Test",
"settings.hardware.test_mic": "Test",
"settings.hardware.testing": "Listening…",
"settings.hardware.play_tone": "Play tone",
"settings.hardware.stress_title": "Quick stress test",
"settings.hardware.stress_hint": "Benchmarks your installed models on each available backend and recommends the most accurate one that still keeps up with live speech. Takes a moment.",
"settings.hardware.stress_run": "Run stress test",
"settings.hardware.stress_running": "Running…",
"settings.hardware.stress_progress": "Benchmarking {pair}…",
"settings.hardware.stress_recommend": "Recommended: {backend} + {model}",
"settings.hardware.stress_apply": "Apply",
"settings.hardware.stress_none": "No installed model keeps up with live speech on this hardware — try a smaller model.",
"settings.hardware.stress_backend": "Backend",
"settings.hardware.stress_model": "Model",
"settings.hardware.stress_rtf": "Speed (×real-time)",
"settings.hardware.stress_realtime": "Live?",
"settings.hardware.stress_yes": "Yes",
"settings.hardware.stress_no": "No",
"settings.hardware.active_backend": "Active backend", "settings.hardware.active_backend": "Active backend",
"settings.hardware.model_meta": "· model {size}", "settings.hardware.model_meta": "· model {size}",
"settings.hardware.preferred_backend": "Preferred backend", "settings.hardware.preferred_backend": "Preferred backend",
@@ -88,6 +107,7 @@
"settings.calendar.title": "Calendar & Outlook .pst", "settings.calendar.title": "Calendar & Outlook .pst",
"settings.calendar.intro_1": "Import events and attendees from a local Outlook", "settings.calendar.intro_1": "Import events and attendees from a local Outlook",
"settings.calendar.intro_2": "backup — read-only, nothing is written back to the file. Nothing leaves this device.", "settings.calendar.intro_2": "backup — read-only, nothing is written back to the file. Nothing leaves this device.",
"settings.calendar.requires": "Needs this on your PATH (not bundled):",
"settings.calendar.pst_file": ".pst file", "settings.calendar.pst_file": ".pst file",
"settings.calendar.no_file": "No file selected", "settings.calendar.no_file": "No file selected",
"settings.calendar.browse": "Browse…", "settings.calendar.browse": "Browse…",
@@ -278,6 +298,7 @@
"settings.privacy.locked_word": "locked", "settings.privacy.locked_word": "locked",
"settings.privacy.vault_locked_2": ". Unlock to read encrypted meetings.", "settings.privacy.vault_locked_2": ". Unlock to read encrypted meetings.",
"settings.privacy.password": "Password", "settings.privacy.password": "Password",
"settings.privacy.show_password": "Show password",
"settings.privacy.unlock": "Unlock", "settings.privacy.unlock": "Unlock",
"settings.privacy.vault_unlocked_1": "Vault is ", "settings.privacy.vault_unlocked_1": "Vault is ",
"settings.privacy.unlocked_word": "unlocked", "settings.privacy.unlocked_word": "unlocked",
@@ -320,12 +341,23 @@
"import.optional": "optional", "import.optional": "optional",
"import.title_placeholder": "Defaults to the file name", "import.title_placeholder": "Defaults to the file name",
"import.filter_av": "Audio / video", "import.filter_av": "Audio / video",
"import.requires_1": "Requires", "import.model_label": "Transcription model",
"import.requires_2": "installed and on your PATH (plus", "import.model_hint": "Recorded with the meeting so you can see how it was transcribed.",
"import.requires_3": "for URLs). WhispAssist doesn't bundle them.", "import.requires": "Needs these on your PATH (not bundled):",
"import.importing": "Importing… this can take a while", "import.background_note": "Runs in the background — track it in the list.",
"import.importing": "Starting…",
"import.import": "Import", "import.import": "Import",
"import.cancel": "Cancel", "import.cancel": "Cancel",
"import.tracker.label": "Import progress",
"import.tracker.running": "Importing…",
"import.tracker.done": "Import complete",
"import.tracker.failed": "Import failed",
"import.tracker.working": "working…",
"import.tracker.stopped": "stopped",
"import.phase.prepare": "Transcode",
"import.phase.transcribe": "Transcribe",
"import.phase.diarize": "Identify speakers",
"import.phase.finalize": "Finalize",
"tagchip.filter": "Filter meetings tagged \"{tag}\"", "tagchip.filter": "Filter meetings tagged \"{tag}\"",
"tagchip.remove": "Remove tag {tag}", "tagchip.remove": "Remove tag {tag}",
@@ -345,6 +377,10 @@
"app.cancel": "Cancel", "app.cancel": "Cancel",
"app.cancel_title": "Discard this recording and delete it", "app.cancel_title": "Discard this recording and delete it",
"app.recording": "Recording…", "app.recording": "Recording…",
"app.mute": "Mute microphone",
"app.unmute": "Unmute microphone",
"app.mute_title": "Mute microphone (M)",
"app.unmute_title": "Unmute microphone (M)",
"app.backend_title": "Active transcription backend", "app.backend_title": "Active transcription backend",
"app.retention_title": "Save audio as .wav for this meeting", "app.retention_title": "Save audio as .wav for this meeting",
"app.saving": "saving", "app.saving": "saving",
@@ -397,6 +433,7 @@
"transcript.heading": "Transcript", "transcript.heading": "Transcript",
"transcript.title_aria": "Meeting title", "transcript.title_aria": "Meeting title",
"transcript.transcribed_with": "Transcribed with",
"transcript.lang_title": "Transcription language", "transcript.lang_title": "Transcription language",
"transcript.lang_auto": "auto-detecting…", "transcript.lang_auto": "auto-detecting…",
"transcript.show": "Show transcript", "transcript.show": "Show transcript",
@@ -428,9 +465,18 @@
"notes.italic": "Italic", "notes.italic": "Italic",
"notes.h1": "Heading 1", "notes.h1": "Heading 1",
"notes.h2": "Heading 2", "notes.h2": "Heading 2",
"notes.h3": "Heading 3",
"notes.bullet": "Bullet list", "notes.bullet": "Bullet list",
"notes.numbered": "Numbered list",
"notes.quote": "Quote",
"notes.divider": "Divider",
"notes.checkbox_title": "Checkbox", "notes.checkbox_title": "Checkbox",
"notes.checkbox_aria": "Checkbox list item", "notes.checkbox_aria": "Checkbox list item",
"notes.enhance": "Enhance",
"notes.enhancing": "Enhancing…",
"notes.enhance_title": "Expand these notes into structured Markdown using the transcript (AI)",
"notes.enhanced_note": "Notes enhanced from the transcript.",
"notes.undo_enhance": "Undo",
"notes.edit_raw": "Edit the raw markdown", "notes.edit_raw": "Edit the raw markdown",
"notes.render": "Render the markdown", "notes.render": "Render the markdown",
"notes.editor": "Editor", "notes.editor": "Editor",
@@ -527,5 +573,9 @@
"settings.recording.consent_ack": "acknowledged", "settings.recording.consent_ack": "acknowledged",
"settings.recording.consent_not": "not yet acknowledged", "settings.recording.consent_not": "not yet acknowledged",
"settings.recording.auto_label": "Auto-start recording when a calendar event begins", "settings.recording.auto_label": "Auto-start recording when a calendar event begins",
"settings.recording.auto_hint": "Only while WhispAssist is open. When an imported calendar event's start time arrives, a recording begins automatically (using your default retention setting above). Nothing runs in the background — the timer is armed only while the app is running. Import events under Settings → Calendar." "settings.recording.auto_hint": "Only while WhispAssist is open. When an imported calendar event's start time arrives, a recording begins automatically (using your default retention setting above). Nothing runs in the background — the timer is armed only while the app is running. Import events under Settings → Calendar.",
"settings.recording.autostart_label": "Launch WhispAssist at login",
"settings.recording.autostart_hint": "Starts WhispAssist automatically when you sign in to Windows. Off by default; installs a per-user startup entry (no admin required) and does not begin recording on its own.",
"settings.recording.close_tray_label": "Close to system tray",
"settings.recording.close_tray_hint": "Closing the window keeps WhispAssist running in the background instead of quitting. Reopen it from the tray icon; use the tray's Quit to exit fully. On by default."
} }
+63
View File
@@ -0,0 +1,63 @@
// Live per-meeting progress of background media imports (feeds ImportTracker).
// Fed entirely by `import://progress` events emitted by `import_media`; kept in
// memory only (the meeting's `status` badge is the persistent story after a
// restart). See commands.rs `run_import_pipeline`.
import { events, type ImportPhase, type ImportProgress, type MeetingId } from "../api";
export type PhaseState = "pending" | "active" | "done" | "error";
// The four phases in the order the backend runs (and the tracker renders) them.
export const IMPORT_PHASES: ImportPhase[] = ["prepare", "transcribe", "diarize", "finalize"];
export interface PhaseInfo {
phase: ImportPhase;
state: PhaseState;
elapsedMs: number | null;
}
export interface ImportRun {
meetingId: MeetingId;
phases: PhaseInfo[];
error: string | null;
done: boolean;
}
function freshRun(meetingId: MeetingId): ImportRun {
return {
meetingId,
phases: IMPORT_PHASES.map((phase) => ({ phase, state: "pending", elapsedMs: null })),
error: null,
done: false,
};
}
class ImportsStore {
runs = $state<Record<MeetingId, ImportRun>>({});
get(meetingId: MeetingId): ImportRun | undefined {
return this.runs[meetingId];
}
async init() {
await events.onImportProgress((p) => this.apply(p));
}
private apply(p: ImportProgress) {
// Re-read through the record after inserting so we mutate the $state proxy,
// not the raw object (Svelte 5 deep reactivity only tracks the proxy).
if (!this.runs[p.meetingId]) this.runs[p.meetingId] = freshRun(p.meetingId);
const run = this.runs[p.meetingId];
const info = run.phases.find((x) => x.phase === p.phase);
if (!info) return;
info.state = p.state;
if (p.state === "done") info.elapsedMs = p.elapsedMs;
if (p.state === "error") {
run.error = p.error;
run.done = true;
}
if (p.phase === "finalize" && p.state === "done") run.done = true;
}
}
export const imports = new ImportsStore();
+34 -2
View File
@@ -1,7 +1,7 @@
// Recording state store (Svelte 5 runes-friendly via a small class). // Recording state store (Svelte 5 runes-friendly via a small class).
// Subscribes to recording/transcript events and exposes reactive state. // Subscribes to recording/transcript events and exposes reactive state.
import { api, events, type TranscriptSegment, type MeetingId } from "../api"; import { api, events, type TranscriptSegment, type SpeakerInfo, type MeetingId } from "../api";
import { settings } from "./settings.svelte"; import { settings } from "./settings.svelte";
import { SvelteMap } from "svelte/reactivity"; import { SvelteMap } from "svelte/reactivity";
@@ -10,6 +10,9 @@ class RecordingStore {
state = $state<"idle" | "recording" | "paused">("idle"); state = $state<"idle" | "recording" | "paused">("idle");
elapsedMs = $state(0); elapsedMs = $state(0);
segments = $state<TranscriptSegment[]>([]); segments = $state<TranscriptSegment[]>([]);
/** Live speaker list from provisional diarization (FR-SPK): resolves labels
* to display names ("You" for the mic speaker) for the live transcript. */
speakers = $state<SpeakerInfo[]>([]);
/** Whether audio is being retained as .wav for the in-flight meeting (ADR-0009). */ /** Whether audio is being retained as .wav for the in-flight meeting (ADR-0009). */
retention = $state(false); retention = $state(false);
/** Live system/loopback level for the waveform/meter (FR-CAP-5); 0 when not recording. */ /** Live system/loopback level for the waveform/meter (FR-CAP-5); 0 when not recording. */
@@ -19,6 +22,9 @@ class RecordingStore {
* (FR-CAP-7); stays 0 when the mic is disabled or not recording. */ * (FR-CAP-7); stays 0 when the mic is disabled or not recording. */
levelRmsMic = $state(0); levelRmsMic = $state(0);
levelPeakMic = $state(0); levelPeakMic = $state(0);
/** Mic muted for the in-flight meeting (FR-CAP-7): mic channel goes silent
* while loopback keeps recording. Toggled by the "M" key / mute button. */
micMuted = $state(false);
/** Set while a capture-device reconnect is in progress; cleared on recovery (FR-CAP-6). */ /** Set while a capture-device reconnect is in progress; cleared on recovery (FR-CAP-6). */
deviceNotice = $state<string | null>(null); deviceNotice = $state<string | null>(null);
/** Live notes redesign: freeform text typed in the Notes pane while recording. */ /** Live notes redesign: freeform text typed in the Notes pane while recording. */
@@ -47,11 +53,16 @@ class RecordingStore {
this.retention = p.record; this.retention = p.record;
}); });
await events.onSegment(({ segment }) => { await events.onSegment(({ segment }) => {
// Replace an interim segment with the same id, else append. // Replace an interim segment with the same id, else append. Live
// diarization also re-emits committed segments here with a refined
// speaker (FR-SPK) — same replace-by-id path applies.
const i = this.segments.findIndex((s) => s.id === segment.id); const i = this.segments.findIndex((s) => s.id === segment.id);
if (i >= 0) this.segments[i] = segment; if (i >= 0) this.segments[i] = segment;
else this.segments.push(segment); else this.segments.push(segment);
}); });
await events.onDiarizationUpdated(({ speakers }) => {
this.speakers = speakers;
});
await events.onLevel(({ rms, peak, mic }) => { await events.onLevel(({ rms, peak, mic }) => {
if (mic) { if (mic) {
this.levelRmsMic = rms; this.levelRmsMic = rms;
@@ -64,12 +75,30 @@ class RecordingStore {
await events.onDeviceChanged(({ recovered, message }) => { await events.onDeviceChanged(({ recovered, message }) => {
this.deviceNotice = recovered ? null : message; this.deviceNotice = recovered ? null : message;
}); });
// Keep mute state in sync even if it was toggled elsewhere (e.g. a future
// tray control), not just from this store's toggleMute().
await events.onMicMuted(({ muted }) => {
this.micMuted = muted;
});
}
/** Toggle mic mute for the active recording (FR-CAP-7); no-op if not
* recording. Optimistically flips, then reconciles with the backend result. */
async toggleMute() {
if (!this.meetingId || this.state === "idle") return;
try {
this.micMuted = await api.toggleMicrophoneMute(this.meetingId);
} catch {
// Mic off for this meeting (or capture gone) — nothing to mute.
}
} }
async start(title?: string, record = false, templateId?: string, calendarEventId?: string) { async start(title?: string, record = false, templateId?: string, calendarEventId?: string) {
this.segments = []; this.segments = [];
this.speakers = [];
this.retention = record; this.retention = record;
this.deviceNotice = null; this.deviceNotice = null;
this.micMuted = false;
this.notesText = ""; this.notesText = "";
this.segmentNotes.clear(); this.segmentNotes.clear();
// T8.7/FR-TRX-4: whatever language is currently configured in Settings // T8.7/FR-TRX-4: whatever language is currently configured in Settings
@@ -89,6 +118,7 @@ class RecordingStore {
this.levelPeak = 0; this.levelPeak = 0;
this.levelRmsMic = 0; this.levelRmsMic = 0;
this.levelPeakMic = 0; this.levelPeakMic = 0;
this.micMuted = false;
this.deviceNotice = null; this.deviceNotice = null;
} }
@@ -101,11 +131,13 @@ class RecordingStore {
if (this.meetingId) await api.cancelRecording(this.meetingId); if (this.meetingId) await api.cancelRecording(this.meetingId);
this.meetingId = null; this.meetingId = null;
this.segments = []; this.segments = [];
this.speakers = [];
this.state = "idle"; this.state = "idle";
this.levelRms = 0; this.levelRms = 0;
this.levelPeak = 0; this.levelPeak = 0;
this.levelRmsMic = 0; this.levelRmsMic = 0;
this.levelPeakMic = 0; this.levelPeakMic = 0;
this.micMuted = false;
this.deviceNotice = null; this.deviceNotice = null;
this.notesText = ""; this.notesText = "";
this.segmentNotes.clear(); this.segmentNotes.clear();
+14
View File
@@ -48,6 +48,8 @@ const DEFAULT_SETTINGS: AppSettings = {
audio_output_device: null, // system default render device (FR-CAP-1) audio_output_device: null, // system default render device (FR-CAP-1)
microphone_enabled: true, // capture the user's mic into the transcript (FR-CAP-7) microphone_enabled: true, // capture the user's mic into the transcript (FR-CAP-7)
audio_input_device: null, // system default capture device audio_input_device: null, // system default capture device
auto_start: false, // launch at login — opt-in, off by default (NFR-RES-4)
close_to_tray: true, // closing the window hides to tray; on by default
}; };
class SettingsStore { class SettingsStore {
@@ -383,6 +385,18 @@ class SettingsStore {
return this.patch({ default_record: on }); return this.patch({ default_record: on });
} }
/** Launch-at-login toggle (NFR-RES-4). Goes through its own command (not
* patch) since the backend also writes the per-user OS Run entry; that
* command persists auto_start itself, so we just mirror it locally. */
async setAutoStart(on: boolean) {
this.settings = { ...this.settings, auto_start: on };
try {
await api.setAutoStart(on);
} catch {
this.backendStub = true;
}
}
setRetentionPolicy(maxAgeDays: number | null, maxSizeGb: number | null) { setRetentionPolicy(maxAgeDays: number | null, maxSizeGb: number | null) {
return this.patch({ retention_max_age_days: maxAgeDays, retention_max_size_gb: maxSizeGb }); return this.patch({ retention_max_age_days: maxAgeDays, retention_max_size_gb: maxSizeGb });
} }
+480 -68
View File
@@ -9,9 +9,16 @@
import { t, i18n, LOCALES } from "../i18n/index.svelte"; import { t, i18n, LOCALES } from "../i18n/index.svelte";
import ConsentNotice from "../components/ConsentNotice.svelte"; import ConsentNotice from "../components/ConsentNotice.svelte";
import HostedAiBanner from "../components/HostedAiBanner.svelte"; import HostedAiBanner from "../components/HostedAiBanner.svelte";
import LevelMeter from "../components/LevelMeter.svelte";
import { open } from "@tauri-apps/plugin-dialog"; import { open } from "@tauri-apps/plugin-dialog";
import { api, errorMessage, events } from "../api"; import { api, errorMessage, events } from "../api";
import type { BackendId, SyncKind, SyncTargetConfig, SyncTargetInfo } from "../api"; import type {
BackendId,
StressTestResult,
SyncKind,
SyncTargetConfig,
SyncTargetInfo,
} from "../api";
import { trapFocus } from "../actions/trapFocus"; import { trapFocus } from "../actions/trapFocus";
import { import {
X, X,
@@ -23,7 +30,11 @@
CalendarDays, CalendarDays,
UploadCloud, UploadCloud,
ShieldCheck, ShieldCheck,
Lock,
LockOpen,
Sparkles, Sparkles,
Volume2,
Zap,
RefreshCw, RefreshCw,
ChevronRight, ChevronRight,
RotateCcw, RotateCcw,
@@ -275,6 +286,11 @@
} }
// ---- Calendar / .pst import (T6.1/T6.2/T6.3, FR-CAL-1/2) ---- // ---- Calendar / .pst import (T6.1/T6.2/T6.3, FR-CAL-1/2) ----
// .pst import shells out to readpst (libpst, ADR-0008) — external like
// ffmpeg/yt-dlp, so link the same kind of download button as the Import
// dialog's. ezwinports is the maintained Windows build of libpst.
const LIBPST_URL =
"https://sourceforge.net/projects/ezwinports/files/libpst-0.6.63-w32-bin.zip/download";
let pstPath = $state(""); let pstPath = $state("");
let pstPassword = $state(""); let pstPassword = $state("");
let eventTitleFilter = $state(""); let eventTitleFilter = $state("");
@@ -381,11 +397,98 @@
let showConsent = $state(false); let showConsent = $state(false);
let testResult = $state<{ ok: boolean; message: string } | null>(null); let testResult = $state<{ ok: boolean; message: string } | null>(null);
// ---- Audio device test (live level meter) ----
let monitorKind = $state<"input" | "loopback" | null>(null);
let monitorRms = $state(0);
let monitorPeak = $state(0);
let monitorUnlisten: (() => void) | null = null;
function stopMonitor() {
monitorUnlisten?.();
monitorUnlisten = null;
monitorKind = null;
monitorRms = 0;
monitorPeak = 0;
}
async function testDevice(kind: "input" | "loopback") {
if (monitorKind) return;
monitorKind = kind;
monitorRms = 0;
monitorPeak = 0;
monitorUnlisten = await events.onDeviceLevel((p) => {
if (p.kind !== kind) return;
if (p.done) {
stopMonitor();
return;
}
monitorRms = p.rms ?? 0;
monitorPeak = p.peak ?? 0;
});
const deviceId =
kind === "input"
? (settings.settings.audio_input_device ?? null)
: (settings.settings.audio_output_device ?? null);
try {
await api.monitorAudioLevel(kind, deviceId, 6000);
} catch (e) {
testResult = { ok: false, message: errorMessage(e) };
} finally {
stopMonitor();
}
}
// A 440Hz beep to the default output so the user can confirm speakers work.
function playTone() {
try {
const ctx = new AudioContext();
const osc = ctx.createOscillator();
const gain = ctx.createGain();
osc.frequency.value = 440;
gain.gain.value = 0.15;
osc.connect(gain).connect(ctx.destination);
osc.start();
osc.stop(ctx.currentTime + 0.5);
osc.onended = () => ctx.close();
} catch {
/* no Web Audio available */
}
}
// ---- Quick hardware stress test ----
let stressRunning = $state(false);
let stressProgress = $state<string | null>(null);
let stressResult = $state<StressTestResult | null>(null);
let stressError = $state<string | null>(null);
async function runStressTest() {
if (stressRunning) return;
stressRunning = true;
stressError = null;
stressResult = null;
const un = await events.onStressProgress((p) => {
stressProgress = `${p.backend} · ${p.model}`;
});
try {
stressResult = await api.stressTestHardware();
} catch (e) {
stressError = errorMessage(e);
} finally {
un();
stressProgress = null;
stressRunning = false;
}
}
async function applyRecommendation() {
const r = stressResult?.recommended;
if (!r) return;
await settings.setPreferredBackend(r.backend as BackendId | "auto");
await settings.patch({ whisper_model: r.model });
}
// ---- At-rest encryption vault (T8.8, FR-SEC-3) ---- // ---- At-rest encryption vault (T8.8, FR-SEC-3) ----
let vault = $state<{ enabled: boolean; unlocked: boolean } | null>(null); let vault = $state<{ enabled: boolean; unlocked: boolean } | null>(null);
let vaultPw = $state(""); let vaultPw = $state("");
let vaultPw2 = $state(""); let vaultPw2 = $state("");
let vaultMsg = $state<string | null>(null); let vaultMsg = $state<string | null>(null);
let vaultMsgError = $state(false);
let showVaultPw = $state(false);
async function loadVault() { async function loadVault() {
try { try {
vault = await api.vaultStatus(); vault = await api.vaultStatus();
@@ -394,42 +497,46 @@
} }
} }
onMount(loadVault); onMount(loadVault);
function setVaultMsg(msg: string | null, isError = false) {
vaultMsg = msg;
vaultMsgError = isError;
}
async function enableVault() { async function enableVault() {
vaultMsg = null; setVaultMsg(null);
try { try {
await api.enableVault(vaultPw); await api.enableVault(vaultPw);
vaultPw = ""; vaultPw = "";
vaultMsg = "Vault enabled and unlocked."; setVaultMsg("Vault enabled and unlocked.");
await loadVault(); await loadVault();
} catch (e) { } catch (e) {
vaultMsg = errorMessage(e); setVaultMsg(errorMessage(e), true);
} }
} }
async function unlockVault() { async function unlockVault() {
vaultMsg = null; setVaultMsg(null);
try { try {
await api.unlockVault(vaultPw); await api.unlockVault(vaultPw);
vaultPw = ""; vaultPw = "";
vaultMsg = "Unlocked."; setVaultMsg("Unlocked.");
await loadVault(); await loadVault();
} catch (e) { } catch (e) {
vaultMsg = errorMessage(e); setVaultMsg(errorMessage(e), true);
} }
} }
async function lockVault() { async function lockVault() {
await api.lockVault(); await api.lockVault();
vaultMsg = "Locked."; setVaultMsg("Locked.");
await loadVault(); await loadVault();
} }
async function changeVaultPassword() { async function changeVaultPassword() {
vaultMsg = null; setVaultMsg(null);
try { try {
await api.changeVaultPassword(vaultPw, vaultPw2); await api.changeVaultPassword(vaultPw, vaultPw2);
vaultPw = ""; vaultPw = "";
vaultPw2 = ""; vaultPw2 = "";
vaultMsg = "Password changed."; setVaultMsg("Password changed.");
} catch (e) { } catch (e) {
vaultMsg = errorMessage(e); setVaultMsg(errorMessage(e), true);
} }
} }
@@ -754,13 +861,40 @@
</label> </label>
<p class="muted">{t("settings.recording.auto_hint")}</p> <p class="muted">{t("settings.recording.auto_hint")}</p>
<label class="row">
<input
type="checkbox"
checked={settings.settings.auto_start}
onchange={(e) => settings.setAutoStart((e.target as HTMLInputElement).checked)}
/>
<span>{t("settings.recording.autostart_label")}</span>
</label>
<p class="muted">{t("settings.recording.autostart_hint")}</p>
<label class="row">
<input
type="checkbox"
checked={settings.settings.close_to_tray}
onchange={(e) =>
settings.patch({ close_to_tray: (e.target as HTMLInputElement).checked })}
/>
<span>{t("settings.recording.close_tray_label")}</span>
</label>
<p class="muted">{t("settings.recording.close_tray_hint")}</p>
{#if showConsent} {#if showConsent}
<ConsentNotice onAccept={acceptConsent} onCancel={() => (showConsent = false)} /> <ConsentNotice onAccept={acceptConsent} onCancel={() => (showConsent = false)} />
{/if} {/if}
</section> </section>
{:else if section === "hardware"} {:else if section === "hardware"}
<section> <section>
<h3>{t("settings.hardware.title")}</h3> <div class="actions">
<h3>{t("settings.hardware.title")}</h3>
<button class="ghost" onclick={() => settings.loadHardware()}>
<RefreshCw size={14} aria-hidden="true" />
{t("settings.hardware.refresh")}
</button>
</div>
{#if settings.hardware} {#if settings.hardware}
<div class="row"> <div class="row">
{t("settings.hardware.active_backend")}<code>{settings.hardware.active}</code> {t("settings.hardware.active_backend")}<code>{settings.hardware.active}</code>
@@ -804,6 +938,20 @@
</select> </select>
</label> </label>
<p class="muted">{t("settings.hardware.recording_device_hint")}</p> <p class="muted">{t("settings.hardware.recording_device_hint")}</p>
<div class="device-test">
<button onclick={() => testDevice("loopback")} disabled={monitorKind !== null}>
<Volume2 size={13} aria-hidden="true" />
{monitorKind === "loopback"
? t("settings.hardware.testing")
: t("settings.hardware.test_output")}
</button>
<button onclick={playTone} disabled={monitorKind !== null}>
{t("settings.hardware.play_tone")}
</button>
{#if monitorKind === "loopback"}
<LevelMeter rms={monitorRms} peak={monitorPeak} />
{/if}
</div>
<label <label
>{t("settings.hardware.microphone")} >{t("settings.hardware.microphone")}
@@ -825,6 +973,77 @@
</select> </select>
</label> </label>
<p class="muted">{t("settings.hardware.mic_hint")}</p> <p class="muted">{t("settings.hardware.mic_hint")}</p>
<div class="device-test">
<button
onclick={() => testDevice("input")}
disabled={monitorKind !== null || !settings.settings.microphone_enabled}
>
<Mic size={13} aria-hidden="true" />
{monitorKind === "input"
? t("settings.hardware.testing")
: t("settings.hardware.test_mic")}
</button>
{#if monitorKind === "input"}
<LevelMeter rms={monitorRms} peak={monitorPeak} />
{/if}
</div>
<h4>{t("settings.hardware.stress_title")}</h4>
<p class="muted">{t("settings.hardware.stress_hint")}</p>
<button onclick={runStressTest} disabled={stressRunning}>
<Zap size={13} aria-hidden="true" />
{stressRunning
? t("settings.hardware.stress_running")
: t("settings.hardware.stress_run")}
</button>
{#if stressProgress}
<p class="muted">{t("settings.hardware.stress_progress", { pair: stressProgress })}</p>
{/if}
{#if stressError}<p class="muted err">{stressError}</p>{/if}
{#if stressResult}
{#if stressResult.recommended}
<div class="stress-rec">
<ShieldCheck size={14} aria-hidden="true" />
<span
>{t("settings.hardware.stress_recommend", {
backend: stressResult.recommended.backend,
model: stressResult.recommended.model,
})}</span
>
<button class="primary" onclick={applyRecommendation}
>{t("settings.hardware.stress_apply")}</button
>
</div>
{:else}
<p class="muted">{t("settings.hardware.stress_none")}</p>
{/if}
<table class="stress-table">
<thead>
<tr>
<th>{t("settings.hardware.stress_backend")}</th>
<th>{t("settings.hardware.stress_model")}</th>
<th>{t("settings.hardware.stress_rtf")}</th>
<th>{t("settings.hardware.stress_realtime")}</th>
</tr>
</thead>
<tbody>
{#each stressResult.results as r (r.backend + r.model)}
<tr>
<td>{r.backend}</td>
<td>{r.model}</td>
<td class="num">{r.rtf.toFixed(2)}×</td>
<td>
{#if r.realtime}
<Check size={13} aria-hidden="true" /> {t("settings.hardware.stress_yes")}
{:else}
{t("settings.hardware.stress_no")}
{/if}
</td>
</tr>
{/each}
</tbody>
</table>
{/if}
{#if settings.hardware.npu?.present} {#if settings.hardware.npu?.present}
{@const npu = settings.hardware.npu} {@const npu = settings.hardware.npu}
@@ -1132,6 +1351,12 @@
{t("settings.calendar.intro_1")} <code>.pst</code> {t("settings.calendar.intro_1")} <code>.pst</code>
{t("settings.calendar.intro_2")} {t("settings.calendar.intro_2")}
</p> </p>
<div class="tools">
<span class="muted small">{t("settings.calendar.requires")}</span>
<button class="tool" type="button" onclick={() => api.openUrl(LIBPST_URL)}>
<Download size={12} aria-hidden="true" /> libpst (readpst)
</button>
</div>
<div class="grid"> <div class="grid">
<label class="wide" <label class="wide"
>{t("settings.calendar.pst_file")} >{t("settings.calendar.pst_file")}
@@ -1918,67 +2143,118 @@
{/if} {/if}
{#if vault} {#if vault}
<h4>{t("settings.privacy.vault_title")}</h4> <div
{#if !vault.enabled} class="vault-card"
<p class="muted">{t("settings.privacy.vault_intro")}</p> class:locked={vault.enabled && !vault.unlocked}
<div class="grid"> class:unlocked={vault.enabled && vault.unlocked}
<label class="wide" >
>{t("settings.privacy.vault_password")}<input <div class="vault-head">
type="password" {#if !vault.enabled}
bind:value={vaultPw} <ShieldCheck size={18} aria-hidden="true" />
/></label {:else if !vault.unlocked}
> <Lock size={18} aria-hidden="true" />
{:else}
<LockOpen size={18} aria-hidden="true" />
{/if}
<h4>{t("settings.privacy.vault_title")}</h4>
{#if vault.enabled}
<span class="badge" class:busy={!vault.unlocked}>
{vault.unlocked
? t("settings.privacy.unlocked_word")
: t("settings.privacy.locked_word")}
</span>
{/if}
</div> </div>
<button class="primary" onclick={enableVault} disabled={vaultPw.length < 8}
>{t("settings.privacy.enable_vault")}</button {#if !vault.enabled}
> <p class="muted">{t("settings.privacy.vault_intro")}</p>
<p class="muted">{t("settings.privacy.vault_pw_hint")}</p> <div class="pw-row">
{:else if !vault.unlocked} <input
<p class="muted"> type={showVaultPw ? "text" : "password"}
{t("settings.privacy.vault_locked_1")}<strong
>{t("settings.privacy.locked_word")}</strong
>{t("settings.privacy.vault_locked_2")}
</p>
<div class="grid">
<label class="wide"
>{t("settings.privacy.password")}<input
type="password"
bind:value={vaultPw} bind:value={vaultPw}
/></label placeholder={t("settings.privacy.vault_password")}
> aria-label={t("settings.privacy.vault_password")}
</div> />
<button class="primary" onclick={unlockVault} disabled={!vaultPw} <button
>{t("settings.privacy.unlock")}</button type="button"
> class="icon pw-toggle"
{:else} onclick={() => (showVaultPw = !showVaultPw)}
<p class="muted"> aria-label={t("settings.privacy.show_password")}
{t("settings.privacy.vault_unlocked_1")}<strong title={t("settings.privacy.show_password")}
>{t("settings.privacy.unlocked_word")}</strong
>{t("settings.privacy.vault_unlocked_2")}
</p>
<button onclick={lockVault}>{t("settings.privacy.lock_now")}</button>
<details>
<summary>{t("settings.privacy.change_password")}</summary>
<div class="grid">
<label class="wide"
>{t("settings.privacy.current_password")}<input
type="password"
bind:value={vaultPw}
/></label
>
<label class="wide"
>{t("settings.privacy.new_password")}<input
type="password"
bind:value={vaultPw2}
/></label
> >
{#if showVaultPw}<EyeOff size={14} aria-hidden="true" />{:else}<Eye
size={14}
aria-hidden="true"
/>{/if}
</button>
</div> </div>
<button onclick={changeVaultPassword} disabled={!vaultPw || vaultPw2.length < 8} <button class="primary" onclick={enableVault} disabled={vaultPw.length < 8}
>{t("settings.privacy.change_password")}</button >{t("settings.privacy.enable_vault")}</button
> >
</details> <p class="muted hint">{t("settings.privacy.vault_pw_hint")}</p>
{/if} {:else if !vault.unlocked}
{#if vaultMsg}<p class="muted">{vaultMsg}</p>{/if} <p class="muted">
{t("settings.privacy.vault_locked_1")}<strong
>{t("settings.privacy.locked_word")}</strong
>{t("settings.privacy.vault_locked_2")}
</p>
<div class="pw-row">
<input
type={showVaultPw ? "text" : "password"}
bind:value={vaultPw}
placeholder={t("settings.privacy.password")}
aria-label={t("settings.privacy.password")}
onkeydown={(e) => e.key === "Enter" && vaultPw && unlockVault()}
/>
<button
type="button"
class="icon pw-toggle"
onclick={() => (showVaultPw = !showVaultPw)}
aria-label={t("settings.privacy.show_password")}
title={t("settings.privacy.show_password")}
>
{#if showVaultPw}<EyeOff size={14} aria-hidden="true" />{:else}<Eye
size={14}
aria-hidden="true"
/>{/if}
</button>
</div>
<button class="primary" onclick={unlockVault} disabled={!vaultPw}
>{t("settings.privacy.unlock")}</button
>
{:else}
<p class="muted">
{t("settings.privacy.vault_unlocked_1")}<strong
>{t("settings.privacy.unlocked_word")}</strong
>{t("settings.privacy.vault_unlocked_2")}
</p>
<button onclick={lockVault}>
<Lock size={14} aria-hidden="true" />
{t("settings.privacy.lock_now")}
</button>
<details>
<summary>{t("settings.privacy.change_password")}</summary>
<div class="grid">
<label class="wide"
>{t("settings.privacy.current_password")}<input
type="password"
bind:value={vaultPw}
/></label
>
<label class="wide"
>{t("settings.privacy.new_password")}<input
type="password"
bind:value={vaultPw2}
/></label
>
</div>
<button onclick={changeVaultPassword} disabled={!vaultPw || vaultPw2.length < 8}
>{t("settings.privacy.change_password")}</button
>
</details>
{/if}
{#if vaultMsg}<p class="vault-msg" class:error={vaultMsgError}>{vaultMsg}</p>{/if}
</div>
{/if} {/if}
</section> </section>
{:else if section === "language"} {:else if section === "language"}
@@ -2121,6 +2397,33 @@
font-size: 0.85rem; font-size: 0.85rem;
line-height: 1.5; line-height: 1.5;
} }
/* External-tool download chip — identical to the Import dialog's
ffmpeg/yt-dlp buttons (ImportMeeting.svelte). */
.tools {
display: flex;
align-items: center;
flex-wrap: wrap;
gap: 0.4rem;
margin-top: 0.9rem;
}
.tool {
display: inline-flex;
align-items: center;
gap: 0.25rem;
background: var(--bg-hover, transparent);
color: var(--accent);
border: 1px solid var(--border);
border-radius: var(--radius-sm);
padding: 0.2rem 0.5rem;
font-size: 0.78rem;
cursor: pointer;
}
.tool:hover {
border-color: var(--accent);
}
.tools .small {
font-size: 0.8rem;
}
.footnote { .footnote {
margin-top: 1rem; margin-top: 1rem;
border-top: 1px solid var(--border); border-top: 1px solid var(--border);
@@ -2429,6 +2732,115 @@
color: var(--accent, #2563eb); color: var(--accent, #2563eb);
border-color: currentColor; border-color: currentColor;
} }
.vault-card {
margin-top: 0.6rem;
padding: 0.85rem 1rem;
border: 1px solid var(--border);
border-radius: var(--radius-md, 8px);
background: var(--bg-elevated);
display: flex;
flex-direction: column;
gap: 0.6rem;
}
.vault-card.locked {
border-color: color-mix(in srgb, var(--accent) 45%, var(--border));
}
.vault-card.unlocked {
border-color: color-mix(in srgb, var(--success, #16a34a) 45%, var(--border));
}
.vault-head {
display: flex;
align-items: center;
gap: 0.5rem;
}
.vault-head h4 {
margin: 0;
}
.vault-head :global(svg) {
color: var(--muted);
}
.vault-card.locked .vault-head :global(svg) {
color: var(--accent);
}
.vault-card.unlocked .vault-head :global(svg) {
color: var(--success, #16a34a);
}
.vault-head .badge {
margin-left: auto;
}
.pw-row {
display: flex;
align-items: center;
gap: 0.4rem;
max-width: 22rem;
}
.pw-row input {
flex: 1;
}
.pw-toggle {
flex: none;
}
.vault-card .hint {
margin: 0;
}
.vault-msg {
margin: 0;
font-size: 0.85rem;
color: var(--success, #16a34a);
}
.vault-msg.error {
color: var(--danger);
}
.device-test {
display: flex;
align-items: center;
gap: 0.5rem;
flex-wrap: wrap;
margin: 0.35rem 0 0.6rem;
}
.device-test :global(.meter) {
flex: 1;
min-width: 8rem;
}
.stress-rec {
display: flex;
align-items: center;
gap: 0.5rem;
margin: 0.6rem 0;
padding: 0.55rem 0.75rem;
border: 1px solid color-mix(in srgb, var(--accent) 40%, var(--border));
border-radius: var(--radius-md, 8px);
background: color-mix(in srgb, var(--accent) 10%, var(--bg));
}
.stress-rec :global(svg) {
color: var(--accent);
}
.stress-rec span {
flex: 1;
}
.stress-table {
width: 100%;
border-collapse: collapse;
margin-top: 0.5rem;
font-size: 0.82rem;
}
.stress-table th,
.stress-table td {
text-align: left;
padding: 0.3rem 0.5rem;
border-bottom: 1px solid var(--border);
}
.stress-table th {
color: var(--muted);
font-weight: 600;
}
.stress-table .num {
font-variant-numeric: tabular-nums;
text-align: right;
}
.err {
color: var(--danger);
}
.npu-package { .npu-package {
margin-top: 0.6rem; margin-top: 0.6rem;
padding: 0.6rem 0.75rem; padding: 0.6rem 0.75rem;
+65 -44
View File
@@ -64,7 +64,7 @@
// nonce so re-clicking the same segment still jumps). Reading seekNonce is // nonce so re-clicking the same segment still jumps). Reading seekNonce is
// what makes this effect re-run. // what makes this effect re-run.
$effect(() => { $effect(() => {
player.seekNonce; void player.seekNonce;
const ms = player.seekMs; const ms = player.seekMs;
if (ms == null || !audioEl) return; if (ms == null || !audioEl) return;
audioEl.currentTime = ms / 1000; audioEl.currentTime = ms / 1000;
@@ -770,47 +770,54 @@
{#if editableItems.length === 0} {#if editableItems.length === 0}
<p class="muted">{t("summary.ai_empty")}</p> <p class="muted">{t("summary.ai_empty")}</p>
{:else} {:else}
<!-- Two-row card per item: the action text owns the full first row (it
was unreadable when six controls shared one row in this narrow
pane); owner/due/reminder are a secondary meta row beneath it. -->
<ul class="action-items"> <ul class="action-items">
{#each editableItems as item, i (i)} {#each editableItems as item, i (i)}
<li> <li>
<input <div class="ai-main">
type="checkbox" <input
bind:checked={item.confirmed} type="checkbox"
aria-label={t("summary.confirmed")} bind:checked={item.confirmed}
title={t("summary.confirmed")} aria-label={t("summary.confirmed")}
/> title={t("summary.confirmed")}
<input />
class="ai-text" <input
bind:value={item.text} class="ai-text"
placeholder={t("summary.ai_text_placeholder")} bind:value={item.text}
aria-label={t("summary.ai_text_aria")} placeholder={t("summary.ai_text_placeholder")}
/> aria-label={t("summary.ai_text_aria")}
<input />
class="ai-owner" <button
value={item.owner ?? ""} class="ai-del"
oninput={(e) => (item.owner = (e.target as HTMLInputElement).value || null)} onclick={() => removeActionItem(i)}
placeholder={t("summary.owner")} title={t("summary.ai_delete_title")}
aria-label={t("summary.owner")} aria-label={t("summary.ai_delete_aria")}
/> >
<input <X size={14} aria-hidden="true" />
type="date" </button>
class="due-date" </div>
aria-label={t("summary.due_date")} <div class="ai-meta">
value={dueDateInput(item.due_at)} <input
onchange={(e) => onDueDateChange(item, (e.target as HTMLInputElement).value)} class="ai-owner"
/> value={item.owner ?? ""}
<label class="remind" title={t("summary.reminder_title")}> oninput={(e) => (item.owner = (e.target as HTMLInputElement).value || null)}
<input type="checkbox" bind:checked={item.reminder_set} disabled={!item.due_at} /> placeholder={t("summary.owner")}
<Bell size={14} aria-hidden="true" /> aria-label={t("summary.owner")}
</label> />
<button <input
class="ai-del" type="date"
onclick={() => removeActionItem(i)} class="due-date"
title={t("summary.ai_delete_title")} aria-label={t("summary.due_date")}
aria-label={t("summary.ai_delete_aria")} value={dueDateInput(item.due_at)}
> onchange={(e) => onDueDateChange(item, (e.target as HTMLInputElement).value)}
<X size={14} aria-hidden="true" /> />
</button> <label class="remind" title={t("summary.reminder_title")}>
<input type="checkbox" bind:checked={item.reminder_set} disabled={!item.due_at} />
<Bell size={14} aria-hidden="true" />
</label>
</div>
</li> </li>
{/each} {/each}
</ul> </ul>
@@ -1080,9 +1087,9 @@
} }
ul.action-items li { ul.action-items li {
display: flex; display: flex;
align-items: center; flex-direction: column;
gap: 0.35rem; gap: 0.35rem;
padding: 0.25rem 0; padding: 0.45rem 0;
border-bottom: 1px solid var(--border); border-bottom: 1px solid var(--border);
} }
ul.action-items label { ul.action-items label {
@@ -1090,14 +1097,28 @@
align-items: center; align-items: center;
gap: 0.4rem; gap: 0.4rem;
} }
.ai-main {
display: flex;
align-items: center;
gap: 0.35rem;
}
.ai-text { .ai-text {
flex: 1; flex: 1;
min-width: 0; min-width: 0;
font-size: 0.82rem; font-size: 0.85rem;
}
/* Meta row indented under the text (past the confirm checkbox), wrapping
rather than crushing its inputs when the pane is narrow. */
.ai-meta {
display: flex;
align-items: center;
flex-wrap: wrap;
gap: 0.35rem;
padding-left: 1.4rem;
} }
.ai-owner { .ai-owner {
flex: none; flex: 1;
width: 5rem; min-width: 5rem;
font-size: 0.75rem; font-size: 0.75rem;
} }
.ai-del { .ai-del {
+207 -6
View File
@@ -6,19 +6,27 @@
import { meetings } from "../stores/meetings.svelte"; import { meetings } from "../stores/meetings.svelte";
import { settings } from "../stores/settings.svelte"; import { settings } from "../stores/settings.svelte";
import { player } from "../stores/player.svelte"; import { player } from "../stores/player.svelte";
import { api, type SpeakerInfo } from "../api"; import { api, errorMessage, type SpeakerInfo } from "../api";
import { t } from "../i18n/index.svelte"; import { t } from "../i18n/index.svelte";
import { renderMarkdown } from "../markdown"; import { renderMarkdown } from "../markdown";
import { save, open } from "@tauri-apps/plugin-dialog"; import { save, open } from "@tauri-apps/plugin-dialog";
import { layout, clamp } from "../stores/layout.svelte"; import { layout, clamp } from "../stores/layout.svelte";
import { imports } from "../stores/imports.svelte";
import Splitter from "../components/Splitter.svelte"; import Splitter from "../components/Splitter.svelte";
import ImportTracker from "../components/ImportTracker.svelte";
import { import {
Bold, Bold,
Italic, Italic,
Heading1, Heading1,
Heading2, Heading2,
Heading3,
List, List,
ListOrdered,
ListChecks, ListChecks,
Quote,
Minus,
Sparkles,
Undo2,
FileDown, FileDown,
FileText, FileText,
FolderOutput, FolderOutput,
@@ -66,6 +74,10 @@
let editorEl: HTMLTextAreaElement | undefined = $state(); let editorEl: HTMLTextAreaElement | undefined = $state();
let saveTimer: ReturnType<typeof setTimeout> | undefined; let saveTimer: ReturnType<typeof setTimeout> | undefined;
let loadedForId: string | null = null; let loadedForId: string | null = null;
// The server copy the buffer was last synced against — lets the effect below
// tell a server-side notes change (speaker rename, reprocess) apart from the
// user's own unsaved edits.
let lastServerNotes: string | null = null;
// Transcript/notes split (FR-UX-1): resizable (drag the Splitter) and // Transcript/notes split (FR-UX-1): resizable (drag the Splitter) and
// each side independently hideable, shared across the finalized-meeting // each side independently hideable, shared across the finalized-meeting
@@ -103,14 +115,23 @@
selectedSegmentMs = null; selectedSegmentMs = null;
}); });
// Sync the editor buffer whenever a different meeting is selected. // Sync the editor buffer whenever a different meeting is selected — and when
// the *server* copy of the same meeting's notes changes underneath us (a
// speaker rename rewrites notes.md's dialogue tags, reprocess regenerates it).
// A buffer with unsaved local edits is never clobbered: it only adopts the
// server copy when it still equals the last-synced one.
$effect(() => { $effect(() => {
const m = meetings.selected; const m = meetings.selected;
if (m && m.id !== loadedForId) { if (m && m.id !== loadedForId) {
notesText = m.notes_markdown; notesText = m.notes_markdown;
lastServerNotes = m.notes_markdown;
loadedForId = m.id; loadedForId = m.id;
} else if (m && m.id === loadedForId && m.notes_markdown !== lastServerNotes) {
if (notesText === lastServerNotes) notesText = m.notes_markdown;
lastServerNotes = m.notes_markdown;
} else if (!m) { } else if (!m) {
loadedForId = null; loadedForId = null;
lastServerNotes = null;
} }
}); });
@@ -158,6 +179,70 @@
scheduleSave(); scheduleSave();
} }
// Slash commands: typing "/todo" (etc.) at the start of a line and pressing
// Space/Enter swaps it for the matching Markdown prefix. Reuses the same
// line-prefix model as the toolbar buttons — no rich inline menu.
// ponytail: line-prefix slash only; add a picker popover if users ask.
const SLASH_COMMANDS: Record<string, string> = {
h1: "# ",
h2: "## ",
h3: "### ",
todo: "- [ ] ",
bullet: "- ",
num: "1. ",
quote: "> ",
divider: "---\n",
};
function handleNotesKeydown(e: KeyboardEvent) {
if (e.key !== "Enter" && e.key !== " ") return;
const el = editorEl;
if (!el) return;
const { selectionStart: s, value } = el;
const lineStart = value.lastIndexOf("\n", s - 1) + 1;
const match = /^\/(\w+)$/.exec(value.slice(lineStart, s));
if (!match) return;
const prefix = SLASH_COMMANDS[match[1].toLowerCase()];
if (prefix === undefined) return;
e.preventDefault();
const head = value.slice(0, lineStart) + prefix;
notesText = head + value.slice(s);
queueMicrotask(() => {
el.focus();
el.selectionStart = el.selectionEnd = head.length;
});
scheduleSave();
}
// AI-enhance (Granola-style): expand the user's rough notes into structured
// Markdown grounded in the transcript, via the configured LlmProvider (local
// by default, no new egress). Keeps a one-step Undo so we never silently lose
// what the user typed.
let enhancing = $state(false);
let enhanceError = $state<string | null>(null);
let notesBeforeEnhance = $state<string | null>(null);
async function enhanceNotes() {
const m = meetings.selected;
if (!m || enhancing) return;
enhancing = true;
enhanceError = null;
try {
const enhanced = await api.enhanceNotes(m.id, notesText);
notesBeforeEnhance = notesText;
notesText = enhanced;
scheduleSave();
} catch (e) {
enhanceError = errorMessage(e);
} finally {
enhancing = false;
}
}
function undoEnhance() {
if (notesBeforeEnhance === null) return;
notesText = notesBeforeEnhance;
notesBeforeEnhance = null;
scheduleSave();
}
async function exportMd() { async function exportMd() {
const m = meetings.selected; const m = meetings.selected;
if (!m) return; if (!m) return;
@@ -247,6 +332,15 @@
reprocessing = true; reprocessing = true;
try { try {
await meetings.reprocess(m.id, reprocessModel, reprocessLanguage || undefined); await meetings.reprocess(m.id, reprocessModel, reprocessLanguage || undefined);
// Re-transcribe rebuilds notes.md server-side (merging saved manual notes),
// but the meeting stays selected (same id), so the buffer-sync $effect —
// which only fires on an id change — won't pick it up. Resync explicitly so
// the notes pane updates in place instead of only after a restart.
const updated = meetings.selected;
if (updated && updated.id === m.id) {
notesText = updated.notes_markdown;
loadedForId = updated.id;
}
} finally { } finally {
reprocessing = false; reprocessing = false;
} }
@@ -262,14 +356,23 @@
onchange={onTitleChange} onchange={onTitleChange}
aria-label={t("transcript.title_aria")} aria-label={t("transcript.title_aria")}
/> />
{#if m.status === "transcribing" || imports.get(m.id)}
<div class="import-strip"><ImportTracker meetingId={m.id} /></div>
{:else if m.model_used}
<p class="engine-meta" title={t("transcript.transcribed_with")}>
{t("transcript.transcribed_with")}
<strong>{m.model_used}</strong>{#if m.backend_used}
· {m.backend_used}{/if}
</p>
{/if}
<div <div
class="split" class="split"
style="grid-template-columns: {splitColumns()};" style="grid-template-columns: {splitColumns()};"
bind:clientWidth={splitWidth} bind:clientWidth={splitWidth}
> >
<!-- svelte-ignore a11y_no_static_element_interactions -- wheel/touchmove <!-- wheel/touchmove here only note "the user scrolled by hand" to pause
here only note "the user scrolled by hand" to pause playback playback auto-scroll; the pane isn't an interactive control. -->
auto-scroll; the pane isn't an interactive control. --> <!-- svelte-ignore a11y_no_static_element_interactions -->
<div <div
class="pane transcript" class="pane transcript"
class:collapsed={layout.transcriptCollapsed} class:collapsed={layout.transcriptCollapsed}
@@ -423,6 +526,43 @@
> >
<ListChecks size={14} aria-hidden="true" /> <ListChecks size={14} aria-hidden="true" />
</button> </button>
<button
onclick={() => insertLinePrefix("### ")}
title={t("notes.h3")}
aria-label={t("notes.h3")}
>
<Heading3 size={14} aria-hidden="true" />
</button>
<button
onclick={() => insertLinePrefix("1. ")}
title={t("notes.numbered")}
aria-label={t("notes.numbered")}
>
<ListOrdered size={14} aria-hidden="true" />
</button>
<button
onclick={() => insertLinePrefix("> ")}
title={t("notes.quote")}
aria-label={t("notes.quote")}
>
<Quote size={14} aria-hidden="true" />
</button>
<button
onclick={() => insertLinePrefix("---\n")}
title={t("notes.divider")}
aria-label={t("notes.divider")}
>
<Minus size={14} aria-hidden="true" />
</button>
<button
class="enhance"
onclick={enhanceNotes}
disabled={enhancing}
title={t("notes.enhance_title")}
>
<Sparkles size={14} aria-hidden="true" class={enhancing ? "spin" : ""} />
{enhancing ? t("notes.enhancing") : t("notes.enhance")}
</button>
<button <button
class="toggle" class="toggle"
onclick={() => (notesPreview = !notesPreview)} onclick={() => (notesPreview = !notesPreview)}
@@ -459,6 +599,17 @@
Obsidian Obsidian
</button> </button>
</div> </div>
{#if enhanceError}
<p class="enhance-bar error" role="alert">{enhanceError}</p>
{:else if notesBeforeEnhance !== null}
<div class="enhance-bar">
<span>{t("notes.enhanced_note")}</span>
<button class="link" onclick={undoEnhance}>
<Undo2 size={13} aria-hidden="true" />
{t("notes.undo_enhance")}
</button>
</div>
{/if}
<div class="editor-preview"> <div class="editor-preview">
{#if notesPreview} {#if notesPreview}
<!-- eslint-disable-next-line svelte/no-at-html-tags -- sanitized via renderMarkdown() --> <!-- eslint-disable-next-line svelte/no-at-html-tags -- sanitized via renderMarkdown() -->
@@ -468,6 +619,7 @@
bind:this={editorEl} bind:this={editorEl}
bind:value={notesText} bind:value={notesText}
oninput={scheduleSave} oninput={scheduleSave}
onkeydown={handleNotesKeydown}
placeholder={t("notes.placeholder")} placeholder={t("notes.placeholder")}
></textarea> ></textarea>
{/if} {/if}
@@ -524,7 +676,7 @@
onclick={() => (selectedSegmentMs = open ? null : s.start_ms)} onclick={() => (selectedSegmentMs = open ? null : s.start_ms)}
> >
<span class="ts">{fmtTs(s.start_ms)}</span> <span class="ts">{fmtTs(s.start_ms)}</span>
<strong>{speakerName(s.speaker)}:</strong> <strong>{speakerName(s.speaker, recording.speakers)}:</strong>
{s.text} {s.text}
{#if hasNote} {#if hasNote}
<span class="note-badge" title={t("transcript.has_note")}>📝</span> <span class="note-badge" title={t("transcript.has_note")}>📝</span>
@@ -610,6 +762,21 @@
background: var(--border); background: var(--border);
outline: none; outline: none;
} }
.import-strip {
flex: none;
padding: 0 1rem 0.5rem;
}
.engine-meta {
flex: none;
margin: 0;
padding: 0 1rem 0.4rem;
font-size: 0.75rem;
color: var(--muted);
}
.engine-meta strong {
font-weight: 600;
color: var(--fg);
}
.pad { .pad {
padding: 1rem; padding: 1rem;
max-width: 760px; max-width: 760px;
@@ -842,6 +1009,40 @@
.toolbar .spacer { .toolbar .spacer {
flex: 1; flex: 1;
} }
.toolbar .enhance {
color: var(--accent);
border-color: color-mix(in srgb, var(--accent) 40%, var(--border));
font-weight: 600;
}
.toolbar .enhance:hover:not(:disabled) {
background: color-mix(in srgb, var(--accent) 12%, var(--bg));
}
.toolbar .enhance:disabled {
opacity: 0.6;
cursor: default;
}
.enhance-bar {
display: flex;
align-items: center;
gap: 0.5rem;
margin-bottom: 0.5rem;
font-size: 0.8rem;
color: var(--muted);
}
.enhance-bar.error {
color: var(--danger);
}
.enhance-bar .link {
display: inline-flex;
align-items: center;
gap: 0.25rem;
background: none;
border: none;
color: var(--accent);
cursor: pointer;
font-size: 0.8rem;
padding: 0;
}
.editor-preview { .editor-preview {
height: calc(100% - 2.5rem); height: calc(100% - 2.5rem);