66 Commits
Author SHA1 Message Date
iamdoubz 57cd46be46 Merge pull request 'Feature chore bug 006' (#20) from feature_chore_bug_006 into main
Reviewed-on: #20
2026-07-11 09:26:39 -05:00
iamdoubz 490a960676 Add optional dependencies section 2026-07-11 09:23:36 -05:00
iamdoubz 379bdf532f chore: sync Cargo.lock to version 0.4.0 2026-07-11 09:10:17 -05:00
iamdoubz 95f07e1dba fix(transcription): scope streamer-test closures so out isn't borrow-locked 2026-07-11 09:08:20 -05:00
iamdoubz 931a2b78b4 fix(media): blank doc line before the tool list (clippy doc_lazy_continuation) 2026-07-11 09:08:19 -05:00
iamdoubz 89254bc070 fix(audio): update audio_level test calls for the new mic arg 2026-07-11 09:08:18 -05:00
iamdoubz 22a37c5f4e docs: README for v0.4.0 — new features + Granola/Meetily/WA comparison (Granola MCP) 2026-07-11 08:58:06 -05:00
iamdoubz cecfe20eec chore: bump tauri config version to 0.4.0 2026-07-11 08:58:05 -05:00
iamdoubz 6fbf967fdb chore: bump crate version to 0.4.0 2026-07-11 08:58:04 -05:00
iamdoubz 9f3c1b65d0 chore: bump version to 0.4.0 2026-07-11 08:58:03 -05:00
iamdoubz a4b2b9a292 feat(transcription): drive live worker with StreamTuning; store only committed segments 2026-07-11 08:42:25 -05:00
iamdoubz 12e23143f6 feat(transcription): streaming growing-window + interim + commit-on-pause (FR-TRX-2) 2026-07-11 08:42:24 -05:00
iamdoubz fd556c57a1 feat(import): import_media — file/URL to a transcribed, diarized meeting; auto_record default 2026-07-11 00:38:19 -05:00
iamdoubz 1ea8011dc8 feat: register media module + import_media command 2026-07-11 00:38:18 -05:00
iamdoubz 002f8caa2b feat(media): transcode local files + URLs to 16kHz wav via ffmpeg/yt-dlp 2026-07-11 00:38:16 -05:00
iamdoubz d72509df5f feat: add auto_record_calendar setting (FR-CAL) 2026-07-11 00:38:15 -05:00
iamdoubz 7ef45bb315 feat(ui): Add meeting button + import modal wiring in the header 2026-07-11 00:37:53 -05:00
iamdoubz bd251b991a feat(ui): add-a-meeting modal for importing a file or URL 2026-07-11 00:37:51 -05:00
iamdoubz dfaeb7c873 feat(ui): import_media command in the typed api 2026-07-11 00:37:50 -05:00
iamdoubz 6b164cd02a feat(ui): auto-start-on-calendar-event toggle in Settings > Recording 2026-07-11 00:31:34 -05:00
iamdoubz 21101054a4 feat(ui): arm a one-shot auto-record timer for the next calendar event (FR-CAL) 2026-07-11 00:31:32 -05:00
iamdoubz 16200a130e feat(ui): let recording.start attach a calendar event id 2026-07-11 00:31:31 -05:00
iamdoubz 86f115642c feat(ui): default auto_record_calendar off in the settings store 2026-07-11 00:31:29 -05:00
iamdoubz 1760d4ae24 feat(ui): add auto_record_calendar to the settings contract 2026-07-11 00:31:16 -05:00
iamdoubz 2681a5d8c8 feat(audio): forward mic/system flag on the recording://level event 2026-07-11 00:27:42 -05:00
iamdoubz 233d641041 feat(audio): emit tagged mic + system level updates from capture (FR-CAP-5/7) 2026-07-11 00:27:41 -05:00
iamdoubz 29589663b2 feat(ui): pass mic level + enabled state to the level meter 2026-07-11 00:27:06 -05:00
iamdoubz 28058e205d feat(ui): overlay mic level on the meter in the accent colour (FR-CAP-5/7) 2026-07-11 00:27:05 -05:00
iamdoubz 235129d0d6 feat(ui): track microphone level separately in the recording store 2026-07-11 00:27:03 -05:00
iamdoubz 59c2d70643 feat(ui): tag recording level events with mic vs system source (FR-CAP-7) 2026-07-11 00:27:02 -05:00
iamdoubz e082f99f32 feat(ui): speaker diarization download cards in Settings > Hardware 2026-07-11 00:23:19 -05:00
iamdoubz 5260a4240b feat(ui): load/download/remove diarization models in the settings store 2026-07-11 00:23:18 -05:00
iamdoubz bf644ea0be feat(ui): expose diarization models over the typed api (FR-MODEL-1) 2026-07-11 00:23:17 -05:00
iamdoubz 13fb9984da feat(ui): model download cards + PST import range + calendar cleanup
Settings > Hardware: the Models list is now a bordered card row per
model with active/installed badges and a real progress bar for
downloads (was plain text links + a bare "downloading NN%" string),
using the app's existing lucide icon set.

Settings > Calendar: an Import range picker (30/90/180/365 days/all
time, persisted, applied to both a manual Import and the pst_auto_sync
startup re-import) and a Clean up section (delete all / older-than-N
threshold, confirm() gated, always reports how many events were kept
because they're linked to a meeting) -- fixes the 14,686-row bloat from
importing a mailbox's entire unbounded history.
2026-07-11 00:13:02 -05:00
iamdoubz 99d50583cf feat(ui): live notes pane, click-to-annotate transcript, resizable split
Granola-style redesign: the live-recording view gains a Notes pane
(bound to recording.notesText) alongside the transcript, and each
transcript line is clickable to attach/edit a note for that moment
(recording.segmentNotes, badge on annotated lines).

Also, for both the live and finalized-meeting layouts: the
transcript/notes split is now drag-resizable and each side
independently collapsible (Splitter + layout store), and the
post-finalize Notes pane defaults to Preview (rendered) instead of raw
Editor mode, since the merged document is now properly sectioned rather
than a single-speaker blob.
2026-07-11 00:13:01 -05:00
iamdoubz 0027becd10 style(ui): polish Upload now / Save action items / provider select
Upload now and Save action items now use the existing .primary
solid-accent button style (with icons) instead of unstyled native
buttons; the AI provider dropdown gets real border/background/hover
states matching the app's other selects.
2026-07-11 00:12:45 -05:00
iamdoubz e3c80b565c feat(ui): resizable/hideable meetings-list and summary panes
Left (meetings list) and right (summary) panes gain drag-to-resize via
Splitter and a collapse/expand toggle (thin rail + panel icon), sizes
and collapsed state persisted via the layout store.
2026-07-11 00:12:44 -05:00
iamdoubz 2a637e78ad feat(ui): default pst_import_range_days in the settings fallback 2026-07-11 00:12:29 -05:00
iamdoubz 80ad2a5f76 feat(ui): calendar store passes rangeDays through to importPst 2026-07-11 00:12:28 -05:00
iamdoubz 6622ae81ff feat(ui): recording store gains live notes state
notesText (debounced save, flushed on stop before finalize reads it),
segmentNotes Map keyed by anchor_ms, setNotesText/setSegmentNote.
2026-07-11 00:12:26 -05:00
iamdoubz 90b2d51d08 feat(ui): API bindings for live notes and calendar range/cleanup
updateLiveNotes/setSegmentNote; import_pst's rangeDays param;
cleanupCalendarEvents; pst_import_range_days on AppSettings.
2026-07-11 00:12:15 -05:00
iamdoubz 7081040475 feat(ui): layout store for resizable/hideable panes
Pane widths, collapsed flags, and the transcript/notes split ratio,
persisted to localStorage (pure client-side UI ephemera, not app data --
no Tauri settings round-trip needed).
2026-07-11 00:12:04 -05:00
iamdoubz 7dfcbdf247 feat(ui): draggable Splitter component for resizable panes
WAI-ARIA "window splitter" pattern (role=separator, keyboard-operable) --
reusable divider used by the resizable/hideable pane layout.
2026-07-11 00:12:01 -05:00
iamdoubz 864cd33a5f docs: document live notes commands and calendar range/cleanup commands
update_live_notes/set_segment_note; import_pst's new rangeDays param;
new cleanup_calendar_events command.
2026-07-11 00:11:53 -05:00
iamdoubz 94eafd4fae docs: document manual_notes.json (Granola-style live notes redesign) 2026-07-11 00:11:52 -05:00
iamdoubz 45ddc2bd3e feat(commands): live notes commands + calendar range/cleanup commands
Live notes (Granola-style redesign):
- update_live_notes/set_segment_note: live-session-only, mutate
  RecordingSession.manual_notes and write-through to manual_notes.json.
- stop_recording, reprocess_transcript, and resume_transcription now call
  MarkdownNotes::merge (loading manual_notes.json from disk for the
  latter two, which have no live RecordingSession) instead of the old
  transcript-only to_markdown, so re-transcription/crash-recovery never
  silently drops what the user typed.
- refresh_notes_and_notify no longer re-renders/overwrites notes.md on a
  post-finalize speaker rename/merge -- that used to clobber manual
  edits; it still updates the speakers table and emits the live-refresh
  event.

Calendar range/cleanup (14,686-row bloat fix):
- import_pst_core/import_pst take an optional range_days, converted to a
  from timestamp for PstSource.
- New cleanup_calendar_events command wrapping the new store method.
2026-07-11 00:11:45 -05:00
iamdoubz bb226dcb5e feat(core): wire up live notes and calendar range/cleanup commands
- RecordingSession.manual_notes; register update_live_notes/set_segment_note.
- Register cleanup_calendar_events; pass pst_import_range_days through to
  the pst_auto_sync startup re-import so it applies the same range as the
  last manual import, not the full mailbox history.
2026-07-11 00:11:32 -05:00
iamdoubz 98b183d2b0 feat(storage): cleanup_calendar_events, protects meeting-linked rows
CalendarCleanupResult + Store::cleanup_calendar_events(older_than_unix):
None deletes every unlinked event ("Delete all"), Some(cutoff) only those
starting before it. An event referenced by meetings.calendar_event_id is
always kept regardless of age or the "delete all" choice; participant
links cascade via the existing ON DELETE CASCADE FK.
2026-07-11 00:11:19 -05:00
iamdoubz 8828fca805 fix(calendar): honor from/to range on PST import
PstSource used to silently ignore CalImport.from/to (only Graph honored
a range) -- a long-lived mailbox has no natural upper bound on history,
so importing "everything" meant every recurring series expanded across
its full lifetime (up to 500 occurrences each) plus every one-off entry
the file ever held (e.g. a decade of Outlook's auto-generated yearly
holidays). Filters parsed events by starts_at after readpst parsing,
after collect_events -- doesn't need to touch how readpst is invoked or
how recurrence expansion works.
2026-07-11 00:11:02 -05:00
iamdoubz 9057b25e92 feat(notes): MarkdownNotes::merge folds manual notes into the transcript
Adds the Notes/Transcript-sectioned merge (freeform notes under ## Notes,
transcript dialogue under ## Transcript with per-moment annotations
placed as > note callouts right after the paragraph they anchor to,
unmatched anchors surfaced under "Other notes" rather than dropped) that
stop_recording now calls instead of the old transcript-only to_markdown.
2026-07-11 00:10:47 -05:00
iamdoubz 41b5553836 feat(core): add ManualNotes/SegmentNote types; pst_import_range_days setting
Two features:
- Granola-style live notes redesign: ManualNotes/SegmentNote are the IPC +
  on-disk shape of manual_notes.json (freeform notes typed during
  recording, plus per-moment annotations anchored by timestamp).
- Calendar bloat fix: Settings.pst_import_range_days persists how far back
  a PST import should go (None = full history), applied to both a manual
  import and the pst_auto_sync startup re-import.
2026-07-11 00:10:37 -05:00
iamdoubz 538db4372f feat(core): add manual_notes.json path helper
Granola-style live notes redesign: manual_notes_file(id) mirrors the
existing meeting_dir()-based helpers for the new per-meeting raw
user-notes artifact.
2026-07-11 00:10:24 -05:00
iamdoubz 8906f8c8ed chore(ui): disable the default WebView right-click menu
Back/Forward/Reload/Inspect doesn't belong in a native-feeling desktop
app. Disabled app-wide for now, per explicit instruction -- a
WhispAssist-specific context menu is a deferred follow-up, not decided
against.
2026-07-10 22:08:25 -05:00
iamdoubz 2f62e4b13a fix(calendar): run readpst headless instead of popping a console window
readpst.exe is a console-subsystem binary; WhispAssist has no console of
its own, so Windows created a brand new visible console window for it on
every PST import. CREATE_NO_WINDOW spawns it fully headless -- its
stdout/stderr are still captured normally via .output().
2026-07-10 22:08:19 -05:00
iamdoubz 3214bd2740 fix(calendar): stable content-based raw_uid for UID-less events; UTC dedup keys
parse_vevents used to leave raw_uid: None for any VEVENT with no UID
line, which the storage layer's unique index exempts from dedup -- so
that event duplicated on every re-import forever. Falls back to
content_uid (subject|organizer|starts_at|ends_at) so re-parsing the same
source resolves to the same key. Also fixes ymd_digits, the per-occurrence
dedup-key suffix for expanded recurring events, to format in UTC instead
of the machine's local timezone -- it could otherwise compute a different
date (and therefore a different key) for the same occurrence across a DST
transition or timezone change between imports.
2026-07-10 19:22:18 -05:00
iamdoubz e7b951cb90 fix(storage): collapse existing duplicate calendar_events rows
One-time cleanup for anyone already hit by the reimport-duplication bug:
groups existing rows by (source, raw_uid) -- backfilling a content-based
key first for rows that had none -- keeps the row a meeting is attached
to (or the first survivor otherwise), repoints any meeting pointing at a
row about to be removed, then deletes the rest. The unique index is
dropped and recreated around this since the backfill step can
momentarily produce rows that collide before they're deduped.
2026-07-10 19:22:10 -05:00
iamdoubz a341afc52f fix(commands): reindex search after generate_summary writes summary.json
summary.json is sealed straight to disk, not through a store write, so
the FTS index needs an explicit nudge to pick up newly generated
summaries (FR-SEARCH-1).
2026-07-10 19:22:04 -05:00
iamdoubz 83babc99cd fix(storage): reindex FTS on tags/summary; default a stable calendar raw_uid
Two bugs, one file:

- Search: reindex_fts now also indexes summary_text and tags_text, and
  runs on every set_tags call (it previously only fired from
  finalize_meeting/update_notes, so tag and summary changes never touched
  the index at all). Promoted from a private helper to a Store trait
  method so commands.rs can call it after summary.json is written
  (summary lives only on disk, not through a store write).

- Calendar dedup: import_calendar_events used to insert whatever raw_uid
  the caller passed, including None -- which the (source, raw_uid)
  unique index explicitly exempts from dedup, so any event without a
  captured UID duplicated on every re-import forever (the reported
  1762 -> 11,764 row bug). Defaults to a deterministic content-derived
  key (calendar::content_uid) when raw_uid is missing, and adds a
  regression test that re-importing the same batch repeatedly doesn't
  grow the table.
2026-07-10 19:21:59 -05:00
iamdoubz 4015b5132e feat(storage): add summary/tags columns to the FTS5 search index
meeting_fts only ever indexed title/transcript/notes -- search silently
never covered summary or tags despite FR-SEARCH-1 requiring it (bug
report: "search not working" for those). meeting_fts is a derived index
(never a source of truth), so drop+recreate is safe: backfill_fts
repopulates every meeting from scratch on next startup.
2026-07-10 19:21:48 -05:00
iamdoubz b5f5ef172c feat(commands): auto-label the mic speaker "You" at stop_recording
Wires VoiceSample creation into start_recording (mic-enabled path) and
runs the voiceprint match after the final diarization pass in
stop_recording: on a confident match, persists "You" and "Speaker 2",
"Speaker 3"... via the existing rename_speaker store path, skipping any
label the user already renamed live.
2026-07-10 19:05:28 -05:00
iamdoubz 69eedc9354 feat(core): track the mic voiceprint sample on RecordingSession
mic_voice_sample carries the VoiceSample handle from start_recording
through to stop_recording so the final diarization pass can run the
mic-speaker voiceprint match.
2026-07-10 19:05:23 -05:00
iamdoubz c9b5b91473 chore(diarization): register the new voiceprint module 2026-07-10 19:05:18 -05:00
iamdoubz dca797aaa3 feat(diarization): voiceprint match to identify the mic speaker cluster
Mic and system audio are already summed into one mono stream before
diarization runs, so clustering alone can't tell which cluster is the
user's own voice. match_mic_speaker compares a mic-only sample's speaker
embedding (same sherpa-onnx model diarization already uses) against each
diarized cluster's own audio and returns a label->name map: best match ->
"You", the rest -> "Speaker 2", "Speaker 3", etc. Returns empty (no
guessing) when the mic sample is too short or no cluster clears the
similarity threshold.
2026-07-10 19:05:14 -05:00
iamdoubz b532acbb3c feat(audio): capture a bounded mic-only sample for voiceprint matching
VoiceSample collects the first ~8s of raw 16kHz-mono mic audio during a
recording, then stops accepting once full. Feeds the mic-speaker
voiceprint match in diarization::voiceprint (bug: mic speaker mislabeled
S1/S2 instead of "You").
2026-07-10 19:05:07 -05:00
iamdoubz 469df7524f fix(transcription): drop whisper-hallucinated segments on silent windows 2026-07-10 18:51:08 -05:00
iamdoubz ec864c24f0 fix(transcription): medium quantized model only exists as q5_0 upstream 2026-07-10 18:51:07 -05:00
35 changed files with 4003 additions and 433 deletions
+60 -29
View File
@@ -7,13 +7,14 @@ on-device acceleration (**NPU → GPU → CPU**), labels speakers, structures th
Markdown notes, and optionally augments them with a locally hosted LLM (Ollama). Audio and
transcripts **never leave the machine** unless you explicitly configure a destination.
> **Status: working application (v0.2.0).** Capture (system audio **+ your microphone**),
> transcription (CPU / Intel NPU / Vulkan GPU), speaker diarization, storage + crash recovery,
> local-LLM summaries, AI tags, opt-in recording with **in-app playback**, at-rest encryption,
> and self-hosted sync are implemented and ship as a single signed **MSI + NSIS** universal
> installer. Outlook `.pst`/calendar context (recurring-event import + filtering) is landing;
> the coding-agent (MCP) handoff is in progress. Build order and remaining tasks are in
> [`docs/05-roadmap.md`](docs/05-roadmap.md).
> **Status: working application (v0.4.0).** Capture (system audio **+ your microphone**, with a
> live **dual level meter**), transcription (CPU / Intel NPU / Vulkan GPU) with a **fluid live
> transcript**, speaker diarization, storage + crash recovery, local-LLM summaries, AI tags,
> opt-in recording with **in-app playback**, at-rest encryption, self-hosted sync, **Outlook
> `.pst`/calendar import with optional auto-record**, **importing an existing recording from a
> file or URL**, and a loopback **MCP server** for coding-agent handoff are all implemented and
> ship as a single signed **MSI + NSIS** universal installer. Build order and remaining tasks
> are in [`docs/05-roadmap.md`](docs/05-roadmap.md).
## Why WhispAssist — Granola vs Meetily vs WhispAssist
@@ -27,17 +28,21 @@ transcripts **never leave the machine** unless you explicitly configure a destin
| **Speaker diarization** | ✅ cloud | ⚠️ limited | ✅ offline (sherpa-onnx) |
| **Summaries / notes AI** | ☁️ cloud LLM | ✅ local (Ollama) / BYO | ✅ local (Ollama, localhost **or LAN**) + optional hosted |
| **Default data egress** | ☁️ audio + notes to cloud | 🔒 local (cloud optional) | 🔒 **none** — everything off by default, allowlist-enforced |
| **Calendar / Outlook `.pst` context** | ✅ cloud calendar | ❌ | ⚙️ local `.pst` — in progress |
| **Calendar / Outlook `.pst` context** | ✅ cloud calendar | ❌ | ✅ local `.pst` import + **auto-record on events** |
| **Import an existing recording (file/URL)** | ❌ live capture only | ❌ | ✅ any file or URL (ffmpeg + yt-dlp) |
| **Opt-in recording + consent notice** | ⚠️ partial | ❌ | ✅ off by default, one-time consent |
| **At-rest encryption** | ☁️ server-side | ❌ | ✅ vault: Argon2id + XChaCha20-Poly1305 |
| **Self-hosted sync w/ client-side encryption** | ❌ | ⚠️ | ✅ WebDAV + OAuth, **encrypt-before-upload** |
| **Coding-agent (MCP) handoff** | ❌ | ❌ | ⚙️ local MCP server — in progress |
| **Coding-agent (MCP) server** | ✅ **cloud** MCP (notes via their servers) | ❌ | ✅ **local, loopback-only**, token-gated, adds no egress |
| **License** | Proprietary | Open source (MIT) | Open source (MIT / Apache-2.0) |
| **Cost** | Subscription | Free | Free |
<sub>Comparison reflects each project's public positioning as of mid-2026. Granola and Meetily
are independent products and their capabilities evolve — verify current details before relying
on any row.</sub>
<sub>Comparison reflects each project's public positioning as of mid-2026. Granola now ships an
MCP server too, but it is **cloud-hosted** — an agent reaching it pulls your notes through
Granola's servers; WhispAssist's MCP server is **loopback-only and adds no egress of its own**
(data leaves only via the connected agent's own provider, which WA discloses). Granola and
Meetily are independent products and their capabilities evolve — verify current details before
relying on any row.</sub>
**The short version:** Granola is the polished cloud option (your audio and notes are processed
on their servers). Meetily is the closest peer — open-source and self-hosted — but is
@@ -46,19 +51,29 @@ accelerated, zero-egress-by-default** option: it exploits the NPU/GPU in modern
everything on the device unless you opt in, and adds Windows-specific context (Outlook) and a
coding-agent handoff.
## What's built (v0.2.0)
## What's built (v0.4.0)
- **Bot-free capture — now both sides** — WASAPI loopback records the system mix (all
participants), and an optional **microphone** path captures your own voice, mixed into both the
live transcript and the saved recording. Pick a specific output/input or turn the mic off in
**Settings ▸ Hardware**. No meeting bot, no per-app plumbing.
- **Local transcription with a hardware ladder** — whisper.cpp via `whisper-rs` on CPU; the
**Intel NPU** via ONNX Runtime + OpenVINO; **GPU via Vulkan** (a single binary that runs on
NVIDIA, AMD, and Intel). WA detects the hardware, picks the best backend
(**NPU → NVIDIA → AMD → Intel → CPU**), streams partial transcripts live, and shows the active
backend in the UI.
- **Speaker diarization** — `sherpa-onnx` (pyannote segmentation + speaker-embedding
clustering), fully offline.
- **Bot-free capture — now both sides, with a live dual meter** — WASAPI loopback records the
system mix (all participants), and an optional **microphone** path captures your own voice,
mixed into both the live transcript and the saved recording. While recording, a **level meter
overlays the system and microphone signals in two colours** so you can see both sides are being
picked up. Pick a specific output/input or turn the mic off in **Settings ▸ Hardware**. No
meeting bot, no per-app plumbing.
- **Local transcription with a hardware ladder — and a fluid live transcript** — whisper.cpp via
`whisper-rs` on CPU; the **Intel NPU** via ONNX Runtime + OpenVINO; **GPU via Vulkan** (a single
binary that runs on NVIDIA, AMD, and Intel). WA detects the hardware, picks the best backend
(**NPU → NVIDIA → AMD → Intel → CPU**), and shows it in the UI. The **live transcript streams a
growing line that refreshes ~once a second and commits at natural pauses** — words appear as
they're spoken instead of in fixed multi-second blocks, so sentences aren't chopped across
lines.
- **Speaker diarization** — `sherpa-onnx` (pyannote segmentation + speaker-embedding clustering),
fully offline. Install the two diarization models in **Settings ▸ Hardware** and finished
recordings are split by speaker (Speaker 1, Speaker 2, …); the microphone speaker is
auto-labelled from a short voiceprint.
- **Import an existing recording** — add a meeting from a **local audio/video file or a URL**
(YouTube, a streaming page, or a direct media link). WhispAssist transcribes and diarizes it
just like a live recording. Uses **`ffmpeg`** (and **`yt-dlp`** for URLs), which you install
yourself — neither is bundled.
- **Notes & summaries** — Markdown notes; local-LLM summaries via **Ollama** on `localhost`
**or a private LAN endpoint** (RFC-1918), with a full advanced-parameter panel (system prompt,
`think`, `keep_alive`, `num_ctx`, sampling/repetition/mirostat, etc.).
@@ -80,12 +95,20 @@ coding-agent handoff.
destination holds only ciphertext. Credentials live only in the OS credential store.
- **Optional hosted AI** — Anthropic and OpenAI-compatible providers behind the same
`LlmProvider` interface, off by default (third-party egress, keys in the OS credential store).
- **Outlook `.pst` / calendar context, with auto-record** — import events and attendees from a
local Outlook `.pst` backup (read-only, range-limited, de-duplicated, with cleanup), attach
meetings to events, and — opt-in — **auto-start recording when a calendar event begins** while
the app is open (a one-shot timer, no background polling).
- **Local MCP server for coding-agent handoff** — hand meeting context to your own coding agents
(Claude, Codex, Copilot, OpenCode) over a **loopback-only, token-gated** MCP server that is off
by default, scope-limited, audited, and **adds no egress** — data leaves only via the agent's
own provider, which WA discloses.
- **One universal installer** — a single signed MSI and NSIS `-setup.exe` that covers every
machine: Vulkan for all GPUs, the Intel NPU path, and CPU fallback. The Vulkan loader is bundled
so it launches even on machines without a GPU driver.
machine: Vulkan for all GPUs, the Intel NPU path (+ DirectML fallback), and CPU. The Vulkan
loader is bundled so it launches even on machines without a GPU driver.
**In progress:** Outlook `.pst` + calendar context, the local **MCP server** that hands meeting
context to your coding agents (Claude, Codex, Copilot, OpenCode), and MS Graph calendar.
**In progress:** Microsoft Graph calendar (cloud calendar via OAuth) and an optional CUDA
(NVIDIA-only) build variant.
## Quick start (install)
@@ -107,6 +130,14 @@ data under `%LOCALAPPDATA%\WhispAssist`.
Prefer the NSIS installer? Grab **`WhispAssist_<version>_x64-setup.exe`** from the same page.
## Optional dependencies
If you do not have these installed, WhispAssist will still work, but some features will be unavailable.
- [yt-dlp](https://github.com/yt-dlp/yt-dlp/releases)
- [ffmpeg](https://www.ffmpeg.org/download.html#build-windows)
- [libpst](https://sourceforge.net/projects/ezwinports/files/libpst-0.6.63-w32-bin.zip/)
## Technology
WhispAssist is a **Tauri 2** application: a small Rust core with a compiled **Svelte + TypeScript**
@@ -121,7 +152,7 @@ and its alternatives are recorded in [`docs/adr/`](docs/adr/) (ADR-0001–0011).
- **Storage:** SQLite + on-disk audio/transcript files
- **Local LLM:** Ollama HTTP API (localhost or a private LAN endpoint)
- **Encryption:** Argon2id + XChaCha20-Poly1305 envelope vault; secrets in the OS credential store
- **Calendar / Outlook:** `outlook-pst` for `.pst`, OS notifications for reminders *(in progress)*
- **Calendar / Outlook:** `readpst` for `.pst` import, OS scheduled toasts for action-item reminders
- **Sync:** WebDAV primary set + OneDrive/Dropbox/Box via OAuth — off by default (ADR-0010)
- **External AI / agents:** hosted providers behind `LlmProvider`; a loopback-only **MCP server**
for coding-agent handoff — off by default (ADR-0011)
+38 -4
View File
@@ -16,7 +16,8 @@ Default root: `%LOCALAPPDATA%\WhispAssist\` (user-configurable, FR-STORE-2).
└── <meeting_id>\ # one folder per meeting (uuid)
├── audio.wav # canonical recording — present ONLY if "Record" was on (ADR-0009)
├── transcript.json # canonical transcript (segments+speakers+timings)
├── notes.md # user-editable Markdown notes
├── manual_notes.json # raw user-authored notes captured live during recording
├── notes.md # the final notes document: manual notes + transcript, merged at finalize
├── summary.json # LLM summary, decisions, action items (if generated)
└── briefs/ # feature briefs distilled from this meeting (ADR-0011), if any
└── <brief_id>.json # agent-ready spec served via the MCP `get_feature_brief` tool
@@ -24,9 +25,42 @@ Default root: `%LOCALAPPDATA%\WhispAssist\` (user-configurable, FR-STORE-2).
Rule: while a meeting is in progress a working WAV is the source of truth for crash recovery. On
finalize, it is **kept** as `audio.wav` if "Record this meeting" was on, or **deleted** if not
(FR-REC-1/4) — deletion happens only after `transcript.json` is finalized. `transcript.json`,
`notes.md`, and `summary.json` are **derived** and regenerable (regenerable only while the audio
still exists — i.e. for recorded meetings).
(FR-REC-1/4) — deletion happens only after `transcript.json` is finalized. `transcript.json` and
`summary.json` are **derived** and regenerable (regenerable only while the audio still exists —
i.e. for recorded meetings).
`notes.md` is **generated once, at finalize**, by merging `manual_notes.json` (freeform notes
typed live during the recording, plus any per-moment annotations — see below) with the rendered,
speaker-tagged transcript (`notes::MarkdownNotes::merge`). After that it is the user's own
document, freely editable via `update_notes` exactly like before this changed — nothing
re-renders or overwrites it afterward. In particular, renaming or merging a speaker after finalize
updates the `speakers` table and the live UI display, but does **not** retroactively rewrite text
already baked into `notes.md` (same as any other manual edit isn't retroactively touched either —
this was a pre-existing clobber bug this redesign also fixes: renaming a speaker used to silently
overwrite the whole file). A crash-recovery finalize (T2.8) and a post-finalize batch
re-transcription (T3.8) both re-render `notes.md` from scratch and so both re-read
`manual_notes.json` from disk to fold the same manual notes back in.
### `manual_notes.json`
```jsonc
{
"schema": 1,
"freeform_md": "string — the user's running notes, typed live in the Notes pane while recording",
"segment_notes": [
{ "anchor_ms": 12345, "text": "string", "created_at": 1735000000, "updated_at": 1735000010 }
]
}
```
`segment_notes[].anchor_ms` is a timestamp into the recording (a clicked transcript segment's
`start_ms`), not a segment id — a later batch re-transcription can renumber/regenerate segment
ids, but never moves the moment in time a note was attached to. At merge time, each note is placed
right after whichever transcript paragraph's time span contains its `anchor_ms`; a note whose
anchor doesn't land inside any paragraph surfaces under an "Other notes" section instead of being
silently dropped. Written to disk on every edit via `update_live_notes`/`set_segment_note`
(`04-api-contracts.md`) — live-session only, same write-through-for-crash-safety spirit as
`transcript.json` accumulating during recording.
## SQLite schema (`wa.db`)
+23 -1
View File
@@ -26,6 +26,15 @@ resume_recording(input: { meetingId: MeetingId }): void
set_recording_retention(input: { meetingId: MeetingId; record: boolean }): void // toggle mid-meeting (FR-REC-1)
acknowledge_recording_consent(): void // one-time (FR-REC-2)
// ---- Live notes (Granola-style redesign, `03-data-model.md`'s manual_notes.json) ----
// Both are live-session only (err "no matching active recording" once finalized — post-finalize,
// notes.md is the single editable document and `update_notes` is the command for it).
update_live_notes(input: { meetingId: MeetingId; markdown: string }): void // freeform notes typed while recording
// anchorMs: the clicked transcript segment's start_ms, not its id (survives re-transcription).
// text: "" clears that moment's note. Merged into notes.md right after the transcript paragraph
// covering anchorMs when the meeting finalizes (notes::MarkdownNotes::merge).
set_segment_note(input: { meetingId: MeetingId; anchorMs: number; text: string }): void
// ---- Hardware ----
hardware_status(): { backends: BackendInfo[]; active: BackendId; modelSize: string; estRtf: number }
set_preferred_backend(input: { backend: BackendId | "auto" }): void
@@ -83,9 +92,19 @@ llm_setup_suggestions(): { ollamaInstalled: boolean; installUrl: string; suggest
pull_ollama_model(input: { model: string }): void // guided download via Ollama's own /api/pull; emits model://progress (T5.7)
// ---- Calendar / .pst ----
import_pst(input: { path: string; password?: string }): number // eventsImported; emits pst://progress (FR-CAL-1)
// rangeDays: only import events starting within the last N days; omitted imports the full mailbox
// history. Bug fix: a long-lived .pst has no natural upper bound on history (every recurring
// series expands to its cap, T4.1's RECURRENCE_MAX_OCCURRENCES, plus every one-off entry the file
// ever held, e.g. a decade of Outlook's auto-generated yearly holidays) -- unbounded import could
// produce tens of thousands of rows. Settings.pst_import_range_days persists the last choice and
// applies it to pst_auto_sync's startup re-import too.
import_pst(input: { path: string; password?: string; rangeDays?: number }): number // eventsImported; emits pst://progress (FR-CAL-1)
list_calendar_events(input: { from?: number; to?: number }): CalendarEvent[]
get_calendar_event(input: { eventId: string }): { event: CalendarEvent; participants: Participant[] } // pre-meeting panel + naming dropdown (FR-CAL-3, FR-SPK-4)
// olderThanDays omitted deletes every unlinked event ("Delete all"); Some(n) only those starting
// more than n days ago. An event attached to a recorded meeting (meetings.calendar_event_id) is
// always kept regardless of the choice -- protected reports how many were skipped for that reason.
cleanup_calendar_events(input: { olderThanDays?: number }): { deleted: number; protected: number }
attach_meeting_to_event(input: { meetingId: MeetingId; eventId: string }): void
// Optional MS Graph calendar source (M4.4, T8.9, FR-CAL-6): opt-in, explicit consent (OAuth PKCE
// + Microsoft's own consent screen), metadata-only (subject/organizer/start/end/attendees, never
@@ -240,6 +259,9 @@ pub trait NotesRenderer: Send + Sync {
fn to_markdown(&self, t: &Transcript, speakers: &[SpeakerInfo], s: Option<&Summary>) -> String;
fn export(&self, md: &str, dest: &Path, fmt: ExportFormat) -> Result<PathBuf, NotesError>;
}
// MarkdownNotes::merge(segments, speakers, manual: &ManualNotes, summary, template) -> String is an
// inherent method (not part of the trait — only one renderer needs it): what `stop_recording` calls
// instead of `to_markdown` to fold manual_notes.json into the generated notes.md (see 03-data-model.md).
// sync/mod.rs
// One impl per provider; `WebDavTarget` covers Nextcloud/ownCloud/Cloudreve/Seafile/Synology.
+1 -1
View File
@@ -1,7 +1,7 @@
{
"name": "whispassist",
"private": true,
"version": "0.3.0",
"version": "0.4.0",
"type": "module",
"description": "Privacy-first, fully local Windows meeting assistant.",
"license": "MIT OR Apache-2.0",
+1 -1
View File
@@ -6043,7 +6043,7 @@ dependencies = [
[[package]]
name = "whispassist"
version = "0.3.0"
version = "0.4.0"
dependencies = [
"argon2",
"async-trait",
+1 -1
View File
@@ -1,6 +1,6 @@
[package]
name = "whispassist"
version = "0.3.0"
version = "0.4.0"
description = "Privacy-first, fully local Windows meeting assistant"
authors = ["WhispAssist contributors"]
license = "MIT OR Apache-2.0"
@@ -0,0 +1,17 @@
-- Bug fix: search (FR-SEARCH-1) never covered summary or tags -- meeting_fts
-- only ever had title/transcript_text/notes_text columns. `meeting_fts` is a
-- derived index (rebuilt from meetings/transcript.json/notes.md/summary.json/
-- tags, never a source of truth), so dropping and recreating it is safe: no
-- data loss, and `backfill_fts` repopulates every meeting on next startup
-- since a freshly created table has no rows for anything yet.
DROP TABLE meeting_fts;
CREATE VIRTUAL TABLE meeting_fts USING fts5(
meeting_id UNINDEXED,
title,
transcript_text,
notes_text,
summary_text,
tags_text,
tokenize = 'porter unicode61'
);
@@ -0,0 +1,68 @@
-- Bug fix: prior imports could accumulate many duplicate rows for the same
-- real calendar event on every re-import -- either because the (source,
-- raw_uid) unique index (added in 0005) never retroactively deduped rows
-- that existed before it, or because an event with no UID in its source data
-- got `raw_uid = NULL`, which that index's `WHERE raw_uid IS NOT NULL` clause
-- explicitly exempts from uniqueness, so it duplicated on every single
-- re-import forever. This collapses whatever's already in the table, then
-- backfills a stable content-based key for anything still missing a UID so
-- future re-imports resolve to the same row instead of minting a new one
-- (see calendar::content_uid in src/calendar/mod.rs, which produces the
-- identical 'content:subject|organizer|starts_at|ends_at' format used here).
PRAGMA foreign_keys = ON;
-- Drop the index first: the backfill below can momentarily produce rows that
-- share a (source, raw_uid) pair before they're deduped a few statements
-- later, which the index would reject mid-UPDATE.
DROP INDEX IF EXISTS idx_calendar_events_source_uid;
UPDATE calendar_events
SET raw_uid = 'content:' || COALESCE(subject, '') || '|' || COALESCE(organizer, '')
|| '|' || COALESCE(starts_at, 0) || '|' || COALESCE(ends_at, 0)
WHERE raw_uid IS NULL;
-- One survivor per (source, raw_uid) group: whichever row a meeting is
-- already attached to (so `attach_meeting_to_event` links don't break), else
-- the lexicographically-first id (arbitrary but deterministic).
CREATE TEMP TABLE calendar_event_survivors AS
SELECT source, raw_uid, MIN(id) AS keep_id
FROM calendar_events
GROUP BY source, raw_uid;
UPDATE calendar_event_survivors
SET keep_id = (
SELECT m.calendar_event_id FROM meetings m
JOIN calendar_events ce ON ce.id = m.calendar_event_id
WHERE ce.source = calendar_event_survivors.source
AND ce.raw_uid = calendar_event_survivors.raw_uid
LIMIT 1
)
WHERE EXISTS (
SELECT 1 FROM meetings m
JOIN calendar_events ce ON ce.id = m.calendar_event_id
WHERE ce.source = calendar_event_survivors.source
AND ce.raw_uid = calendar_event_survivors.raw_uid
);
-- Repoint any meeting attached to a duplicate that's about to be deleted
-- onto the group's survivor instead.
UPDATE meetings
SET calendar_event_id = (
SELECT s.keep_id FROM calendar_event_survivors s
JOIN calendar_events ce ON ce.source = s.source AND ce.raw_uid = s.raw_uid
WHERE ce.id = meetings.calendar_event_id
)
WHERE calendar_event_id IN (
SELECT ce.id FROM calendar_events ce
JOIN calendar_event_survivors s ON ce.source = s.source AND ce.raw_uid = s.raw_uid
WHERE ce.id != s.keep_id
);
-- Drop the duplicates (cascades to calendar_event_participants).
DELETE FROM calendar_events
WHERE id NOT IN (SELECT keep_id FROM calendar_event_survivors);
DROP TABLE calendar_event_survivors;
CREATE UNIQUE INDEX idx_calendar_events_source_uid ON calendar_events(source, raw_uid)
WHERE raw_uid IS NOT NULL;
+108 -25
View File
@@ -58,6 +58,9 @@ pub type FrameSink = SyncSender<Vec<f32>>;
pub struct AudioLevel {
pub rms: f32,
pub peak: f32,
/// `true` for the microphone stream, `false` for system/loopback audio —
/// lets the UI overlay the two meters in different colours (FR-CAP-5/7).
pub mic: bool,
}
/// Out-of-band capture notices, lower-volume than `FrameSink`/level updates.
@@ -200,10 +203,48 @@ impl MicBridge {
}
}
/// A one-shot, bounded capture of raw mic-only audio (16kHz mono, same format
/// the transcriber and diarizer both use) taken early in a recording — enough
/// to compute a voiceprint that identifies which diarized speaker cluster is
/// the mic (so it can be labeled "You" instead of a clustered "S1"/"S2"; see
/// `diarization::voiceprint`). Unlike `MicBridge`, this is filled once and
/// never drained: the first `cap` samples are kept and everything after is
/// dropped, since a voiceprint only needs a few seconds of real speech, not
/// the whole meeting.
pub struct VoiceSample {
cap: usize,
buf: Mutex<Vec<f32>>,
}
impl VoiceSample {
pub fn new(cap_samples: usize) -> Arc<Self> {
Arc::new(Self {
cap: cap_samples,
buf: Mutex::new(Vec::new()),
})
}
fn push(&self, samples: &[f32]) {
if let Ok(mut buf) = self.buf.lock() {
if buf.len() < self.cap {
buf.extend_from_slice(samples);
buf.truncate(self.cap);
}
}
}
/// The samples captured so far (may be shorter than `cap` on a short
/// recording, or empty if the mic never produced usable audio).
pub fn samples(&self) -> Vec<f32> {
self.buf.lock().map(|b| b.clone()).unwrap_or_default()
}
}
/// Number of audio frames in a raw WASAPI byte buffer of the given format.
#[cfg(feature = "audio")]
fn frame_count(bytes: &[u8], format: &WaveFormat) -> usize {
let bytes_per_frame = (format.get_bitspersample() as usize / 8) * format.get_nchannels() as usize;
let bytes_per_frame =
(format.get_bitspersample() as usize / 8) * format.get_nchannels() as usize;
if bytes_per_frame == 0 {
0
} else {
@@ -231,6 +272,8 @@ impl WasapiCapture {
frame_sink: FrameSink,
event_sink: EventSink,
bridge: Option<Arc<MicBridge>>,
voice_sample: Option<Arc<VoiceSample>>,
emit_level: bool,
) -> Result<CaptureHandle, AudioError> {
let running = Arc::new(AtomicBool::new(true));
let paused = Arc::new(AtomicBool::new(false));
@@ -251,6 +294,8 @@ impl WasapiCapture {
&running_th,
&paused_th,
bridge.as_ref(),
voice_sample.as_ref(),
emit_level,
)
})
.map_err(|e| AudioError::Capture(format!("spawn failed: {e}")))?;
@@ -281,17 +326,22 @@ impl WasapiCapture {
frame_sink,
event_sink,
Some(bridge),
None,
true,
)
}
/// Microphone capture that both feeds the live-transcript mixer (`frame_sink`)
/// and pushes its audio into `bridge` for the loopback thread to record.
/// `voice_sample`, when given, also collects the first few seconds of raw
/// mic audio for a post-recording voiceprint match (mic-speaker labeling).
pub fn start_microphone_recording(
&self,
device_id: Option<&str>,
frame_sink: FrameSink,
event_sink: EventSink,
bridge: Arc<MicBridge>,
voice_sample: Option<Arc<VoiceSample>>,
) -> Result<CaptureHandle, AudioError> {
self.start_capture(
"wa-mic-capture",
@@ -301,6 +351,8 @@ impl WasapiCapture {
frame_sink,
event_sink,
Some(bridge),
voice_sample,
true,
)
}
}
@@ -322,6 +374,8 @@ impl AudioCapture for WasapiCapture {
frame_sink,
event_sink,
None,
None,
true,
)
}
@@ -339,6 +393,8 @@ impl AudioCapture for WasapiCapture {
frame_sink,
event_sink,
None,
None,
true,
)
}
@@ -376,7 +432,10 @@ struct CaptureSession {
/// was picked) — same "degrade gracefully rather than fail the recording" spirit
/// as the device-recovery reconnect below.
#[cfg(feature = "audio")]
fn find_device(direction: &Direction, device_id: Option<&str>) -> Result<wasapi::Device, AudioError> {
fn find_device(
direction: &Direction,
device_id: Option<&str>,
) -> Result<wasapi::Device, AudioError> {
let Some(id) = device_id else {
return wasapi::get_default_device(direction)
.map_err(|e| AudioError::Device(format!("no default audio device: {e}")));
@@ -458,12 +517,14 @@ fn format_compatible(a: &WaveFormat, b: &WaveFormat) -> bool {
}
/// Amplitude of one chunk of mono samples, for the live level meter (FR-CAP-5).
/// `mic` tags which stream this level came from so the UI can overlay both.
#[cfg(feature = "audio")]
fn audio_level(mono: &[f32]) -> AudioLevel {
fn audio_level(mono: &[f32], mic: bool) -> AudioLevel {
if mono.is_empty() {
return AudioLevel {
rms: 0.0,
peak: 0.0,
mic,
};
}
let sum_sq: f32 = mono.iter().map(|s| s * s).sum();
@@ -471,6 +532,7 @@ fn audio_level(mono: &[f32]) -> AudioLevel {
AudioLevel {
rms: (sum_sq / mono.len() as f32).sqrt(),
peak,
mic,
}
}
@@ -492,15 +554,16 @@ fn capture_loop(
running: &AtomicBool,
paused: &AtomicBool,
bridge: Option<&Arc<MicBridge>>,
voice_sample: Option<&Arc<VoiceSample>>,
emit_level: bool,
) -> Result<CaptureSummary, AudioError> {
wasapi::initialize_mta()
.ok()
.map_err(|e| AudioError::Device(format!("COM init failed: {e}")))?;
let is_loopback = matches!(direction, Direction::Render);
// Only the loopback stream emits level updates (FR-CAP-5) and writes a WAV;
// the mic stream just contributes frames to the transcript mixer (FR-CAP-7).
let emit_level = wav_path.is_some();
// Both streams emit level updates (FR-CAP-5/7) — tagged by `mic` below so the
// UI can overlay them — but only the loopback stream writes a WAV.
let mut session = open_capture_session(&direction, device_id)?;
// Loopback publishes its rate so the mic knows what to resample to before
// pushing into the shared bridge (mic-into-recording, FR-CAP-7).
@@ -612,12 +675,20 @@ fn capture_loop(
}
if emit_level && last_level_emit.elapsed() >= LEVEL_EMIT_INTERVAL {
let _ = event_sink.try_send(CaptureEvent::Level(audio_level(&mono)));
let _ = event_sink.try_send(CaptureEvent::Level(audio_level(&mono, !is_loopback)));
last_level_emit = Instant::now();
}
let resampled = resampler.process(&mono);
if !resampled.is_empty() {
// Mic-only, best-effort: a few seconds of raw mic audio for the
// post-recording voiceprint match (see `VoiceSample`). No-ops
// itself once its cap is reached.
if !is_loopback {
if let Some(vs) = voice_sample {
vs.push(&resampled);
}
}
let _ = frame_sink.try_send(resampled); // drop on backpressure; disk write is unaffected
}
}
@@ -688,7 +759,8 @@ fn write_wav_bytes(
for frame in bytes.chunks_exact(2 * channels) {
let add = mic.get(frames as usize).copied().unwrap_or(0.0);
for c in frame.chunks_exact(2) {
let v = i16::from_le_bytes(c.try_into().unwrap()) as f32 / i16::MAX as f32 + add;
let v =
i16::from_le_bytes(c.try_into().unwrap()) as f32 / i16::MAX as f32 + add;
writer
.write_sample(f32_to_i16(v))
.map_err(|e| AudioError::Capture(format!("wav write: {e}")))?;
@@ -1013,6 +1085,16 @@ mod tests {
assert!(m.loopback.is_empty());
}
#[test]
fn voice_sample_caps_and_stops_accepting_once_full() {
let vs = VoiceSample::new(4);
vs.push(&[1.0, 2.0, 3.0]);
vs.push(&[4.0, 5.0]); // would overflow the cap of 4
assert_eq!(vs.samples(), vec![1.0, 2.0, 3.0, 4.0]);
vs.push(&[9.0]); // already full — ignored
assert_eq!(vs.samples(), vec![1.0, 2.0, 3.0, 4.0]);
}
#[test]
fn wav_writer_writes_16bit_from_a_float_mix_and_keeps_duration() {
// A 32-bit-float mix is captured but written as 16-bit PCM at the same
@@ -1063,10 +1145,7 @@ mod tests {
let format = WaveFormat::new(32, 32, &SampleType::Float, 48_000, 2, None);
let mut writer = WavWriter::create(&path, wav_spec_for(&format).unwrap()).unwrap();
// One stereo frame of silence (L=0.0, R=0.0).
let bytes: Vec<u8> = [0.0f32, 0.0]
.iter()
.flat_map(|s| s.to_le_bytes())
.collect();
let bytes: Vec<u8> = [0.0f32, 0.0].iter().flat_map(|s| s.to_le_bytes()).collect();
let frames = write_wav_bytes(&mut writer, &bytes, &format, &[0.5]).unwrap();
writer.finalize().unwrap();
assert_eq!(frames, 1);
@@ -1328,17 +1407,21 @@ mod tests {
let (ev_tx, _ev) = std::sync::mpsc::sync_channel::<CaptureEvent>(8);
let (ev_tx2, _ev2) = std::sync::mpsc::sync_channel::<CaptureEvent>(8);
let loop_h =
match WasapiCapture.start_loopback_recording(&path, None, loop_tx, ev_tx, bridge.clone())
{
Ok(h) => h,
Err(e) => {
eprintln!("[mixrec] no loopback device: {e} (skipping)");
return;
}
};
let loop_h = match WasapiCapture.start_loopback_recording(
&path,
None,
loop_tx,
ev_tx,
bridge.clone(),
) {
Ok(h) => h,
Err(e) => {
eprintln!("[mixrec] no loopback device: {e} (skipping)");
return;
}
};
let mic_h = WasapiCapture
.start_microphone_recording(None, mic_tx, ev_tx2, bridge)
.start_microphone_recording(None, mic_tx, ev_tx2, bridge, None)
.ok();
let _ = std::process::Command::new("powershell")
@@ -1401,7 +1484,7 @@ mod tests {
#[test]
fn audio_level_of_empty_chunk_is_silence() {
let level = audio_level(&[]);
let level = audio_level(&[], false);
assert_eq!(level.rms, 0.0);
assert_eq!(level.peak, 0.0);
}
@@ -1409,14 +1492,14 @@ mod tests {
#[test]
fn audio_level_computes_rms_and_peak() {
// Two samples of equal magnitude: RMS equals that magnitude, peak too.
let level = audio_level(&[0.5, -0.5]);
let level = audio_level(&[0.5, -0.5], false);
assert!((level.rms - 0.5).abs() < 1e-6);
assert!((level.peak - 0.5).abs() < 1e-6);
}
#[test]
fn audio_level_peak_tracks_the_largest_magnitude_sample() {
let level = audio_level(&[0.1, -0.9, 0.3]);
let level = audio_level(&[0.1, -0.9, 0.3], false);
assert!((level.peak - 0.9).abs() < 1e-6);
}
+178 -14
View File
@@ -62,10 +62,41 @@ impl CalendarSource for PstSource {
let result = run_readpst(&input.path, &out_dir).and_then(|_| collect_events(&out_dir));
let _ = std::fs::remove_dir_all(&out_dir); // best-effort cleanup either way
result
// Bug fix: `from`/`to` used to be silently ignored for PST (only
// Graph honored a range) — a long-lived mailbox has no natural
// upper bound on history, so "import everything" meant every
// recurring series expanded across its full lifetime (up to 500
// occurrences each, T4.1's RECURRENCE_MAX_OCCURRENCES) plus every
// one-off entry (e.g. a decade of Outlook's auto-generated yearly
// holidays) the file has ever held. Filtering here, after parsing,
// is the simplest correct place: it doesn't need to change how
// readpst is invoked or how recurrence expansion works.
result.map(|events| filter_by_range(events, input.from, input.to))
}
}
#[cfg(feature = "pst")]
fn filter_by_range(
events: Vec<ImportedEvent>,
from: Option<i64>,
to: Option<i64>,
) -> Vec<ImportedEvent> {
if from.is_none() && to.is_none() {
return events;
}
events
.into_iter()
.filter(|e| match e.event.starts_at {
// An event with no known start time can't be range-tested —
// keep it rather than silently drop something the user might
// still want (this is rare; most PST appointments have a start).
None => true,
Some(start) => from.map_or(true, |f| start >= f) && to.map_or(true, |t| start <= t),
})
.collect()
}
#[cfg(feature = "pst")]
fn run_readpst(pst_path: &str, out_dir: &Path) -> Result<(), CalError> {
// -S: one file per item. -e: extension matches item type (.ics for
@@ -78,15 +109,25 @@ fn run_readpst(pst_path: &str, out_dir: &Path) -> Result<(), CalError> {
// still handled safely (collect_ics_files skips a file that doesn't
// read as valid UTF-8 rather than erroring), just silently dropped
// instead of correctly decoded. Revisit if that's observed in practice.
let output = Command::new("readpst")
.args(["-S", "-e", "-t", "a", "-o"])
let mut cmd = Command::new("readpst");
cmd.args(["-S", "-e", "-t", "a", "-o"])
.arg(out_dir)
.arg(pst_path)
.output()
.map_err(|e| match e.kind() {
std::io::ErrorKind::NotFound => CalError::ToolMissing,
_ => CalError::Open(e.to_string()),
})?;
.arg(pst_path);
// Bug fix: readpst.exe is a console-subsystem binary, and WhispAssist is
// a GUI app with no console of its own — Windows was popping a brand
// new console window for it on every import. CREATE_NO_WINDOW spawns it
// fully headless instead; readpst's own stdout/stderr are still
// captured normally via `.output()` below.
#[cfg(windows)]
{
use std::os::windows::process::CommandExt;
const CREATE_NO_WINDOW: u32 = 0x0800_0000;
cmd.creation_flags(CREATE_NO_WINDOW);
}
let output = cmd.output().map_err(|e| match e.kind() {
std::io::ErrorKind::NotFound => CalError::ToolMissing,
_ => CalError::Open(e.to_string()),
})?;
if !output.status.success() {
// readpst writes some errors to stdout rather than stderr; show
// whichever stream actually has text, stderr first.
@@ -460,6 +501,17 @@ fn parse_vevents(ics_text: &str, source: &str) -> Vec<ImportedEvent> {
}
}
_ => {
// Bug fix: a VEVENT with no UID line used to import
// with `raw_uid: None`, which the dedup unique index
// (source, raw_uid) explicitly exempts (`WHERE
// raw_uid IS NOT NULL`) — so every re-import created
// a brand-new duplicate row for it forever. Fall back
// to a deterministic hash of the event's own content
// so re-imports of the same source still resolve to
// the same key and upsert instead of duplicating.
let raw_uid = uid.take().or_else(|| {
Some(content_uid(&summary, &organizer, starts_at, ends_at))
});
events.push(ImportedEvent {
event: CalendarEvent {
id: uuid::Uuid::new_v4().to_string(),
@@ -469,7 +521,7 @@ fn parse_vevents(ics_text: &str, source: &str) -> Vec<ImportedEvent> {
starts_at,
ends_at,
description: description.take(),
raw_uid: uid.take(),
raw_uid,
},
attendees,
});
@@ -615,12 +667,37 @@ fn local_to_utc_secs(dt: chrono::NaiveDateTime) -> i64 {
}
}
/// Deterministic stand-in identity for a VEVENT that has no `UID` of its own
/// (see the `_` arm of `parse_vevents` above), so re-running the same import
/// resolves to the same `raw_uid` and upserts in place rather than mints a
/// fresh duplicate row every time. Format matches migration
/// `0008_calendar_dedup_cleanup.sql`'s SQL-side backfill exactly, so a
/// re-import after that cleanup migration converges onto the same row it
/// already collapsed duplicates into rather than minting one more.
pub(crate) fn content_uid(
subject: &Option<String>,
organizer: &Option<String>,
starts_at: Option<i64>,
ends_at: Option<i64>,
) -> String {
format!(
"content:{}|{}|{}|{}",
subject.as_deref().unwrap_or(""),
organizer.as_deref().unwrap_or(""),
starts_at.unwrap_or(0),
ends_at.unwrap_or(0),
)
}
/// Bug fix: this used to convert to the machine's *local* timezone before
/// formatting, so an occurrence's date (and therefore its dedup key, see
/// `raw_uid` below) could come out differently on two imports run either
/// side of a DST transition or a timezone change — silently producing a
/// second row for the same occurrence on re-import. UTC is stable no matter
/// when/where the import runs.
fn ymd_digits(unix_secs: i64) -> String {
match Utc.timestamp_opt(unix_secs, 0).single() {
Some(dt) => {
let d = dt.with_timezone(&Local).date_naive();
format!("{:04}{:02}{:02}", d.year(), d.month(), d.day())
}
Some(dt) => format!("{:04}{:02}{:02}", dt.year(), dt.month(), dt.day()),
None => String::new(),
}
}
@@ -1082,4 +1159,91 @@ END:VEVENT\r\n";
assert_eq!(e.event.ends_at.unwrap() - e.event.starts_at.unwrap(), 1800);
}
}
/// Regression for the duplicate-import bug: a VEVENT with no `UID` line
/// used to get `raw_uid: None`, which the storage layer's unique index
/// exempts from dedup entirely -- so re-importing the same source
/// duplicated it on every single run. It must now get a stable,
/// content-derived `raw_uid` so re-parsing the identical source resolves
/// to the same key.
#[test]
fn parse_vevents_gives_a_uid_less_event_a_stable_content_based_raw_uid() {
let ics = "BEGIN:VEVENT\r\n\
SUMMARY:No UID here\r\n\
DTSTART:20260112T163000Z\r\n\
DTEND:20260112T170000Z\r\n\
END:VEVENT\r\n";
let first = parse_vevents(ics, "pst");
let second = parse_vevents(ics, "pst");
assert_eq!(first.len(), 1);
assert_eq!(second.len(), 1);
assert!(first[0].event.raw_uid.is_some());
assert_eq!(first[0].event.raw_uid, second[0].event.raw_uid);
}
/// Distinct UID-less events (different subjects) must not collide onto
/// the same content-based key.
#[test]
fn content_based_raw_uid_differs_for_distinct_uid_less_events() {
let ics_a = "BEGIN:VEVENT\r\nSUMMARY:Event A\r\nDTSTART:20260112T163000Z\r\nDTEND:20260112T170000Z\r\nEND:VEVENT\r\n";
let ics_b = "BEGIN:VEVENT\r\nSUMMARY:Event B\r\nDTSTART:20260112T163000Z\r\nDTEND:20260112T170000Z\r\nEND:VEVENT\r\n";
let a = parse_vevents(ics_a, "pst");
let b = parse_vevents(ics_b, "pst");
assert_ne!(a[0].event.raw_uid, b[0].event.raw_uid);
}
#[test]
fn ymd_digits_is_timezone_independent() {
// A UTC midnight timestamp must format to the same UTC calendar date
// regardless of the machine's local timezone (the original bug:
// this used to convert to `Local` first, so the same recurring
// occurrence could compute a different dedup-key suffix on a machine
// in a different timezone, or after a DST transition).
let utc_new_year = Utc
.with_ymd_and_hms(2026, 1, 1, 0, 0, 0)
.unwrap()
.timestamp();
assert_eq!(ymd_digits(utc_new_year), "20260101");
}
fn dated_event(subject: &str, starts_at: Option<i64>) -> ImportedEvent {
ImportedEvent {
event: CalendarEvent {
id: uuid::Uuid::new_v4().to_string(),
source: "pst".to_string(),
subject: Some(subject.to_string()),
organizer: None,
starts_at,
ends_at: starts_at.map(|s| s + 1800),
description: None,
raw_uid: Some(subject.to_string()),
},
attendees: vec![],
}
}
#[test]
fn filter_by_range_keeps_everything_when_unbounded() {
let events = vec![dated_event("a", Some(0)), dated_event("b", Some(1_000_000))];
assert_eq!(filter_by_range(events.clone(), None, None).len(), 2);
}
#[test]
fn filter_by_range_excludes_events_outside_the_window() {
let events = vec![
dated_event("too old", Some(100)),
dated_event("in range", Some(500)),
dated_event("too new", Some(900)),
];
let kept = filter_by_range(events, Some(200), Some(800));
assert_eq!(kept.len(), 1);
assert_eq!(kept[0].event.subject.as_deref(), Some("in range"));
}
#[test]
fn filter_by_range_keeps_undated_events_rather_than_guessing() {
let events = vec![dated_event("no date", None)];
let kept = filter_by_range(events, Some(200), Some(800));
assert_eq!(kept.len(), 1);
}
}
+472 -32
View File
@@ -15,8 +15,8 @@ use crate::hardware::{HardwareDetector, WinHardwareDetector};
use crate::models::*;
use crate::notes::NotesRenderer;
use crate::paths::{
diarization_embedding_model_file, diarization_segmentation_model_file, meeting_dir,
settings_path, wa_root, whisper_model_file,
diarization_embedding_model_file, diarization_segmentation_model_file, manual_notes_file,
meeting_dir, settings_path, wa_root, whisper_model_file,
};
use crate::storage::{FinalizeMeeting, Meeting, NewMeeting, SummaryFile, SyncTargetRow};
use crate::transcription::{
@@ -29,6 +29,11 @@ use std::path::{Path, PathBuf};
use std::sync::{Arc, Mutex as StdMutex};
use tauri::{AppHandle, Emitter, Manager, State};
/// ~8s of 16kHz mono mic audio — enough for a stable speaker-embedding
/// voiceprint (see `diarization::voiceprint`) without holding minutes of raw
/// audio in memory for the whole meeting.
const MIC_VOICEPRINT_SAMPLES: usize = 16_000 * 8;
#[derive(Deserialize)]
pub struct StartRecordingArgs {
pub meeting_title: Option<String>,
@@ -70,6 +75,8 @@ fn default_settings() -> Settings {
retention_max_size_gb: None,
pst_last_path: None,
pst_auto_sync: false,
pst_import_range_days: None,
auto_record_calendar: false,
graph_calendar_enabled: false,
graph_calendar_credential_ref: None,
audio_output_device: None,
@@ -332,11 +339,14 @@ pub async fn start_recording(
// microphone that fails to open must not sink the meeting: we log and fall
// back to loopback-only (the mixer forwards loopback alone once its sink
// drops).
let (capture, mic_capture) = if settings.microphone_enabled {
let (capture, mic_capture, mic_voice_sample) = if settings.microphone_enabled {
// The mixer sums both streams for the live transcript; the bridge carries
// the mic into the loopback thread so the recorded WAV holds both sides
// at native quality (FR-CAP-7).
let bridge = crate::audio::MicBridge::shared();
// A few seconds of raw mic audio for the post-stop voiceprint match
// (bug: mic speaker mislabeled "S1"/"S2" instead of "You").
let voice_sample = crate::audio::VoiceSample::new(MIC_VOICEPRINT_SAMPLES);
let (loop_sink, mic_sink) = crate::audio::spawn_mixer(frame_tx);
let capture = WasapiCapture
.start_loopback_recording(
@@ -353,10 +363,12 @@ pub async fn start_recording(
mic_sink,
event_tx,
bridge,
Some(voice_sample.clone()),
)
.map_err(|e| tracing::warn!("microphone capture unavailable: {e}"))
.ok();
(capture, mic)
let voice_sample = mic.is_some().then_some(voice_sample);
(capture, mic, voice_sample)
} else {
let capture = WasapiCapture
.start(
@@ -366,7 +378,7 @@ pub async fn start_recording(
event_tx,
)
.map_err(|e| WaError::new("audio", e.to_string()))?;
(capture, None)
(capture, None, None)
};
// Fire-and-forget: exits on its own once `event_tx` drops at capture stop;
@@ -385,6 +397,7 @@ pub async fn start_recording(
"meetingId": meeting_id_for_capture_events,
"rms": level.rms,
"peak": level.peak,
"mic": level.mic,
}),
);
}
@@ -417,6 +430,9 @@ pub async fn start_recording(
let language_for_worker = language.clone();
let app_for_worker = app.clone();
let meeting_id_for_worker = meeting_id.clone();
// Live-transcript cadence; the low-overhead preset trades a touch of
// immediacy for half the decode load (see StreamTuning).
let stream_tuning = crate::transcription::StreamTuning::new(settings.low_overhead);
let transcription_worker = std::thread::Builder::new()
.name("wa-transcription".into())
.spawn(move || {
@@ -447,9 +463,19 @@ pub async fn start_recording(
if let Ok(mut lang) = language_state_for_worker.lock() {
*lang = transcriber.effective_language();
}
run_streaming_worker(transcriber.as_ref(), frame_rx, |segment| {
if let Ok(mut buf) = segments_for_worker.lock() {
buf.push(segment.clone());
run_streaming_worker(transcriber.as_ref(), frame_rx, stream_tuning, |segment| {
// Only committed lines are the real transcript; interim updates
// are live-UI only (emitted below). Storing interims would push
// a duplicate row per refresh into the buffer stop_recording and
// the live-diarization preview both read. A committed line's id
// is stable, so replace-in-place guards against any re-commit.
if !segment.interim {
if let Ok(mut buf) = segments_for_worker.lock() {
match buf.iter_mut().find(|s| s.id == segment.id) {
Some(existing) => *existing = segment.clone(),
None => buf.push(segment.clone()),
}
}
}
let _ = app_for_worker.emit(
"transcript://segment",
@@ -554,6 +580,8 @@ pub async fn start_recording(
language: language_state,
diarizer,
speaker_names,
mic_voice_sample,
manual_notes: Arc::new(StdMutex::new(ManualNotes::default())),
});
drop(guard);
@@ -615,17 +643,57 @@ pub async fn stop_recording(
// provisional passes (T4.3) produced. Skipped if diarization models
// aren't installed — `speaker_infos_from_segments` then falls back to
// the single pre-diarization "S1" placeholder, same as before Phase 4.
let mut final_spans: Option<Vec<SpeakerSpan>> = None;
if let Some(diarizer) = session.diarizer.clone() {
let diarizer_for_task = diarizer.clone();
let wav_path = session.wav_path.clone();
match tauri::async_runtime::spawn_blocking(move || diarizer_for_task.diarize(&wav_path))
.await
{
Ok(Ok(spans)) => diarizer.assign(&mut segments, &spans),
Ok(Ok(spans)) => {
diarizer.assign(&mut segments, &spans);
final_spans = Some(spans);
}
Ok(Err(e)) => tracing::warn!("final diarization pass failed: {e}"),
Err(e) => tracing::warn!("final diarization task failed: {e}"),
}
}
// Bug fix: identify which diarized cluster is the mic (voiceprint match
// against the mic-only sample) and auto-label it "You" — otherwise the
// mic speaker is just whichever cluster sherpa-onnx happened to call
// "S1". Never overrides a name the user already set live (T4.4).
if let (Some(voice_sample), Some(spans)) = (&session.mic_voice_sample, &final_spans) {
let mic_samples = voice_sample.samples();
match crate::diarization::voiceprint::match_mic_speaker(
&diarization_embedding_model_file(),
&mic_samples,
&session.wav_path,
spans,
) {
Ok(auto_names) => {
for (label, name) in auto_names {
let already_named = session
.speaker_names
.lock()
.map(|g| g.contains_key(&label))
.unwrap_or(true); // poisoned lock: don't guess, skip
if already_named {
continue;
}
if let Err(e) = state.store.rename_speaker(&meeting_id, &label, &name).await {
tracing::warn!("failed to persist auto speaker name: {e}");
continue;
}
if let Ok(mut names) = session.speaker_names.lock() {
names.insert(label, name);
}
}
}
Err(e) => tracing::warn!("mic voiceprint match failed: {e}"),
}
}
let speaker_names = session
.speaker_names
.lock()
@@ -665,8 +733,21 @@ pub async fn stop_recording(
let template = template_id
.as_deref()
.and_then(crate::notes::note_template_by_id);
let notes_md =
crate::notes::MarkdownNotes.to_markdown(&segments, &speakers, None, template.as_ref());
// Granola-style redesign: fold in whatever was captured live (freeform
// notes typed during the meeting + per-moment annotations) instead of
// generating notes.md from the transcript alone.
let manual_notes = session
.manual_notes
.lock()
.map(|g| g.clone())
.unwrap_or_default();
let notes_md = crate::notes::MarkdownNotes.merge(
&segments,
&speakers,
&manual_notes,
None,
template.as_ref(),
);
let _ = state.store.update_notes(&meeting_id, &notes_md).await;
let _ = app.emit(
@@ -854,13 +935,127 @@ pub async fn acknowledge_recording_consent() -> WaResult<()> {
save_settings(&settings)
}
// ---- Live notes: Granola-style redesign (freeform + per-moment, during recording) ----
/// Best-effort write-through of `manual_notes.json` — crash safety for what
/// the user typed live, same spirit as T2.8's recover-scan. Never fails the
/// calling command on a disk error; the in-memory copy (what `stop_recording`
/// reads) is already updated by the time this runs.
fn persist_manual_notes(meeting_id: &MeetingId, manual: &ManualNotes) {
match serde_json::to_vec_pretty(manual) {
Ok(bytes) => {
if let Err(e) = std::fs::write(manual_notes_file(meeting_id), bytes) {
tracing::warn!("failed to persist manual notes for {meeting_id}: {e}");
}
}
Err(e) => tracing::warn!("failed to serialize manual notes for {meeting_id}: {e}"),
}
}
/// Reads `manual_notes.json` for a finalize path with no live
/// `RecordingSession` to read it from in-memory — crash recovery
/// (`resume_transcription`, T2.8) and post-finalize batch re-transcription
/// (`reprocess_transcript`, T3.8) both re-render `notes.md` from scratch, and
/// must not silently drop whatever manual notes were captured during the
/// original recording. Defaults to empty if the file is missing (a meeting
/// with the mic-notes feature never used, or nothing typed) or unreadable.
fn load_manual_notes(meeting_id: &MeetingId) -> ManualNotes {
std::fs::read(manual_notes_file(meeting_id))
.ok()
.and_then(|bytes| serde_json::from_slice(&bytes).ok())
.unwrap_or_default()
}
/// Update the freeform notes typed live during an in-progress recording —
/// the "Notes pane, open and typable while recording" feature. Live-session
/// only: once a meeting is finalized, `notes.md` is the single editable
/// document and `update_notes` is the command for it.
#[tauri::command]
pub async fn update_live_notes(
state: State<'_, AppState>,
meeting_id: MeetingId,
markdown: String,
) -> WaResult<()> {
let guard = state.session.lock().await;
let session = guard
.as_ref()
.filter(|s| s.meeting_id == meeting_id)
.ok_or_else(|| WaError::new("recording", "no matching active recording"))?;
let manual = {
let mut manual = session
.manual_notes
.lock()
.unwrap_or_else(|e| e.into_inner());
manual.freeform_md = markdown;
manual.clone()
};
drop(guard);
persist_manual_notes(&meeting_id, &manual);
Ok(())
}
/// Attach (or clear, with `text: ""`) a note to a specific moment in an
/// in-progress recording — the "click a transcript line, add a note to it"
/// feature. Anchored by timestamp rather than segment id: a later batch
/// re-transcription (T3.8) can renumber segments, but never moves the moment
/// in time the note pointed at. Live-session only, same reasoning as
/// `update_live_notes`.
#[tauri::command]
pub async fn set_segment_note(
state: State<'_, AppState>,
meeting_id: MeetingId,
anchor_ms: u64,
text: String,
) -> WaResult<()> {
let guard = state.session.lock().await;
let session = guard
.as_ref()
.filter(|s| s.meeting_id == meeting_id)
.ok_or_else(|| WaError::new("recording", "no matching active recording"))?;
let manual = {
let mut manual = session
.manual_notes
.lock()
.unwrap_or_else(|e| e.into_inner());
upsert_segment_note(&mut manual.segment_notes, anchor_ms, text, now_unix());
manual.clone()
};
drop(guard);
persist_manual_notes(&meeting_id, &manual);
Ok(())
}
/// Update the note at `anchor_ms` in place if one already exists (matches
/// a re-click on an already-annotated segment), else append a new one.
fn upsert_segment_note(notes: &mut Vec<SegmentNote>, anchor_ms: u64, text: String, now: i64) {
match notes.iter_mut().find(|n| n.anchor_ms == anchor_ms) {
Some(existing) => {
existing.text = text;
existing.updated_at = now;
}
None => notes.push(SegmentNote {
anchor_ms,
text,
created_at: now,
updated_at: now,
}),
}
}
// ---- Speakers (Phase 4) ----
/// Re-renders and persists `notes.md` from a finalized meeting's current
/// (post-rename/post-merge) segments+speakers, and tells the frontend what
/// changed (T4.5/4.6, FR-SPK-3/5). This is what keeps `export_meeting` — which
/// just copies the already-rendered `notes.md` — in sync with naming changes
/// made after the meeting ends; `transcript.json` itself is untouched.
/// Tells the frontend a finalized meeting's speaker names/mapping changed
/// (T4.5/4.6, FR-SPK-3/5), e.g. after a rename or merge.
///
/// Bug fix: this used to also re-render and overwrite `notes.md` from the
/// current segments+speakers on every call — which, once the notes redesign
/// made `notes.md` the user's actual freely-edited document (manual notes +
/// transcript merged at finalize, see `MarkdownNotes::merge`), would have
/// silently destroyed whatever the user had written. `notes.md` is only ever
/// generated once, at finalize; a later rename updates the `speakers` table
/// and live UI display, and simply doesn't retroactively rewrite text
/// already baked into notes.md — same as any other manual edit isn't
/// retroactively touched either.
async fn refresh_notes_and_notify(
app: &AppHandle,
state: &State<'_, AppState>,
@@ -871,17 +1066,6 @@ async fn refresh_notes_and_notify(
.get_meeting(meeting_id)
.await
.map_err(|e| WaError::new("storage", e.to_string()))?;
let template = meeting
.template_id
.as_deref()
.and_then(crate::notes::note_template_by_id);
let notes_md = crate::notes::MarkdownNotes.to_markdown(
&meeting.segments,
&meeting.speakers,
None,
template.as_ref(),
);
let _ = state.store.update_notes(meeting_id, &notes_md).await;
let _ = app.emit(
"diarization://updated",
serde_json::json!({ "meetingId": meeting_id, "speakers": meeting.speakers }),
@@ -1537,9 +1721,13 @@ pub async fn reprocess_transcript(
.await
.map_err(|e| WaError::new("storage", e.to_string()))?;
let notes_md = crate::notes::MarkdownNotes.to_markdown(
// Bug fix: re-transcribing must not silently drop manual notes the user
// typed live during the original recording (see `load_manual_notes`).
let manual_notes = load_manual_notes(&meeting_id);
let notes_md = crate::notes::MarkdownNotes.merge(
&segments,
&meeting.speakers,
&manual_notes,
None,
template.as_ref(),
);
@@ -1552,6 +1740,173 @@ pub async fn reprocess_transcript(
Ok(())
}
/// Filename stem of a local path, for a sensible default meeting title; `None`
/// for a URL (yt-dlp's real title isn't fetched — the user can rename).
fn default_title_from_source(source: &str) -> Option<String> {
if crate::media::is_url(source) {
return None;
}
std::path::Path::new(source)
.file_stem()
.and_then(|s| s.to_str())
.map(|s| s.to_string())
.filter(|s| !s.trim().is_empty())
}
/// Manually add a meeting from an existing recording: a local audio/video file
/// or a URL (YouTube/streaming page, or a direct media URL). Shells out to
/// `ffmpeg` (transcode) and — for URLs — `yt-dlp` (both external, not bundled;
/// a missing tool is a clear error). The produced 16kHz-mono WAV becomes the
/// meeting's retained `audio.wav`, then goes through the same
/// transcription + diarization + finalize path as a live recording.
#[tauri::command]
pub async fn import_media(
app: AppHandle,
state: State<'_, AppState>,
source: String,
title: Option<String>,
) -> WaResult<MeetingId> {
let settings = load_settings();
let model_id = model_id_for(&settings);
let backend = backend_for(&settings);
let model_path = whisper_model_file(&model_id);
if !model_path.exists() {
return Err(WaError::new(
"transcription",
format!(
"whisper model '{model_id}' not found at {}; download it from Settings first",
model_path.display()
),
));
}
let language: Option<String> =
normalize_language(settings.whisper_language.as_deref()).map(|s| s.to_string());
let title = title
.filter(|t| !t.trim().is_empty())
.or_else(|| default_title_from_source(&source))
.unwrap_or_else(|| "Imported meeting".to_string());
let meeting_id = state
.store
.create_meeting(NewMeeting {
title,
calendar_event_id: None,
template_id: None,
language: language.clone(),
})
.await
.map_err(|e| WaError::new("storage", e.to_string()))?;
let dir = meeting_dir(&meeting_id);
let wav_path = dir.join("audio.wav");
// Transcode into the meeting's audio.wav off the async runtime (shells out
// to ffmpeg/yt-dlp). On any failure, drop the empty meeting so a bad import
// doesn't leave a husk row behind.
let transcode = tauri::async_runtime::spawn_blocking({
let source = source.clone();
let wav_path = wav_path.clone();
let dir = dir.clone();
move || {
let work = dir.join("import-tmp");
std::fs::create_dir_all(&work)?;
let r = crate::media::import_to_wav(&source, &wav_path, &work);
let _ = std::fs::remove_dir_all(&work);
r
}
})
.await
.map_err(|e| WaError::new("import", e.to_string()))?;
if let Err(e) = transcode {
let _ = state.store.delete_meeting(&meeting_id).await;
return Err(WaError::new("import", e.to_string()));
}
// Transcribe the produced WAV (same batch path as reprocess_transcript).
let (mut segments, resolved_language) = tauri::async_runtime::spawn_blocking({
let wav_path = wav_path.clone();
let model_path = model_path.clone();
let requested_language = language.clone();
move || {
let (transcriber, _used) =
load_transcriber(backend, &model_path, requested_language.as_deref())?;
let segments = transcriber.transcribe_file(&wav_path)?;
let resolved = transcriber
.detected_language()
.or_else(|| transcriber.effective_language());
Ok::<_, crate::transcription::TrxError>((segments, resolved))
}
})
.await
.map_err(|e| WaError::new("transcription", e.to_string()))?
.map_err(|e| WaError::new("transcription", e.to_string()))?;
// One diarization pass if the models are installed, exactly like
// stop_recording — otherwise every line stays the single "S1" placeholder.
let diarizer: Option<Arc<dyn Diarizer>> =
tauri::async_runtime::spawn_blocking(diarizer_from_installed_models)
.await
.ok()
.flatten()
.map(|d| Arc::new(d) as Arc<dyn Diarizer>);
if let Some(diarizer) = diarizer {
let diarizer_for_task = diarizer.clone();
let wp = wav_path.clone();
match tauri::async_runtime::spawn_blocking(move || diarizer_for_task.diarize(&wp)).await {
Ok(Ok(spans)) => diarizer.assign(&mut segments, &spans),
Ok(Err(e)) => tracing::warn!("import diarization pass failed: {e}"),
Err(e) => tracing::warn!("import diarization task failed: {e}"),
}
}
let speakers = speaker_infos_from_segments(&segments, &HashMap::new());
let duration_secs = segments
.last()
.map(|s| (s.end_ms / 1000) as i64)
.unwrap_or(0);
state
.store
.finalize_meeting(
&meeting_id,
FinalizeMeeting {
segments: segments.clone(),
speakers: speakers.clone(),
duration_secs,
recorded: true, // the imported WAV is the recording — keep it
language: resolved_language,
backend_used: Some(backend.as_str().to_string()),
model_used: Some(model_id.clone()),
},
)
.await
.map_err(|e| WaError::new("storage", e.to_string()))?;
let notes_md = crate::notes::MarkdownNotes.merge(
&segments,
&speakers,
&crate::models::ManualNotes::default(),
None,
None,
);
let _ = state.store.update_notes(&meeting_id, &notes_md).await;
// Seal the retained recording at rest when the vault is unlocked (T8.8),
// matching stop_recording so imports aren't left as plaintext outliers.
if crate::vault::is_unlocked() {
if let Ok(raw) = std::fs::read(&wav_path) {
if let Ok(sealed) = crate::vault::seal(&raw) {
let _ = std::fs::write(&wav_path, sealed);
}
}
}
let _ = app.emit(
"transcript://finalized",
serde_json::json!({ "meetingId": meeting_id, "segmentCount": segments.len() }),
);
Ok(meeting_id)
}
/// Re-run transcription from a `recovering` meeting's working `audio.wav`
/// (T2.8, FR-REL-1). CPU-bound, so it runs on a blocking task rather than
/// tying up an async worker.
@@ -1643,8 +1998,16 @@ pub async fn resume_transcription(
let template = template_id
.as_deref()
.and_then(crate::notes::note_template_by_id);
let notes_md =
crate::notes::MarkdownNotes.to_markdown(&segments, &speakers, None, template.as_ref());
// Crash recovery: no RecordingSession survives a crash, so read whatever
// manual notes were write-through persisted to disk before it happened.
let manual_notes = load_manual_notes(&meeting_id);
let notes_md = crate::notes::MarkdownNotes.merge(
&segments,
&speakers,
&manual_notes,
None,
template.as_ref(),
);
let _ = state.store.update_notes(&meeting_id, &notes_md).await;
let _ = app.emit(
@@ -2342,6 +2705,11 @@ pub async fn generate_summary(
if let Ok(sealed) = crate::vault::seal(json.as_bytes()) {
let _ = std::fs::write(meeting_dir(&meeting_id).join("summary.json"), sealed);
}
// Summary lives only in summary.json (not a store write), so the search
// index needs an explicit nudge to pick it up (FR-SEARCH-1).
if let Err(e) = state.store.reindex_fts(&meeting_id).await {
tracing::warn!("failed to reindex search after summary generation: {e}");
}
let _ = app.emit(
"llm://done",
@@ -2432,12 +2800,14 @@ pub(crate) async fn import_pst_core(
store: &dyn crate::storage::Store,
path: String,
password: Option<String>,
range_days: Option<u32>,
) -> WaResult<u32> {
let from = range_days.map(|d| now_unix() - (d as i64) * 86_400);
let events = tauri::async_runtime::spawn_blocking(move || {
PstSource.import(CalImport {
path,
password,
from: None,
from,
to: None,
})
})
@@ -2469,8 +2839,9 @@ pub async fn import_pst(
state: State<'_, AppState>,
path: String,
password: Option<String>,
range_days: Option<u32>,
) -> WaResult<u32> {
import_pst_core(&app, state.store.as_ref(), path, password).await
import_pst_core(&app, state.store.as_ref(), path, password, range_days).await
}
/// Browse imported calendar events (T6.3, FR-CAL-2).
@@ -2501,6 +2872,25 @@ pub async fn get_calendar_event(
.map_err(|e| WaError::new("storage", e.to_string()))
}
/// Prune imported calendar events (bug fix: unbounded PST history could
/// grow to tens of thousands of rows). `older_than_days: None` deletes
/// every unlinked event ("Delete all"); `Some(n)` only those starting more
/// than `n` days ago. An event attached to a recorded meeting is always
/// kept regardless of the choice.
#[tauri::command]
pub async fn cleanup_calendar_events(
state: State<'_, AppState>,
older_than_days: Option<u32>,
) -> WaResult<crate::storage::CalendarCleanupResult> {
let cutoff = older_than_days.map(|d| now_unix() - (d as i64) * 86_400);
let (deleted, protected) = state
.store
.cleanup_calendar_events(cutoff)
.await
.map_err(|e| WaError::new("storage", e.to_string()))?;
Ok(crate::storage::CalendarCleanupResult { deleted, protected })
}
/// Link a meeting (current or historical) to a calendar event (T6.3/T6.6,
/// FR-CAL-2/4).
#[tauri::command]
@@ -3945,6 +4335,56 @@ mod tests {
assert!(diarizer_from_installed_models().is_none());
}
// ---- Live notes redesign: upsert_segment_note / load_manual_notes ----
#[test]
fn upsert_segment_note_appends_a_new_anchor() {
let mut notes = Vec::new();
upsert_segment_note(&mut notes, 1_000, "first".to_string(), 100);
assert_eq!(notes.len(), 1);
assert_eq!(notes[0].anchor_ms, 1_000);
assert_eq!(notes[0].text, "first");
assert_eq!(notes[0].created_at, 100);
assert_eq!(notes[0].updated_at, 100);
}
#[test]
fn upsert_segment_note_edits_the_existing_anchor_in_place_without_resetting_created_at() {
let mut notes = vec![SegmentNote {
anchor_ms: 1_000,
text: "first".to_string(),
created_at: 100,
updated_at: 100,
}];
upsert_segment_note(&mut notes, 1_000, "edited".to_string(), 200);
assert_eq!(
notes.len(),
1,
"re-clicking the same segment must not duplicate it"
);
assert_eq!(notes[0].text, "edited");
assert_eq!(notes[0].created_at, 100);
assert_eq!(notes[0].updated_at, 200);
}
#[test]
fn upsert_segment_note_with_empty_text_clears_rather_than_removes() {
// Kept (not deleted) so `updated_at` still reflects the clear, and
// `transcript_with_notes` already skips blank-text notes when
// rendering (see notes/mod.rs).
let mut notes = Vec::new();
upsert_segment_note(&mut notes, 1_000, String::new(), 100);
assert_eq!(notes.len(), 1);
assert_eq!(notes[0].text, "");
}
#[test]
fn load_manual_notes_defaults_when_the_file_does_not_exist() {
let manual = load_manual_notes(&"no-such-meeting-id".to_string());
assert_eq!(manual.freeform_md, "");
assert!(manual.segment_notes.is_empty());
}
fn meeting_fixture() -> Meeting {
Meeting {
id: "m1".to_string(),
+1
View File
@@ -10,6 +10,7 @@ use crate::models::{SpeakerSpan, TranscriptSegment};
use std::path::Path;
pub mod models;
pub mod voiceprint;
#[derive(Debug, thiserror::Error)]
pub enum DiarError {
+258
View File
@@ -0,0 +1,258 @@
//! Voiceprint matching: identifies which diarized speaker cluster is the
//! meeting's own microphone, so it can be auto-labeled "You" instead of a
//! clustered "S1"/"S2" (bug: the mic speaker wasn't reliably first/labeled).
//! Mic and system audio are already summed into one mono stream before
//! diarization ever runs, so the only way to tell them apart afterwards is a
//! voiceprint: a short mic-only sample, captured live, compared by embedding
//! similarity against each cluster's own audio from the finished recording.
//! Runs once per meeting, entirely offline via the same sherpa-onnx
//! speaker-embedding model diarization already uses (ADR-0005).
use crate::models::SpeakerSpan;
use std::collections::HashMap;
use std::path::Path;
/// At least this much clean audio (mic sample or candidate cluster) before an
/// embedding computed from it is trusted at all — a fragment of a word gives
/// an unstable embedding that's as likely to mismatch as match.
const MIN_VOICEPRINT_SAMPLES: usize = 16_000; // 1s @ 16kHz
/// Per-candidate audio is capped so one very long-talking speaker doesn't
/// blow up embedding compute time; a few seconds is already stable.
const MAX_CANDIDATE_SAMPLES: usize = 16_000 * 10;
/// sherpa's own default "is this a match" similarity threshold
/// (`speaker_id::DEFAULT_SIMILARITY_THRESHOLD`) — kept as a local constant so
/// this module doesn't need the `diarization` feature just to state its
/// policy (used by both the real and no-op builds' doc comments/tests).
const SIMILARITY_THRESHOLD: f32 = 0.5;
#[derive(Debug, thiserror::Error)]
pub enum VoiceprintError {
#[error("model load failed: {0}")]
Load(String),
#[error("embedding failed: {0}")]
Embed(String),
#[error("failed to read the recording: {0}")]
Read(String),
}
/// The label -> display-name map to auto-apply after diarization: whichever
/// speaker's audio matches `mic_samples` best -> `"You"`; every other label,
/// in first-appearance order, -> `"Speaker 2"`, `"Speaker 3"`, … An empty map
/// means "couldn't tell" (too little mic audio, no cluster cleared the
/// similarity threshold, embedding model unavailable) — callers leave the
/// existing "S1"/"S2" labels alone rather than guess (FR-SPK-5).
#[cfg(feature = "diarization")]
pub fn match_mic_speaker(
embedding_model: &Path,
mic_samples: &[f32],
wav_path: &Path,
spans: &[SpeakerSpan],
) -> Result<HashMap<String, String>, VoiceprintError> {
if mic_samples.len() < MIN_VOICEPRINT_SAMPLES || spans.is_empty() {
return Ok(HashMap::new());
}
let labels_in_order = first_appearance_order(spans);
let wav_samples = crate::audio::read_wav_mono_16k(wav_path)
.map_err(|e| VoiceprintError::Read(e.to_string()))?;
let mut extractor =
sherpa_rs::speaker_id::EmbeddingExtractor::new(sherpa_rs::speaker_id::ExtractorConfig {
model: embedding_model.to_string_lossy().to_string(),
..Default::default()
})
.map_err(|e| VoiceprintError::Load(e.to_string()))?;
let mic_embedding = extractor
.compute_speaker_embedding(mic_samples.to_vec(), 16_000)
.map_err(|e| VoiceprintError::Embed(e.to_string()))?;
let mut best: Option<(&str, f32)> = None;
for label in &labels_in_order {
let candidate_samples = candidate_audio(&wav_samples, spans, label);
if candidate_samples.len() < MIN_VOICEPRINT_SAMPLES {
continue;
}
let embedding = extractor
.compute_speaker_embedding(candidate_samples, 16_000)
.map_err(|e| VoiceprintError::Embed(e.to_string()))?;
let score = cosine_similarity(&mic_embedding, &embedding);
let is_better = match best {
Some((_, best_score)) => score > best_score,
None => true,
};
if is_better {
best = Some((label, score));
}
}
let Some((mic_label, score)) = best else {
return Ok(HashMap::new());
};
if score < SIMILARITY_THRESHOLD {
return Ok(HashMap::new());
}
Ok(build_name_map(&labels_in_order, mic_label))
}
#[cfg(not(feature = "diarization"))]
pub fn match_mic_speaker(
_embedding_model: &Path,
_mic_samples: &[f32],
_wav_path: &Path,
_spans: &[SpeakerSpan],
) -> Result<HashMap<String, String>, VoiceprintError> {
Ok(HashMap::new())
}
/// Distinct speaker labels in first-appearance order — spans come back from
/// the diarizer already sorted by start time.
fn first_appearance_order(spans: &[SpeakerSpan]) -> Vec<String> {
let mut seen = std::collections::HashSet::new();
spans
.iter()
.filter(|s| seen.insert(s.speaker.clone()))
.map(|s| s.speaker.clone())
.collect()
}
/// Concatenates up to `MAX_CANDIDATE_SAMPLES` of `label`'s audio out of the
/// full 16kHz-mono recording, using each span's millisecond range.
fn candidate_audio(wav_samples: &[f32], spans: &[SpeakerSpan], label: &str) -> Vec<f32> {
const SAMPLES_PER_MS: u64 = 16; // 16_000 Hz / 1000
let mut out = Vec::new();
for span in spans.iter().filter(|s| s.speaker == label) {
if out.len() >= MAX_CANDIDATE_SAMPLES {
break;
}
let start = (span.start_ms * SAMPLES_PER_MS) as usize;
let end = ((span.end_ms * SAMPLES_PER_MS) as usize).min(wav_samples.len());
if start < end {
out.extend_from_slice(&wav_samples[start..end]);
}
}
out.truncate(MAX_CANDIDATE_SAMPLES);
out
}
fn cosine_similarity(a: &[f32], b: &[f32]) -> f32 {
let dot: f32 = a.iter().zip(b).map(|(x, y)| x * y).sum();
let norm_a = a.iter().map(|x| x * x).sum::<f32>().sqrt();
let norm_b = b.iter().map(|x| x * x).sum::<f32>().sqrt();
if norm_a == 0.0 || norm_b == 0.0 {
0.0
} else {
dot / (norm_a * norm_b)
}
}
/// `mic_label` -> "You"; every other label, in first-appearance order ->
/// "Speaker 2", "Speaker 3", … (numbering starts at 2 — "You" stands in for
/// "Speaker 1" without ever being called that).
fn build_name_map(labels_in_order: &[String], mic_label: &str) -> HashMap<String, String> {
let mut names = HashMap::new();
let mut next_speaker_number = 2;
for label in labels_in_order {
if label == mic_label {
names.insert(label.clone(), "You".to_string());
} else {
names.insert(label.clone(), format!("Speaker {next_speaker_number}"));
next_speaker_number += 1;
}
}
names
}
#[cfg(test)]
mod tests {
use super::*;
fn span(start_ms: u64, end_ms: u64, speaker: &str) -> SpeakerSpan {
SpeakerSpan {
start_ms,
end_ms,
speaker: speaker.to_string(),
}
}
#[test]
fn first_appearance_order_dedupes_in_encounter_order() {
let spans = vec![
span(0, 1000, "S2"),
span(1000, 2000, "S1"),
span(2000, 3000, "S2"),
];
assert_eq!(first_appearance_order(&spans), vec!["S2", "S1"]);
}
#[test]
fn candidate_audio_concatenates_only_that_speakers_spans() {
let wav: Vec<f32> = (0..32_000).map(|i| i as f32).collect(); // 2s @16kHz
let spans = vec![
span(0, 500, "S1"),
span(500, 1000, "S2"),
span(1000, 1500, "S1"),
];
let s1 = candidate_audio(&wav, &spans, "S1");
// 500ms + 500ms of S1 = 1s = 16_000 samples, taken from [0,8000) and [16000,24000).
assert_eq!(s1.len(), 16_000);
assert_eq!(s1[0], 0.0);
assert_eq!(s1[8000], 16_000.0);
}
#[test]
fn candidate_audio_caps_at_the_maximum() {
let wav: Vec<f32> = vec![0.0; MAX_CANDIDATE_SAMPLES + 10_000];
let spans = vec![span(0, (MAX_CANDIDATE_SAMPLES as u64 + 10_000) / 16, "S1")];
assert_eq!(
candidate_audio(&wav, &spans, "S1").len(),
MAX_CANDIDATE_SAMPLES
);
}
#[test]
fn cosine_similarity_of_identical_vectors_is_one() {
let v = [1.0, 2.0, 3.0];
assert!((cosine_similarity(&v, &v) - 1.0).abs() < 1e-6);
}
#[test]
fn cosine_similarity_of_opposite_vectors_is_negative_one() {
let a = [1.0, 0.0];
let b = [-1.0, 0.0];
assert!((cosine_similarity(&a, &b) + 1.0).abs() < 1e-6);
}
#[test]
fn cosine_similarity_handles_a_zero_vector_without_dividing_by_zero() {
let a = [0.0, 0.0];
let b = [1.0, 1.0];
assert_eq!(cosine_similarity(&a, &b), 0.0);
}
#[test]
fn build_name_map_labels_the_mic_you_and_numbers_the_rest_from_two() {
let labels = vec!["S2".to_string(), "S1".to_string(), "S3".to_string()];
let names = build_name_map(&labels, "S1");
assert_eq!(names.get("S1"), Some(&"You".to_string()));
assert_eq!(names.get("S2"), Some(&"Speaker 2".to_string()));
assert_eq!(names.get("S3"), Some(&"Speaker 3".to_string()));
}
#[test]
fn match_mic_speaker_returns_empty_when_mic_sample_is_too_short() {
let spans = vec![span(0, 1000, "S1")];
let names = match_mic_speaker(
Path::new("model.onnx"),
&[0.0; 100],
Path::new("audio.wav"),
&spans,
)
.unwrap();
assert!(names.is_empty());
}
}
+26 -3
View File
@@ -14,6 +14,7 @@ pub mod error;
pub mod hardware;
pub mod llm;
pub mod mcp;
pub mod media;
pub mod models;
pub mod notes;
pub mod paths;
@@ -78,6 +79,19 @@ pub struct RecordingSession {
/// (T4.4, FR-SPK-2). Never rewritten onto segments (FR-SPK-5); resolved
/// at render/finalize time instead.
pub speaker_names: Arc<StdMutex<std::collections::HashMap<String, String>>>,
/// A few seconds of raw mic-only audio, captured once early in the
/// recording — used at `stop_recording` to voiceprint-match the mic
/// against the diarized speaker clusters so the mic speaker can be
/// auto-labeled "You" instead of a clustered "S1"/"S2". `None` when the
/// mic is disabled (same conditions as `mic_capture`).
pub mic_voice_sample: Option<Arc<audio::VoiceSample>>,
/// Live notes redesign: raw user-authored notes accumulated *during* the
/// recording (freeform text + per-moment annotations) — see
/// `models::ManualNotes`. Mutated by `update_live_notes`/`set_segment_note`
/// and write-through persisted to `manual_notes.json` on every edit (crash
/// safety, same spirit as T2.8 recovery); folded into the final `notes.md`
/// at `stop_recording` via `notes::MarkdownNotes::merge`.
pub manual_notes: Arc<StdMutex<models::ManualNotes>>,
}
/// Wraps the tray icon so it can be looked up from commands to update its
@@ -190,9 +204,14 @@ pub fn run() {
let pst_settings = commands::load_settings();
if pst_settings.pst_auto_sync {
if let Some(path) = pst_settings.pst_last_path {
if let Err(e) =
commands::import_pst_core(&startup_app, store.as_ref(), path, None)
.await
if let Err(e) = commands::import_pst_core(
&startup_app,
store.as_ref(),
path,
None,
pst_settings.pst_import_range_days,
)
.await
{
tracing::warn!("startup PST auto-sync failed: {e:?}");
}
@@ -210,6 +229,8 @@ pub fn run() {
commands::resume_recording,
commands::set_recording_retention,
commands::acknowledge_recording_consent,
commands::update_live_notes,
commands::set_segment_note,
commands::resume_transcription,
commands::app_info,
commands::open_url,
@@ -225,6 +246,7 @@ pub fn run() {
commands::download_model,
commands::remove_model,
commands::reprocess_transcript,
commands::import_media,
commands::list_meetings,
commands::search,
commands::set_tags,
@@ -246,6 +268,7 @@ pub fn run() {
commands::llm_setup_suggestions,
commands::pull_ollama_model,
commands::import_pst,
commands::cleanup_calendar_events,
commands::list_calendar_events,
commands::get_calendar_event,
commands::attach_meeting_to_event,
+157
View File
@@ -0,0 +1,157 @@
//! Media import: turn an arbitrary local audio/video file, a direct media URL,
//! or a streaming/YouTube page URL into WhispAssist's canonical 16kHz-mono
//! 16-bit WAV, so it can go through the same transcription/diarization path as a
//! live recording (manual "add a meeting from a file/URL" feature).
//!
//! Two external tools do the work and are deliberately **not bundled** (same
//! call as `readpst` for .pst, ADR-0008): they must be installed and on PATH.
//! - `ffmpeg` transcodes whatever we have to the target WAV.
//! - `yt-dlp` resolves URLs (YouTube and other sites via its extractors, and
//! direct media URLs via its generic extractor) down to an audio file that
//! ffmpeg can then convert.
//!
//! A missing tool surfaces as a clear, named error rather than a generic
//! "program not found".
use std::path::{Path, PathBuf};
use std::process::Command;
#[derive(Debug, thiserror::Error)]
pub enum MediaError {
#[error("{0} isn't installed or on PATH — install it and try again")]
ToolMissing(&'static str),
#[error("{tool} failed: {message}")]
Failed {
tool: &'static str,
message: String,
},
#[error("io error: {0}")]
Io(#[from] std::io::Error),
}
/// Whether `source` should be resolved as a URL (via yt-dlp) rather than opened
/// as a local file path.
pub fn is_url(source: &str) -> bool {
let s = source.trim_start();
s.starts_with("http://") || s.starts_with("https://")
}
/// A `Command` for `program` that, on Windows, never pops a console window —
/// WhispAssist is a GUI app with no console of its own (same fix as `readpst`).
fn command(program: &str) -> Command {
let mut cmd = Command::new(program);
#[cfg(windows)]
{
use std::os::windows::process::CommandExt;
const CREATE_NO_WINDOW: u32 = 0x0800_0000;
cmd.creation_flags(CREATE_NO_WINDOW);
}
cmd
}
/// Run `cmd`, mapping a missing binary to `ToolMissing(program)` and a non-zero
/// exit to `Failed` carrying the tail of the tool's output (where the real
/// error message from ffmpeg/yt-dlp lives — both are verbose).
fn run(program: &'static str, cmd: &mut Command) -> Result<(), MediaError> {
let output = cmd.output().map_err(|e| match e.kind() {
std::io::ErrorKind::NotFound => MediaError::ToolMissing(program),
_ => MediaError::Io(e),
})?;
if !output.status.success() {
let stderr = String::from_utf8_lossy(&output.stderr);
let stdout = String::from_utf8_lossy(&output.stdout);
let text = if stderr.trim().is_empty() { stdout } else { stderr };
let tail: Vec<&str> = text.lines().filter(|l| !l.trim().is_empty()).collect();
let start = tail.len().saturating_sub(6);
return Err(MediaError::Failed {
tool: program,
message: tail[start..].join("\n"),
});
}
Ok(())
}
/// Transcode `source` (a local file path or a URL) into a 16kHz-mono 16-bit WAV
/// at `dest_wav`. URLs are first fetched with yt-dlp into `work_dir` (which the
/// caller creates and cleans up), then ffmpeg converts whatever landed. Blocks;
/// run it off the async runtime.
pub fn import_to_wav(source: &str, dest_wav: &Path, work_dir: &Path) -> Result<(), MediaError> {
if is_url(source) {
// bestaudio keeps the download small; ffmpeg does the actual 16kHz-mono
// conversion in one predictable pass. `--no-part` avoids a leftover
// `.part` file so `first_file_in` finds the finished download.
let template = work_dir.join("download.%(ext)s");
let mut cmd = command("yt-dlp");
cmd.args(["-f", "bestaudio/best", "--no-playlist", "--no-part", "-o"])
.arg(&template)
.arg(source);
run("yt-dlp", &mut cmd)?;
let downloaded = first_file_in(work_dir)?.ok_or(MediaError::Failed {
tool: "yt-dlp",
message: "no media file was produced".into(),
})?;
ffmpeg_to_wav(&downloaded, dest_wav)
} else {
let input = Path::new(source);
if !input.exists() {
return Err(MediaError::Failed {
tool: "import",
message: format!("file not found: {source}"),
});
}
ffmpeg_to_wav(input, dest_wav)
}
}
/// ffmpeg: any input → 16kHz mono 16-bit PCM WAV (drops video, matches the
/// format `audio::read_wav_mono_16k` and the diarizer both expect).
fn ffmpeg_to_wav(input: &Path, dest_wav: &Path) -> Result<(), MediaError> {
let mut cmd = command("ffmpeg");
cmd.args(["-hide_banner", "-loglevel", "error", "-y", "-i"])
.arg(input)
.args(["-vn", "-ac", "1", "-ar", "16000", "-c:a", "pcm_s16le"])
.arg(dest_wav);
run("ffmpeg", &mut cmd)
}
/// The first regular file in `dir` (the single yt-dlp download; the caller uses
/// a fresh temp dir per import so there's nothing else there).
fn first_file_in(dir: &Path) -> Result<Option<PathBuf>, MediaError> {
for entry in std::fs::read_dir(dir)? {
let entry = entry?;
if entry.file_type()?.is_file() {
return Ok(Some(entry.path()));
}
}
Ok(None)
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn is_url_distinguishes_urls_from_paths() {
assert!(is_url("https://youtube.com/watch?v=abc"));
assert!(is_url("http://example.com/a.mp4"));
assert!(is_url(" https://leading-space.example/x")); // trimmed
assert!(!is_url(r"C:\Users\me\meeting.mp4"));
assert!(!is_url("/home/me/meeting.m4a"));
assert!(!is_url("meeting.wav"));
assert!(!is_url("ftp://not-http.example/x"));
}
#[test]
fn import_reports_a_missing_local_file_without_touching_a_tool() {
let dir = std::env::temp_dir().join(format!("wa-media-{}", uuid::Uuid::new_v4()));
std::fs::create_dir_all(&dir).unwrap();
let err = import_to_wav(
"no-such-file.mp4",
&dir.join("out.wav"),
&dir,
)
.unwrap_err();
assert!(matches!(err, MediaError::Failed { tool: "import", .. }));
let _ = std::fs::remove_dir_all(&dir);
}
}
+49
View File
@@ -114,6 +114,41 @@ pub struct SpeakerInfo {
pub participant_id: Option<String>,
}
/// A user-typed note attached to a moment in the recording, anchored by
/// timestamp rather than segment id — a segment id can be invalidated by a
/// later batch re-transcription (T3.8), but the moment in time it pointed at
/// never changes. `text: ""` marks a cleared note (kept rather than removed
/// so `updated_at` still reflects the clear).
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct SegmentNote {
pub anchor_ms: u64,
pub text: String,
pub created_at: i64,
pub updated_at: i64,
}
/// On-disk shape of `manual_notes.json` (`docs/03-data-model.md`) — the raw
/// user-authored input a live recording accumulates (freeform notes typed
/// while recording, plus any per-moment annotations), kept distinct from the
/// transcript-derived `notes.md` so a re-render never has to guess which
/// parts of `notes.md` were hand-written.
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct ManualNotes {
pub schema: u32,
pub freeform_md: String,
pub segment_notes: Vec<SegmentNote>,
}
impl Default for ManualNotes {
fn default() -> Self {
Self {
schema: 1,
freeform_md: String::new(),
segment_notes: Vec::new(),
}
}
}
/// A diarization result span before alignment to transcript segments.
#[derive(Debug, Clone)]
pub struct SpeakerSpan {
@@ -242,6 +277,20 @@ pub struct Settings {
pub pst_last_path: Option<String>,
#[serde(default)]
pub pst_auto_sync: bool,
/// How far back to import (days before "now"); `None` = full mailbox
/// history (the original, unbounded behavior). Applied to both a manual
/// Import click and the `pst_auto_sync` startup re-import — a long-lived
/// mailbox otherwise re-imports its entire multi-year history (every
/// recurring series expanded to its cap, every one-off holiday entry
/// Outlook ever generated) on every launch.
#[serde(default)]
pub pst_import_range_days: Option<u32>,
/// Auto-start recording when a calendar event begins while the app is open
/// (FR-CAL, opt-in). OFF by default. No background timer runs for this: the
/// UI arms a single one-shot timer to the next event while the app is open
/// and disarms it on close, so idle resource use stays at zero (NFR-RES-1).
#[serde(default)]
pub auto_record_calendar: bool,
// Microsoft Graph calendar source (M4.4, T8.9, ADR-0008, FR-CAL-6). Opt-in,
// explicit consent via OAuth PKCE — off by default. The credential ref
// points into the OS credential store; the token itself never lives here.
+220 -44
View File
@@ -3,7 +3,7 @@
//! Renders speaker-tagged Markdown from transcript + speaker names (+ optional
//! summary). Names are resolved here from the mapping; segments keep internal IDs.
use crate::models::{SpeakerInfo, TranscriptSegment};
use crate::models::{ManualNotes, SegmentNote, SpeakerInfo, TranscriptSegment};
use pulldown_cmark::{Event, HeadingLevel, Options, Parser, Tag, TagEnd};
use serde::{Deserialize, Serialize};
use std::path::{Path, PathBuf};
@@ -101,6 +101,156 @@ pub trait NotesRenderer: Send + Sync {
pub struct MarkdownNotes;
impl MarkdownNotes {
/// Bug fix / redesign: `notes.md` used to be generated *only* from the
/// transcript (`to_markdown`), then blindly overwritten on every
/// finalize/speaker-rename, discarding anything the user typed. `merge`
/// is what `stop_recording` now calls instead — it folds in whatever was
/// captured live in `manual` (freeform notes typed during the meeting,
/// plus any per-moment annotations) alongside the transcript, so the
/// generated document isn't transcript-only and isn't a single
/// same-speaker-collapsed blob (`## Notes` / `## Transcript` sections,
/// with each annotation placed right after the paragraph it points at).
pub fn merge(
&self,
segments: &[TranscriptSegment],
speakers: &[SpeakerInfo],
manual: &ManualNotes,
summary_md: Option<&str>,
template: Option<&NoteTemplate>,
) -> String {
let mut out = String::new();
push_prelude(&mut out, summary_md, template);
let freeform = manual.freeform_md.trim();
if !freeform.is_empty() {
out.push_str("## Notes\n\n");
out.push_str(freeform);
out.push_str("\n\n");
}
let transcript = transcript_with_notes(segments, speakers, &manual.segment_notes);
if !transcript.is_empty() {
out.push_str("## Transcript\n\n");
out.push_str(&transcript);
out.push_str("\n\n");
}
out.trim_end().to_string()
}
}
/// Template section scaffold + summary block shared by `to_markdown` and `merge`.
fn push_prelude(out: &mut String, summary_md: Option<&str>, template: Option<&NoteTemplate>) {
if let Some(template) = template {
for section in &template.sections {
out.push_str(&format!("## {section}\n\n"));
}
}
if let Some(summary) = summary_md {
let summary = summary.trim();
if !summary.is_empty() {
out.push_str(summary);
out.push_str("\n\n---\n\n");
}
}
}
/// Groups consecutive same-speaker segments into `(start_ms, end_ms,
/// "**Name:** text")` paragraphs — the ms range is each paragraph's span in
/// the original recording, used by `transcript_with_notes` to place a
/// per-moment note right after the paragraph it was anchored to.
fn dialogue_paragraphs(
segments: &[TranscriptSegment],
speakers: &[SpeakerInfo],
) -> Vec<(u64, u64, String)> {
let name_for = |label: &str| -> String {
speakers
.iter()
.find(|s| s.label == label)
.and_then(|s| s.display_name.clone())
.unwrap_or_else(|| label.to_string())
};
let mut out = Vec::new();
let mut current_speaker: Option<&str> = None;
let mut buffer = String::new();
let mut range: (u64, u64) = (0, 0);
for seg in segments {
let text = seg.text.trim();
if text.is_empty() {
continue;
}
if current_speaker != Some(seg.speaker.as_str()) {
if let Some(speaker) = current_speaker {
out.push((
range.0,
range.1,
format!("**{}:** {}", name_for(speaker), buffer.trim()),
));
}
current_speaker = Some(seg.speaker.as_str());
buffer.clear();
range = (seg.start_ms, seg.end_ms);
}
if !buffer.is_empty() {
buffer.push(' ');
}
buffer.push_str(text);
range.1 = seg.end_ms;
}
if let Some(speaker) = current_speaker {
out.push((
range.0,
range.1,
format!("**{}:** {}", name_for(speaker), buffer.trim()),
));
}
out
}
/// The dialogue paragraphs, each followed by a `> 📝` blockquote for any
/// `segment_notes` whose `anchor_ms` falls inside that paragraph's span.
/// A note whose anchor doesn't land inside any paragraph (its segment fell
/// in a gap, or vanished in a later re-transcription) still isn't dropped —
/// it surfaces under "Other notes" at the end instead of silently
/// disappearing.
fn transcript_with_notes(
segments: &[TranscriptSegment],
speakers: &[SpeakerInfo],
segment_notes: &[SegmentNote],
) -> String {
let mut out = String::new();
let mut matched = vec![false; segment_notes.len()];
for (start, end, paragraph) in dialogue_paragraphs(segments, speakers) {
out.push_str(&paragraph);
out.push_str("\n\n");
for (i, note) in segment_notes.iter().enumerate() {
let text = note.text.trim();
if text.is_empty() || note.anchor_ms < start || note.anchor_ms > end {
continue;
}
out.push_str(&format!("> 📝 {text}\n\n"));
matched[i] = true;
}
}
let orphans: Vec<&str> = segment_notes
.iter()
.zip(matched.iter())
.filter(|(n, was_matched)| !**was_matched && !n.text.trim().is_empty())
.map(|(n, _)| n.text.trim())
.collect();
if !orphans.is_empty() {
out.push_str("### Other notes\n\n");
for text in orphans {
out.push_str(&format!("> 📝 {text}\n\n"));
}
}
out.trim_end().to_string()
}
impl NotesRenderer for MarkdownNotes {
fn to_markdown(
&self,
@@ -110,50 +260,11 @@ impl NotesRenderer for MarkdownNotes {
template: Option<&NoteTemplate>,
) -> String {
let mut out = String::new();
if let Some(template) = template {
for section in &template.sections {
out.push_str(&format!("## {section}\n\n"));
}
}
if let Some(summary) = summary_md {
let summary = summary.trim();
if !summary.is_empty() {
out.push_str(summary);
out.push_str("\n\n---\n\n");
}
}
push_prelude(&mut out, summary_md, template);
let name_for = |label: &str| -> String {
speakers
.iter()
.find(|s| s.label == label)
.and_then(|s| s.display_name.clone())
.unwrap_or_else(|| label.to_string())
};
// Group consecutive segments from the same speaker into one paragraph
// (matters once Phase 4 diarization produces more than one speaker).
let mut current_speaker: Option<&str> = None;
let mut buffer = String::new();
for seg in segments {
let text = seg.text.trim();
if text.is_empty() {
continue;
}
if current_speaker != Some(seg.speaker.as_str()) {
if let Some(speaker) = current_speaker {
out.push_str(&format!("**{}:** {}\n\n", name_for(speaker), buffer.trim()));
}
current_speaker = Some(seg.speaker.as_str());
buffer.clear();
}
if !buffer.is_empty() {
buffer.push(' ');
}
buffer.push_str(text);
}
if let Some(speaker) = current_speaker {
out.push_str(&format!("**{}:** {}\n\n", name_for(speaker), buffer.trim()));
for (_, _, paragraph) in dialogue_paragraphs(segments, speakers) {
out.push_str(&paragraph);
out.push_str("\n\n");
}
out.trim_end().to_string()
@@ -350,6 +461,71 @@ mod tests {
assert_eq!(md, "## Discussion\n\n## Action Items\n\n**S1:** Hi");
}
fn note(anchor_ms: u64, text: &str) -> SegmentNote {
SegmentNote {
anchor_ms,
text: text.to_string(),
created_at: 0,
updated_at: 0,
}
}
#[test]
fn merge_with_no_manual_notes_matches_to_markdown() {
let segments = vec![seg(0, "S1", "Hello"), seg(1, "S2", "Hi there.")];
let manual = ManualNotes::default();
assert_eq!(
MarkdownNotes.merge(&segments, &[], &manual, None, None),
"## Transcript\n\n**S1:** Hello\n\n**S2:** Hi there."
);
}
#[test]
fn merge_adds_a_notes_section_for_freeform_text() {
let segments = vec![seg(0, "S1", "Hello")];
let manual = ManualNotes {
freeform_md: "Remember to follow up.".to_string(),
..Default::default()
};
assert_eq!(
MarkdownNotes.merge(&segments, &[], &manual, None, None),
"## Notes\n\nRemember to follow up.\n\n## Transcript\n\n**S1:** Hello"
);
}
#[test]
fn merge_places_a_segment_note_right_after_its_paragraph() {
// seg 0 spans 0-900ms ("S1"); anchor 500 falls inside it.
let segments = vec![seg(0, "S1", "Hello"), seg(1, "S2", "Hi there.")];
let manual = ManualNotes {
segment_notes: vec![note(500, "circle back on this")],
..Default::default()
};
assert_eq!(
MarkdownNotes.merge(&segments, &[], &manual, None, None),
"## Transcript\n\n**S1:** Hello\n\n> \u{1f4dd} circle back on this\n\n**S2:** Hi there."
);
}
#[test]
fn merge_surfaces_an_unmatched_anchor_under_other_notes() {
let segments = vec![seg(0, "S1", "Hello")]; // spans 0-900ms
let manual = ManualNotes {
segment_notes: vec![note(50_000, "way outside any paragraph")],
..Default::default()
};
assert_eq!(
MarkdownNotes.merge(&segments, &[], &manual, None, None),
"## Transcript\n\n**S1:** Hello\n\n### Other notes\n\n> \u{1f4dd} way outside any paragraph"
);
}
#[test]
fn merge_omits_empty_sections_for_an_empty_meeting() {
let manual = ManualNotes::default();
assert_eq!(MarkdownNotes.merge(&[], &[], &manual, None, None), "");
}
#[test]
fn markdown_to_blocks_parses_heading_bold_prefix_and_task_list() {
let md = "## Summary\n\n**Alice:** Hello world.\n\n- [ ] Follow up\n- [x] Done thing";
+7
View File
@@ -29,6 +29,13 @@ pub fn meeting_dir(id: &MeetingId) -> PathBuf {
meetings_dir().join(id)
}
/// Raw user-authored notes (freeform + per-moment annotations) accumulated
/// live during a recording — see `models::ManualNotes`. Distinct from the
/// derived `notes.md` the same directory holds after finalize.
pub fn manual_notes_file(id: &MeetingId) -> PathBuf {
meeting_dir(id).join("manual_notes.json")
}
pub fn models_dir() -> PathBuf {
wa_root().join("models")
}
+397 -45
View File
@@ -115,6 +115,15 @@ pub struct CalendarEventDetail {
pub participants: Vec<Participant>,
}
/// Result of `cleanup_calendar_events` — `protected` is always reported
/// alongside `deleted` so the UI can show the user their meeting-linked
/// events weren't touched, not just a silent lower-than-expected count.
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct CalendarCleanupResult {
pub deleted: u32,
pub protected: u32,
}
/// On-disk shape of `summary.json` (`docs/03-data-model.md`). Drafted action
/// items here are NOT yet rows in the `action_items` table — the user
/// reviews/edits them first; `confirm_action_items` is what persists them
@@ -254,6 +263,13 @@ pub trait Store: Send + Sync {
async fn get_meeting(&self, id: &MeetingId) -> Result<Meeting, StoreError>;
async fn delete_meeting(&self, id: &MeetingId) -> Result<(), StoreError>;
async fn update_notes(&self, id: &MeetingId, markdown: &str) -> Result<(), StoreError>;
/// (Re)builds this meeting's FTS index row from the current title and
/// whatever's on disk/in the DB for transcript/notes/summary/tags (Phase
/// 8, FR-SEARCH-1). `finalize_meeting`/`update_notes`/`set_tags` already
/// call this themselves; callers that write searchable content some
/// other way (e.g. `generate_summary` sealing `summary.json` straight to
/// disk) must call it afterward so search doesn't silently miss it.
async fn reindex_fts(&self, id: &MeetingId) -> Result<(), StoreError>;
/// Set (or create) a speaker's display name; works whether or not the
/// meeting has finalized yet (T4.4, FR-SPK-2/5).
async fn rename_speaker(
@@ -298,6 +314,16 @@ pub trait Store: Send + Sync {
/// A single event with its attendees (T6.3/T6.4, FR-CAL-1/3) — backs the
/// pre-meeting context panel and the speaker-naming attendee dropdown.
async fn get_calendar_event(&self, id: &str) -> Result<CalendarEventDetail, StoreError>;
/// Deletes imported calendar events not linked to any meeting —
/// `older_than_unix: None` deletes every unlinked event ("Delete all"),
/// `Some(cutoff)` only those with `starts_at < cutoff`. An event
/// referenced by `meetings.calendar_event_id` is always kept regardless
/// of age (FR-CAL-2 pre-meeting context / FR-CAL-4 continuity would
/// break otherwise). Returns `(deleted, protected)`.
async fn cleanup_calendar_events(
&self,
older_than_unix: Option<i64>,
) -> Result<(u32, u32), StoreError>;
/// Link a meeting (current or historical) to a calendar event (T6.3/T6.6,
/// FR-CAL-2/4). Errs if either id doesn't exist. Also mirrors the event's
/// subject onto the meeting's title when it has one — linking is meant to
@@ -576,49 +602,6 @@ impl SqliteStore {
.fetch_all(&self.pool)
.await?)
}
/// (Re)builds this meeting's `meeting_fts` row from the current title and
/// whatever's on disk for transcript/notes (Phase 8, FR-SEARCH-1).
/// `meeting_fts` is a plain (non-`content=`) FTS5 table, so nothing keeps
/// it in sync automatically — called from `finalize_meeting`/
/// `update_notes` so the index can't drift from what's actually stored.
/// Deletes-then-inserts rather than `INSERT OR REPLACE`: FTS5 has no
/// unique constraint to conflict on.
async fn reindex_fts(&self, id: &MeetingId) -> Result<(), StoreError> {
let title: String = sqlx::query_scalar("SELECT title FROM meetings WHERE id = ?")
.bind(id)
.fetch_one(&self.pool)
.await?;
let transcript_text = read_artifact(&paths::meeting_dir(id).join("transcript.json"))
.and_then(|s| serde_json::from_str::<TranscriptFile>(&s).ok())
.map(|t| {
t.segments
.iter()
.map(|s| s.text.as_str())
.collect::<Vec<_>>()
.join(" ")
})
.unwrap_or_default();
let notes_text =
read_artifact(&paths::meeting_dir(id).join("notes.md")).unwrap_or_default();
sqlx::query("DELETE FROM meeting_fts WHERE meeting_id = ?")
.bind(id)
.execute(&self.pool)
.await?;
sqlx::query(
"INSERT INTO meeting_fts (meeting_id, title, transcript_text, notes_text) VALUES (?, ?, ?, ?)",
)
.bind(id)
.bind(&title)
.bind(&transcript_text)
.bind(&notes_text)
.execute(&self.pool)
.await?;
Ok(())
}
}
/// Follows a `label -> merged_into` chain to its canonical label. Capped at 8
@@ -963,6 +946,71 @@ impl Store for SqliteStore {
Ok(())
}
/// `meeting_fts` is a plain (non-`content=`) FTS5 table, so nothing keeps
/// it in sync automatically — every write path that changes title,
/// transcript, notes, summary, or tags calls this afterward so search
/// (FR-SEARCH-1) can't drift from what's actually stored. Deletes-then-
/// inserts rather than `INSERT OR REPLACE`: FTS5 has no unique
/// constraint to conflict on.
async fn reindex_fts(&self, id: &MeetingId) -> Result<(), StoreError> {
let title: String = sqlx::query_scalar("SELECT title FROM meetings WHERE id = ?")
.bind(id)
.fetch_one(&self.pool)
.await?;
let transcript_text = read_artifact(&paths::meeting_dir(id).join("transcript.json"))
.and_then(|s| serde_json::from_str::<TranscriptFile>(&s).ok())
.map(|t| {
t.segments
.iter()
.map(|s| s.text.as_str())
.collect::<Vec<_>>()
.join(" ")
})
.unwrap_or_default();
let notes_text =
read_artifact(&paths::meeting_dir(id).join("notes.md")).unwrap_or_default();
// `read_artifact` already unseals (T8.8's vault, passthrough when
// locked/plaintext), same as the transcript/notes reads above.
let summary_text = read_artifact(&paths::meeting_dir(id).join("summary.json"))
.and_then(|s| serde_json::from_str::<SummaryFile>(&s).ok())
.map(|s| {
let mut text = s.summary_md;
for decision in &s.decisions {
text.push(' ');
text.push_str(decision);
}
for item in &s.action_items {
text.push(' ');
text.push_str(&item.text);
}
text
})
.unwrap_or_default();
let tags_text = self.tags_for_meeting(id).await?.join(" ");
sqlx::query("DELETE FROM meeting_fts WHERE meeting_id = ?")
.bind(id)
.execute(&self.pool)
.await?;
sqlx::query(
"INSERT INTO meeting_fts (meeting_id, title, transcript_text, notes_text, summary_text, tags_text)
VALUES (?, ?, ?, ?, ?, ?)",
)
.bind(id)
.bind(&title)
.bind(&transcript_text)
.bind(&notes_text)
.bind(&summary_text)
.bind(&tags_text)
.execute(&self.pool)
.await?;
Ok(())
}
async fn rename_speaker(
&self,
id: &MeetingId,
@@ -1040,6 +1088,60 @@ impl Store for SqliteStore {
})
}
async fn cleanup_calendar_events(
&self,
older_than_unix: Option<i64>,
) -> Result<(u32, u32), StoreError> {
// Protected = referenced by any meeting's calendar_event_id, full
// stop — never deleted regardless of age or the "delete all" choice.
let protected: i64 = match older_than_unix {
Some(cutoff) => {
sqlx::query_scalar(
"SELECT COUNT(*) FROM calendar_events
WHERE starts_at < ?
AND id IN (SELECT calendar_event_id FROM meetings WHERE calendar_event_id IS NOT NULL)",
)
.bind(cutoff)
.fetch_one(&self.pool)
.await?
}
None => {
sqlx::query_scalar(
"SELECT COUNT(*) FROM calendar_events
WHERE id IN (SELECT calendar_event_id FROM meetings WHERE calendar_event_id IS NOT NULL)",
)
.fetch_one(&self.pool)
.await?
}
};
// calendar_event_participants cascades via its ON DELETE CASCADE FK.
let deleted = match older_than_unix {
Some(cutoff) => {
sqlx::query(
"DELETE FROM calendar_events
WHERE starts_at < ?
AND id NOT IN (SELECT calendar_event_id FROM meetings WHERE calendar_event_id IS NOT NULL)",
)
.bind(cutoff)
.execute(&self.pool)
.await?
.rows_affected()
}
None => {
sqlx::query(
"DELETE FROM calendar_events
WHERE id NOT IN (SELECT calendar_event_id FROM meetings WHERE calendar_event_id IS NOT NULL)",
)
.execute(&self.pool)
.await?
.rows_affected()
}
};
Ok((deleted as u32, protected as u32))
}
async fn attach_meeting_to_event(
&self,
meeting_id: &MeetingId,
@@ -1201,6 +1303,19 @@ impl Store for SqliteStore {
async fn import_calendar_events(&self, events: Vec<ImportedEvent>) -> Result<u32, StoreError> {
let mut imported = 0u32;
for ImportedEvent { event, attendees } in events {
// Bug fix: `raw_uid = NULL` is explicitly exempt from the
// (source, raw_uid) unique index below, so any caller that
// passes one through duplicated that event on every re-import
// forever. `parse_vevents` no longer produces one, but default
// here too — defense in depth for any other/future import path.
let raw_uid = event.raw_uid.clone().or_else(|| {
Some(crate::calendar::content_uid(
&event.subject,
&event.organizer,
event.starts_at,
event.ends_at,
))
});
sqlx::query(
"INSERT INTO calendar_events (id, source, subject, organizer, starts_at, ends_at, description, raw_uid)
VALUES (?, ?, ?, ?, ?, ?, ?, ?)
@@ -1218,14 +1333,14 @@ impl Store for SqliteStore {
.bind(event.starts_at)
.bind(event.ends_at)
.bind(&event.description)
.bind(&event.raw_uid)
.bind(&raw_uid)
.execute(&self.pool)
.await?;
// A re-import may have updated an *existing* row rather than
// inserting `event.id` — resolve the row that's actually there
// before linking attendees to it.
let row_id: String = match &event.raw_uid {
let row_id: String = match &raw_uid {
Some(raw_uid) => {
sqlx::query("SELECT id FROM calendar_events WHERE source = ? AND raw_uid = ?")
.bind(&event.source)
@@ -1308,6 +1423,7 @@ impl Store for SqliteStore {
.execute(&self.pool)
.await?;
}
self.reindex_fts(meeting_id).await?;
Ok(())
}
@@ -1846,6 +1962,84 @@ mod tests {
assert_eq!(meeting.language.as_deref(), Some("fr"));
}
/// End-to-end regression for the search bug: `meeting_fts` originally had
/// no summary/tags columns at all and nothing reindexed on those writes,
/// so search silently missed anything that wasn't in the title,
/// transcript, or notes (FR-SEARCH-1).
#[tokio::test]
async fn search_finds_hits_via_transcript_notes_summary_and_tags() {
let store = SqliteStore::connect_in_memory().await.unwrap();
let id = store
.create_meeting(NewMeeting {
title: "Weekly sync".to_string(),
calendar_event_id: None,
template_id: None,
language: None,
})
.await
.unwrap();
store
.finalize_meeting(
&id,
FinalizeMeeting {
segments: vec![TranscriptSegment {
id: 0,
start_ms: 0,
end_ms: 1000,
speaker: "S1".to_string(),
text: "let's discuss the transcriptword rollout".to_string(),
confidence: None,
interim: false,
}],
speakers: Vec::new(),
duration_secs: 60,
recorded: false,
language: Some("en".to_string()),
backend_used: Some("cpu".to_string()),
model_used: Some("small-q5_1".to_string()),
},
)
.await
.unwrap();
store
.update_notes(&id, "action: follow up on notesword")
.await
.unwrap();
store.set_tags(&id, &["tagword".to_string()]).await.unwrap();
// summary.json isn't written through the store (generate_summary
// seals it straight to disk in commands.rs), so reindex_fts must
// pick it up when explicitly told to, same as that command does.
let summary = SummaryFile {
schema: 1,
generated_at: 0,
provider: "test".to_string(),
model: "test".to_string(),
summary_md: "summaryword recap".to_string(),
decisions: Vec::new(),
action_items: Vec::new(),
};
write_artifact(
&paths::meeting_dir(&id).join("summary.json"),
serde_json::to_string(&summary).unwrap().as_bytes(),
)
.unwrap();
store.reindex_fts(&id).await.unwrap();
for (query, source) in [
("transcriptword", "transcript"),
("notesword", "notes"),
("summaryword", "summary"),
("tagword", "tags"),
] {
let hits = store.search(query).await.unwrap();
assert!(
hits.iter().any(|h| h.id == id),
"expected a hit from {source} for query {query:?}, got {hits:?}"
);
}
}
#[tokio::test]
async fn rename_meeting_updates_the_title() {
let store = SqliteStore::connect_in_memory().await.unwrap();
@@ -1960,6 +2154,164 @@ mod tests {
);
}
/// Regression for the calendar duplicate-import bug: re-importing the
/// exact same events (same `(source, raw_uid)`) must update the existing
/// rows in place, not add new ones -- whether or not the source
/// supplied a `raw_uid` at all.
#[tokio::test]
async fn reimporting_the_same_events_does_not_duplicate_rows() {
let store = SqliteStore::connect_in_memory().await.unwrap();
let events = || {
vec![
ImportedEvent {
event: CalendarEvent {
id: "ev1".to_string(),
source: "pst".to_string(),
subject: Some("Weekly 1:1".to_string()),
organizer: None,
starts_at: Some(1_000),
ends_at: Some(2_000),
description: None,
raw_uid: Some("uid-1".to_string()),
},
attendees: vec![],
},
// No raw_uid at all -- the case that used to duplicate on
// every re-import (exempt from the unique index).
ImportedEvent {
event: CalendarEvent {
id: "ev2".to_string(),
source: "pst".to_string(),
subject: Some("No UID event".to_string()),
organizer: None,
starts_at: Some(3_000),
ends_at: Some(4_000),
description: None,
raw_uid: None,
},
attendees: vec![],
},
]
};
store.import_calendar_events(events()).await.unwrap();
assert_eq!(
store.list_calendar_events(None, None).await.unwrap().len(),
2
);
// Re-import the identical batch a few times, as a user retrying an
// import would.
for _ in 0..3 {
store.import_calendar_events(events()).await.unwrap();
}
assert_eq!(
store.list_calendar_events(None, None).await.unwrap().len(),
2
);
}
/// Regression for the "14,686 calendar events" bloat report: cleanup
/// must delete unlinked old events while never touching one attached to
/// a recorded meeting, whether age-bounded or "delete all".
#[tokio::test]
async fn cleanup_calendar_events_protects_meeting_linked_rows() {
let store = SqliteStore::connect_in_memory().await.unwrap();
let event = |id: &str, starts_at: i64| ImportedEvent {
event: CalendarEvent {
id: id.to_string(),
source: "pst".to_string(),
subject: Some(id.to_string()),
organizer: None,
starts_at: Some(starts_at),
ends_at: Some(starts_at + 1_800),
description: None,
raw_uid: Some(id.to_string()),
},
attendees: vec![],
};
store
.import_calendar_events(vec![
event("old-unlinked", 1_000),
event("old-linked", 1_000),
event("recent-unlinked", 1_000_000_000),
])
.await
.unwrap();
let meeting_id = store
.create_meeting(NewMeeting {
title: "Untitled meeting".to_string(),
calendar_event_id: None,
template_id: None,
language: None,
})
.await
.unwrap();
store
.attach_meeting_to_event(&meeting_id, "old-linked")
.await
.unwrap();
// Cutoff between the two "old" timestamps and the "recent" one.
let (deleted, protected) = store.cleanup_calendar_events(Some(500_000)).await.unwrap();
assert_eq!(deleted, 1, "only old-unlinked should go");
assert_eq!(protected, 1, "old-linked is attached to a meeting");
let remaining: Vec<String> = store
.list_calendar_events(None, None)
.await
.unwrap()
.into_iter()
.map(|e| e.id)
.collect();
assert!(!remaining.contains(&"old-unlinked".to_string()));
assert!(remaining.contains(&"old-linked".to_string()));
assert!(remaining.contains(&"recent-unlinked".to_string()));
}
#[tokio::test]
async fn cleanup_calendar_events_delete_all_still_protects_linked_rows() {
let store = SqliteStore::connect_in_memory().await.unwrap();
let event = |id: &str| ImportedEvent {
event: CalendarEvent {
id: id.to_string(),
source: "pst".to_string(),
subject: Some(id.to_string()),
organizer: None,
starts_at: Some(1_000),
ends_at: Some(2_800),
description: None,
raw_uid: Some(id.to_string()),
},
attendees: vec![],
};
store
.import_calendar_events(vec![event("unlinked"), event("linked")])
.await
.unwrap();
let meeting_id = store
.create_meeting(NewMeeting {
title: "Untitled meeting".to_string(),
calendar_event_id: None,
template_id: None,
language: None,
})
.await
.unwrap();
store
.attach_meeting_to_event(&meeting_id, "linked")
.await
.unwrap();
let (deleted, protected) = store.cleanup_calendar_events(None).await.unwrap();
assert_eq!(deleted, 1);
assert_eq!(protected, 1);
assert_eq!(
store.list_calendar_events(None, None).await.unwrap().len(),
1
);
}
#[tokio::test]
async fn sync_target_crud_round_trips() {
let store = SqliteStore::connect_in_memory().await.unwrap();
+329 -39
View File
@@ -87,6 +87,17 @@ fn resolve_language(requested: Option<&str>, multilingual: bool) -> Option<Strin
}
}
/// Segments whisper.cpp itself flags as more likely silence than speech are
/// dropped rather than emitted (see `run_full`) — whisper.cpp's CLI ships
/// this same 0.6 default for `--no-speech-thold`.
#[cfg(feature = "cpu-transcription")]
const NO_SPEECH_THRESHOLD: f32 = 0.6;
#[cfg(feature = "cpu-transcription")]
fn is_likely_speech(no_speech_probability: f32) -> bool {
no_speech_probability <= NO_SPEECH_THRESHOLD
}
/// whisper.cpp-backed transcriber (CPU baseline; GPU via Cargo features).
#[cfg(feature = "cpu-transcription")]
pub struct WhisperTranscriber {
@@ -185,6 +196,15 @@ impl WhisperTranscriber {
if text.is_empty() {
continue;
}
// whisper.cpp's own `no_speech_thold` gate is a no-op (unimplemented
// upstream as of whisper-rs 0.16 / whisper.cpp v1.3.0+), so silent/
// near-silent windows still decode — and greedy short-window decode
// reliably hallucinates a short filler word ("you", "Thank you.")
// instead of emitting nothing. Drop those ourselves: matches
// whisper.cpp's own CLI default threshold for "this was silence".
if !is_likely_speech(seg.no_speech_probability()) {
continue;
}
// Whisper timestamps are centiseconds (10ms units).
let start_ms = offset_ms + seg.start_timestamp().max(0) as u64 * 10;
let end_ms = if single_segment {
@@ -297,63 +317,215 @@ pub mod onnx_models;
#[cfg(feature = "npu")]
pub use npu::OnnxTranscriber;
/// Streaming window worker (Phase 1, T1.5/T1.6): accumulates raw 16kHz-mono
/// chunks from the `audio` service into fixed-size, **non-overlapping** windows
/// and runs one `transcribe_stream` pass per window as it fills, forwarding
/// each produced segment to `on_segment` (e.g. a Tauri event emit).
const STREAM_SAMPLE_RATE: usize = 16_000;
/// Cadence/length knobs for the live streaming worker (`Streamer`). Tuned for a
/// fluid transcript that shows words ~1s after they're spoken and breaks lines
/// at natural pauses rather than on a fixed clock.
#[derive(Debug, Clone, Copy)]
pub struct StreamTuning {
/// How often the growing window is re-decoded (and the interim line
/// refreshed). Lower = more responsive, but more CPU (each decode re-runs
/// whisper over the whole in-progress window).
pub step_ms: u64,
/// Hard cap on an uncommitted window: once the current line reaches this
/// without a natural pause, it's force-committed so the window (and its
/// per-step decode cost) can't grow without bound.
pub max_window_ms: u64,
/// Don't commit a line shorter than this on a detected pause — avoids
/// chopping a brief hesitation into its own one-word line.
pub min_commit_ms: u64,
}
impl StreamTuning {
/// `low_overhead` doubles the decode step (halving CPU) at the cost of a
/// slightly less immediate transcript — matches the Settings "low overhead"
/// preset (CPU + smallest model, for battery/background use).
pub fn new(low_overhead: bool) -> Self {
Self {
step_ms: if low_overhead { 2000 } else { 1000 },
max_window_ms: 12_000,
min_commit_ms: if low_overhead { 2000 } else { 1500 },
}
}
}
impl Default for StreamTuning {
fn default() -> Self {
Self::new(false)
}
}
/// Incremental live-transcript state machine, independent of any transcription
/// engine (a `decode: &[f32] -> String` closure is injected) so its
/// commit/interim logic is unit-testable without whisper.
///
/// `transcribe_stream` takes a `SegmentSink` per the `Transcriber` trait (so a
/// future async/threaded engine can push mid-inference), but whisper.cpp's
/// `full()` call is synchronous — by the time a window's `transcribe_stream`
/// call returns, every segment it produced is already sitting in a fresh
/// per-window channel, so this drains it inline rather than needing a second
/// long-lived thread just to bridge segments out.
/// The model: keep one growing "uncommitted" window of audio. Every `step_ms`,
/// re-decode the whole window and emit it as an **interim** segment with a
/// stable id — so the current line grows in place (fluid, low-latency) instead
/// of popping in whole every few seconds. When the decoded text stops changing
/// for a step (the speaker paused, so the extra audio was silence) the line is
/// **committed** (`interim = false`, same id) and a fresh window/line begins —
/// so lines break at natural sentence pauses, not on a fixed 4s clock. A
/// `max_window_ms` backstop force-commits a pause-free monologue.
///
/// True incremental/partial-word streaming (and window overlap for continuity)
/// are out of scope for Phase 1: whisper.cpp transcribes each window from
/// scratch, so an overlapping window would re-emit the overlapped words a
/// second time with no stitching logic to merge them — a worse rough edge for
/// a live transcript than the occasional word clipped at a window boundary.
/// "Near real time" (FR-TRX-2) is met by short (~4s) windows; `interim` stays
/// `false` for every segment produced here.
/// whisper.cpp isn't a true streaming recognizer (it re-decodes from scratch),
/// so this trades CPU — the growing window is re-decoded every step — for a
/// natural-looking transcript. `audio_ctx_for_window` keeps each decode's cost
/// proportional to the window length rather than a full 30s encode, and
/// committing on pauses keeps the window short for conversational speech.
pub struct Streamer {
step_len: usize,
max_len: usize,
min_commit_len: usize,
window: Vec<f32>,
committed_offset_ms: u64,
since_last_decode: usize,
next_id: u64,
last_text: String,
}
impl Streamer {
pub fn new(tuning: StreamTuning) -> Self {
let per_ms = STREAM_SAMPLE_RATE / 1000;
Self {
step_len: tuning.step_ms as usize * per_ms,
max_len: tuning.max_window_ms as usize * per_ms,
min_commit_len: tuning.min_commit_ms as usize * per_ms,
window: Vec::new(),
committed_offset_ms: 0,
since_last_decode: 0,
next_id: 0,
last_text: String::new(),
}
}
/// Append newly captured audio; decode + emit once a step's worth has
/// accumulated. `decode(samples, offset_ms)` returns the transcript of the
/// window so far (empty for silence).
pub fn feed<D, F>(&mut self, chunk: &[f32], decode: &D, on_segment: &mut F)
where
D: Fn(&[f32], u64) -> String,
F: FnMut(TranscriptSegment),
{
self.window.extend_from_slice(chunk);
self.since_last_decode += chunk.len();
if self.since_last_decode >= self.step_len {
self.since_last_decode = 0;
self.tick(false, decode, on_segment);
}
}
/// Commit whatever's in flight — called once when capture stops so the last
/// in-progress line is finalized rather than left interim.
pub fn flush<D, F>(&mut self, decode: &D, on_segment: &mut F)
where
D: Fn(&[f32], u64) -> String,
F: FnMut(TranscriptSegment),
{
self.tick(true, decode, on_segment);
}
fn tick<D, F>(&mut self, force: bool, decode: &D, on_segment: &mut F)
where
D: Fn(&[f32], u64) -> String,
F: FnMut(TranscriptSegment),
{
if self.window.is_empty() {
return;
}
let window_ms = (self.window.len() as u64 * 1000) / STREAM_SAMPLE_RATE as u64;
let text = decode(&self.window, self.committed_offset_ms);
let over_max = self.window.len() >= self.max_len;
// Nothing recognized yet (leading/standalone silence): don't show an
// empty line, but still drop the buffer once it's grown too big so we
// aren't re-decoding a long silence every step.
if text.is_empty() {
if over_max || force {
self.reset(window_ms, false);
}
return;
}
let stable = text == self.last_text;
let commit = force || over_max || (stable && self.window.len() >= self.min_commit_len);
on_segment(TranscriptSegment {
id: self.next_id,
start_ms: self.committed_offset_ms,
end_ms: self.committed_offset_ms + window_ms,
// Provisional speaker; the post-stop diarization pass reassigns.
speaker: "S1".to_string(),
text: text.clone(),
confidence: None,
interim: !commit,
});
if commit {
self.reset(window_ms, true);
} else {
self.last_text = text;
}
}
/// Start a fresh window/line after a commit (`new_line`) or after dropping
/// leading silence (`!new_line`, which reuses the id since nothing was
/// emitted for it).
fn reset(&mut self, window_ms: u64, new_line: bool) {
self.committed_offset_ms += window_ms;
self.window.clear();
self.last_text.clear();
self.since_last_decode = 0;
if new_line {
self.next_id += 1;
}
}
}
/// Live streaming worker (FR-TRX-2): drives a `Streamer` from the `audio`
/// service's 16kHz-mono frames, decoding each growing window with `transcriber`
/// and forwarding interim-then-final segments to `on_segment` (a Tauri emit).
///
/// `transcribe_stream` is synchronous for whisper.cpp — by the time it returns,
/// every segment it produced is already in the per-call channel — so this
/// drains it inline and joins the text into the window's transcript for the
/// `Streamer` to diff/commit.
#[cfg(feature = "cpu-transcription")]
pub fn run_streaming_worker<T, F>(transcriber: &T, frame_rx: Receiver<Vec<f32>>, mut on_segment: F)
where
pub fn run_streaming_worker<T, F>(
transcriber: &T,
frame_rx: Receiver<Vec<f32>>,
tuning: StreamTuning,
mut on_segment: F,
) where
// `?Sized` so the dispatcher can hand us a `&dyn Transcriber` (whisper.cpp
// or the NPU engine, chosen at runtime) rather than a concrete type.
T: Transcriber + ?Sized,
F: FnMut(TranscriptSegment),
{
const SAMPLE_RATE: usize = 16_000;
const WINDOW_SECS: f32 = 4.0;
let window_len = (WINDOW_SECS * SAMPLE_RATE as f32) as usize;
let mut buf: Vec<f32> = Vec::new();
let mut offset_ms: u64 = 0;
let run_window = |transcriber: &T, samples: Vec<f32>, offset_ms: u64, on_segment: &mut F| {
let decode = |samples: &[f32], offset_ms: u64| -> String {
let (tx, rx) = std::sync::mpsc::channel();
let window = AudioWindow { samples, offset_ms };
let window = AudioWindow {
samples: samples.to_vec(),
offset_ms,
};
if let Err(e) = transcriber.transcribe_stream(window, tx) {
tracing::warn!("transcription window failed: {e}");
return String::new();
}
let mut parts = Vec::new();
while let Ok(segment) = rx.try_recv() {
on_segment(segment);
let t = segment.text.trim();
if !t.is_empty() {
parts.push(t.to_string());
}
}
parts.join(" ")
};
let mut streamer = Streamer::new(tuning);
while let Ok(chunk) = frame_rx.recv() {
buf.extend_from_slice(&chunk);
while buf.len() >= window_len {
let samples: Vec<f32> = buf.drain(..window_len).collect();
run_window(transcriber, samples, offset_ms, &mut on_segment);
offset_ms += (window_len as u64 * 1000) / SAMPLE_RATE as u64;
}
}
// Final partial window on stop, if there's enough audio to be worth a pass.
if buf.len() > SAMPLE_RATE / 2 {
run_window(transcriber, buf, offset_ms, &mut on_segment);
streamer.feed(&chunk, &decode, &mut on_segment);
}
streamer.flush(&decode, &mut on_segment);
}
#[cfg(test)]
@@ -361,6 +533,113 @@ where
mod tests {
use super::*;
// Collect a step's emitted segments into `out` — the pushing closure lives
// and dies inside the call, so `out` is free to read in the asserts after
// (a single long-lived `on` closure would keep `out` mutably borrowed).
fn feed_into<D: Fn(&[f32], u64) -> String>(
s: &mut Streamer,
chunk: &[f32],
decode: &D,
out: &mut Vec<TranscriptSegment>,
) {
s.feed(chunk, decode, &mut |seg| out.push(seg));
}
fn flush_into<D: Fn(&[f32], u64) -> String>(
s: &mut Streamer,
decode: &D,
out: &mut Vec<TranscriptSegment>,
) {
s.flush(decode, &mut |seg| out.push(seg));
}
#[test]
fn streamer_grows_interim_then_commits_on_a_pause() {
// Text grows for two steps, then repeats (the speaker paused, so the
// extra second was silence) → the line commits.
let tuning = StreamTuning {
step_ms: 1000,
max_window_ms: 12_000,
min_commit_ms: 1500,
};
let mut s = Streamer::new(tuning);
let script = std::cell::Cell::new(0usize);
let texts = ["hello", "hello world", "hello world"];
let decode = |_: &[f32], _off: u64| -> String {
let i = script.get();
script.set(i + 1);
texts.get(i).copied().unwrap_or("hello world").to_string()
};
let mut out: Vec<TranscriptSegment> = Vec::new();
let step = vec![0.0f32; 16_000]; // exactly one 1s step
feed_into(&mut s, &step, &decode, &mut out); // "hello" — interim
feed_into(&mut s, &step, &decode, &mut out); // "hello world" — interim
feed_into(&mut s, &step, &decode, &mut out); // stable + past min_commit → commit
assert_eq!(out.len(), 3);
assert!(out[0].interim && out[0].text == "hello");
assert!(out[1].interim && out[1].text == "hello world");
assert!(!out[2].interim && out[2].text == "hello world");
assert_eq!(out[0].id, out[2].id, "same line id until it commits");
// A new line after the commit uses a fresh id and a later offset.
let d2 = |_: &[f32], _o: u64| "next sentence".to_string();
feed_into(&mut s, &step, &d2, &mut out);
assert!(out[3].interim && out[3].text == "next sentence");
assert_ne!(out[3].id, out[2].id);
assert!(out[3].start_ms >= 3000, "starts after the 3 committed steps");
}
#[test]
fn streamer_shows_no_line_for_pure_silence() {
let mut s = Streamer::new(StreamTuning::new(false));
let decode = |_: &[f32], _o: u64| String::new();
let mut out: Vec<TranscriptSegment> = Vec::new();
let step = vec![0.0f32; 16_000];
for _ in 0..5 {
feed_into(&mut s, &step, &decode, &mut out);
}
assert!(out.is_empty(), "silence must not emit an empty line");
}
#[test]
fn streamer_flush_commits_the_in_progress_line() {
let mut s = Streamer::new(StreamTuning::new(false));
let decode = |_: &[f32], _o: u64| "partial".to_string();
let mut out: Vec<TranscriptSegment> = Vec::new();
feed_into(&mut s, &vec![0.0f32; 16_000], &decode, &mut out);
assert!(out.last().unwrap().interim, "still growing before flush");
flush_into(&mut s, &decode, &mut out);
let last = out.last().unwrap();
assert!(!last.interim && last.text == "partial", "flush finalizes it");
}
#[test]
fn streamer_force_commits_a_pause_free_monologue_at_max() {
// Text never repeats (continuous speech), so only the max-window
// backstop can commit it — otherwise the window (and decode cost) grows
// without bound.
let tuning = StreamTuning {
step_ms: 1000,
max_window_ms: 3000,
min_commit_ms: 1500,
};
let mut s = Streamer::new(tuning);
let n = std::cell::Cell::new(0usize);
let decode = |_: &[f32], _o: u64| {
let i = n.get();
n.set(i + 1);
format!("word{i}")
};
let mut out: Vec<TranscriptSegment> = Vec::new();
let step = vec![0.0f32; 16_000];
feed_into(&mut s, &step, &decode, &mut out); // 1s
feed_into(&mut s, &step, &decode, &mut out); // 2s
feed_into(&mut s, &step, &decode, &mut out); // 3s == max → force commit
assert!(!out.last().unwrap().interim);
}
#[test]
fn audio_ctx_scales_proportionally_to_window_length() {
// The actual streaming WINDOW_SECS (4.0) -> ~200 (201 after `.ceil()`
@@ -383,6 +662,17 @@ mod tests {
assert_eq!(audio_ctx_for_window(60 * 16_000), 1500); // 60s window
}
#[test]
fn is_likely_speech_drops_high_no_speech_probability_windows() {
// Silence/near-silence: whisper.cpp's own greedy hallucination case
// ("you", "Thank you.") on an otherwise-quiet window.
assert!(!is_likely_speech(0.9));
assert!(!is_likely_speech(NO_SPEECH_THRESHOLD + 0.01));
// Confident speech kept, including right at the threshold.
assert!(is_likely_speech(0.0));
assert!(is_likely_speech(NO_SPEECH_THRESHOLD));
}
/// GPU spike (opt-in): times whisper.cpp on a chosen backend against a real
/// model + wav. With `--features vulkan` and `WA_BACKEND=intel` (or nvidia/amd)
/// whisper.cpp offloads to the GPU; `WA_BACKEND=cpu` is the baseline. Run:
+5 -2
View File
@@ -40,7 +40,9 @@ const CATALOG: &[Catalog] = &[
multilingual: false,
},
Catalog {
id: "medium.en-q5_1",
// ggerganov/whisper.cpp only ships a q5_0 quantization for medium
// (q5_1 doesn't exist upstream for this size) — q5_1 here 404s.
id: "medium.en-q5_0",
label: "Medium (English, quantized) — best accuracy, slowest",
size_mb: 540,
multilingual: false,
@@ -64,7 +66,8 @@ const CATALOG: &[Catalog] = &[
multilingual: true,
},
Catalog {
id: "medium-q5_1",
// Same upstream-availability caveat as medium.en above.
id: "medium-q5_0",
label: "Medium (multilingual, quantized) — best accuracy, slowest",
size_mb: 540,
multilingual: true,
+1 -1
View File
@@ -1,7 +1,7 @@
{
"$schema": "https://schema.tauri.app/config/2",
"productName": "WhispAssist",
"version": "0.3.0",
"version": "0.4.0",
"identifier": "bet.dou.whispassist",
"build": {
"frontendDist": "../dist",
+212 -14
View File
@@ -7,17 +7,39 @@
import Settings from "./lib/views/Settings.svelte";
import ConsentNotice from "./lib/components/ConsentNotice.svelte";
import LevelMeter from "./lib/components/LevelMeter.svelte";
import ImportMeeting from "./lib/components/ImportMeeting.svelte";
import { trapFocus } from "./lib/actions/trapFocus";
import { recording } from "./lib/stores/recording.svelte";
import { settings } from "./lib/stores/settings.svelte";
import { meetings } from "./lib/stores/meetings.svelte";
import { api, type NoteTemplate } from "./lib/api";
import { calendar } from "./lib/stores/calendar.svelte";
import { api, type NoteTemplate, type CalendarEvent } from "./lib/api";
import { onMount } from "svelte";
import ThemeToggle from "./lib/components/ThemeToggle.svelte";
import { Circle, Square, Trash2, Settings as SettingsIcon, AlertTriangle } from "@lucide/svelte";
import Splitter from "./lib/components/Splitter.svelte";
import { layout, clamp } from "./lib/stores/layout.svelte";
import {
Circle,
Square,
Trash2,
FilePlus,
Settings as SettingsIcon,
AlertTriangle,
PanelLeftClose,
PanelLeftOpen,
PanelRightClose,
PanelRightOpen,
} from "@lucide/svelte";
let showSettings = $state(false);
let showConsent = $state(false);
let showImport = $state(false);
// A freshly imported meeting: refresh the list and open it.
async function onImported(id: string) {
await meetings.load();
await meetings.select(id);
}
// Vault unlock gate (T8.8): if the vault is enabled but locked at startup,
// prompt for the password so encrypted meetings are readable.
@@ -67,10 +89,48 @@
document.documentElement.dataset.theme = resolvedTheme;
});
// Auto-start recording on calendar events (opt-in, FR-CAL). No background
// work: while the app is open we arm a single one-shot timer to the next
// event's start; nothing polls (NFR-RES-1). Events already auto-started this
// session are remembered so stopping a recording doesn't re-trigger the same
// one, and the horizon caps the timer at setTimeout's safe range.
let autoRecordTimer: ReturnType<typeof setTimeout> | undefined;
const autoStarted = new Set<string>();
const AUTO_RECORD_HORIZON_MS = 24 * 60 * 60 * 1000;
function autoStartForEvent(ev: CalendarEvent) {
if (recording.state !== "idle") return;
autoStarted.add(ev.id);
meetings.deselect();
recording.start(
ev.subject ?? undefined,
settings.settings.default_record,
selectedTemplateId || undefined,
ev.id,
);
}
$effect(() => {
if (!settings.settings.auto_record_calendar || recording.state !== "idle") return;
const now = Date.now();
const next = calendar.events
.filter((e) => e.starts_at && !autoStarted.has(e.id))
.filter((e) => ((e.ends_at ?? e.starts_at) as number) * 1000 > now) // not already over
.sort((a, b) => (a.starts_at as number) - (b.starts_at as number))[0];
if (!next) return;
const delay = (next.starts_at as number) * 1000 - now;
if (delay <= 0) {
autoStartForEvent(next); // event is happening right now
return;
}
if (delay > AUTO_RECORD_HORIZON_MS) return; // too far out; re-armed on state/event change
autoRecordTimer = setTimeout(() => autoStartForEvent(next), delay);
return () => clearTimeout(autoRecordTimer);
});
onMount(() => {
recording.init();
settings.load();
meetings.init();
calendar.load(); // events power the auto-record timer above
checkVault();
api
.listNoteTemplates()
@@ -186,6 +246,14 @@
<option value={t.id}>{t.name}</option>
{/each}
</select>
<button
class="import-btn"
onclick={() => (showImport = true)}
title="Add a meeting from a file or URL"
>
<FilePlus size={13} aria-hidden="true" />
Add meeting
</button>
<button
class="record-btn"
onclick={startRecording}
@@ -217,7 +285,13 @@
<span class="rec-dot" aria-hidden="true"></span>
Recording…
</span>
<LevelMeter rms={recording.levelRms} peak={recording.levelPeak} />
<LevelMeter
rms={recording.levelRms}
peak={recording.levelPeak}
micRms={recording.levelRmsMic}
micPeak={recording.levelPeakMic}
showMic={settings.settings.microphone_enabled}
/>
{#if settings.hardware}
<span class="backend" title="Active transcription backend">{settings.hardware.active}</span>
{/if}
@@ -264,6 +338,10 @@
<Settings onClose={() => (showSettings = false)} />
{/if}
{#if showImport}
<ImportMeeting onClose={() => (showImport = false)} {onImported} />
{/if}
{#if vaultLocked}
<div
class="vault-overlay"
@@ -291,10 +369,89 @@
</div>
{/if}
<main class="panes">
<aside class="left"><MeetingsList /></aside>
<main
class="panes"
style="grid-template-columns: {layout.leftCollapsed
? 'auto'
: layout.leftWidth + 'px'} auto 1fr auto {layout.rightCollapsed
? 'auto'
: layout.rightWidth + 'px'};"
>
<aside class="side left" class:collapsed={layout.leftCollapsed}>
{#if layout.leftCollapsed}
<button
class="pane-toggle"
onclick={() => {
layout.leftCollapsed = false;
layout.persist();
}}
title="Show meetings list"
aria-label="Show meetings list"
>
<PanelLeftOpen size={16} aria-hidden="true" />
</button>
{:else}
<button
class="pane-toggle inline"
onclick={() => {
layout.leftCollapsed = true;
layout.persist();
}}
title="Hide meetings list"
aria-label="Hide meetings list"
>
<PanelLeftClose size={14} aria-hidden="true" />
</button>
<div class="side-content"><MeetingsList /></div>
{/if}
</aside>
{#if !layout.leftCollapsed}
<Splitter
label="Resize meetings list"
onResize={(d) => (layout.leftWidth = clamp(layout.leftWidth + d, 200, 480))}
onResizeEnd={() => layout.persist()}
/>
{:else}
<span></span>
{/if}
<section class="center"><TranscriptNotes /></section>
<aside class="right"><SummaryPanel /></aside>
{#if !layout.rightCollapsed}
<Splitter
label="Resize summary panel"
onResize={(d) => (layout.rightWidth = clamp(layout.rightWidth - d, 240, 560))}
onResizeEnd={() => layout.persist()}
/>
{:else}
<span></span>
{/if}
<aside class="side right" class:collapsed={layout.rightCollapsed}>
{#if layout.rightCollapsed}
<button
class="pane-toggle"
onclick={() => {
layout.rightCollapsed = false;
layout.persist();
}}
title="Show summary panel"
aria-label="Show summary panel"
>
<PanelRightOpen size={16} aria-hidden="true" />
</button>
{:else}
<button
class="pane-toggle inline"
onclick={() => {
layout.rightCollapsed = true;
layout.persist();
}}
title="Hide summary panel"
aria-label="Hide summary panel"
>
<PanelRightClose size={14} aria-hidden="true" />
</button>
<div class="side-content"><SummaryPanel /></div>
{/if}
</aside>
</main>
</div>
@@ -442,7 +599,8 @@
font-size: 0.85rem;
}
.record-btn,
.stop-btn {
.stop-btn,
.import-btn {
display: inline-flex;
align-items: center;
gap: 0.4rem;
@@ -457,9 +615,18 @@
transform 100ms ease-out;
}
.record-btn:active,
.stop-btn:active {
.stop-btn:active,
.import-btn:active {
transform: scale(0.97);
}
.import-btn {
background: var(--bg-elevated);
color: var(--fg);
border-color: var(--border);
}
.import-btn:hover {
background: var(--bg-hover);
}
.record-btn {
background: var(--danger);
color: #ffffff;
@@ -592,25 +759,56 @@
}
.panes {
display: grid;
grid-template-columns: 260px 1fr 320px;
/* grid-template-columns set inline — depends on collapse/resize state (FR-UX-1). */
flex: 1;
min-height: 0;
}
.left,
.right {
.side {
display: flex;
flex-direction: column;
min-height: 0;
border-color: var(--border);
overflow: auto;
background: var(--bg-subtle);
}
.left {
.side.left {
border-right: 1px solid var(--border);
}
.right {
.side.right {
border-left: 1px solid var(--border);
}
.side.collapsed {
align-items: center;
padding-top: 0.4rem;
}
.side-content {
flex: 1;
min-height: 0;
overflow: auto;
}
.pane-toggle {
display: flex;
align-items: center;
justify-content: center;
flex: none;
background: none;
border: none;
color: var(--muted);
cursor: pointer;
padding: 0.4rem;
border-radius: var(--radius-sm);
}
.pane-toggle:hover {
background: var(--bg-hover);
color: var(--fg);
}
.pane-toggle.inline {
align-self: flex-end;
margin: 0.3rem 0.3rem 0;
}
.center {
overflow: auto;
background: var(--bg);
min-width: 0;
}
.vault-overlay {
position: fixed;
+41 -3
View File
@@ -209,6 +209,11 @@ export interface CalendarEventDetail {
participants: Participant[];
}
export interface CalendarCleanupResult {
deleted: number;
protected: number;
}
export type SyncKind = "webdav" | "onedrive" | "dropbox" | "box";
// What the UI sees about a target — NEVER includes the secret (FR-SYNC-6).
@@ -318,6 +323,12 @@ export interface AppSettings {
retention_max_size_gb: number | null;
pst_last_path: string | null;
pst_auto_sync: boolean;
// Days of history to import (both a manual Import click and pst_auto_sync);
// null = full mailbox history.
pst_import_range_days: number | null;
/** Auto-start recording when a calendar event begins while the app is open
* (opt-in, off by default). Armed as a one-shot UI timer — nothing polls. */
auto_record_calendar: boolean;
audio_output_device: string | null;
microphone_enabled: boolean;
audio_input_device: string | null;
@@ -387,6 +398,14 @@ export const api = {
invoke<void>("set_recording_retention", { meetingId, record }),
acknowledgeRecordingConsent: () => invoke<void>("acknowledge_recording_consent"),
// Live notes (Granola-style redesign): both live-session only, err once
// the meeting is finalized — use updateNotes on the merged notes.md instead.
updateLiveNotes: (meetingId: MeetingId, markdown: string) =>
invoke<void>("update_live_notes", { meetingId, markdown }),
// anchorMs: the clicked segment's start_ms (not its id). text: "" clears it.
setSegmentNote: (meetingId: MeetingId, anchorMs: number, text: string) =>
invoke<void>("set_segment_note", { meetingId, anchorMs, text }),
appInfo: () => invoke<AppInfo>("app_info"),
openUrl: (url: string) => invoke<void>("open_url", { url }),
hardwareStatus: () => invoke<HardwareStatus>("hardware_status"),
@@ -397,16 +416,27 @@ export const api = {
downloadNpuPackage: () => invoke<void>("download_npu_package"),
downloadDirectmlPackage: () => invoke<void>("download_directml_package"),
listModels: () => invoke<ModelInfo[]>("list_models"),
// The fixed segmentation+embedding pair that speaker diarization needs
// installed before it can separate speakers (T4.7, FR-MODEL-1). Same
// ModelInfo shape as whisper models; download via `downloadModel` with the
// `diar-seg`/`diar-emb` kind, remove via the shared `removeModel`.
listDiarizationModels: () => invoke<ModelInfo[]>("list_diarization_models"),
// T8.7/FR-TRX-4: static catalog of whisper.cpp-recognized language codes
// for the Settings dropdown; "Auto-detect" is a frontend-only addition.
listWhisperLanguages: () => invoke<LanguageOption[]>("list_whisper_languages"),
downloadModel: (id: string, kind: "whisper" = "whisper") =>
downloadModel: (id: string, kind: "whisper" | "diar-seg" | "diar-emb" = "whisper") =>
invoke<void>("download_model", { args: { kind, id } }),
removeModel: (id: string) => invoke<void>("remove_model", { id }),
// `language` (T8.7): omit/undefined reuses whatever language the meeting
// already had rather than resetting it to auto.
reprocessTranscript: (meetingId: MeetingId, model: string, language?: string) =>
invoke<void>("reprocess_transcript", { meetingId, model, language }),
// Manually add a meeting from an existing recording — a local audio/video
// file path or a URL (YouTube/streaming page or direct media URL). Requires
// ffmpeg (and yt-dlp for URLs) on PATH; neither is bundled. Returns the new
// meeting's id once transcription + diarization have finished.
importMedia: (source: string, title?: string) =>
invoke<MeetingId>("import_media", { source, title }),
resumeTranscription: (meetingId: MeetingId) =>
invoke<void>("resume_transcription", { meetingId }),
listMeetings: (filter?: MeetingFilter) =>
@@ -458,7 +488,15 @@ export const api = {
invoke<void>("confirm_action_items", { meetingId, items }),
generateTags: (meetingId: MeetingId) => invoke<string[]>("generate_tags", { meetingId }),
importPst: (path: string, password?: string) => invoke<number>("import_pst", { path, password }),
// rangeDays: only import events starting within the last N days; omitted/undefined imports
// the full mailbox history (a long-lived .pst otherwise re-imports years of recurring/holiday
// entries on every launch when pst_auto_sync is on).
importPst: (path: string, password?: string, rangeDays?: number) =>
invoke<number>("import_pst", { path, password, rangeDays }),
// olderThanDays: undefined deletes every unlinked event ("Delete all").
// An event attached to a recorded meeting is always kept either way.
cleanupCalendarEvents: (olderThanDays?: number) =>
invoke<CalendarCleanupResult>("cleanup_calendar_events", { olderThanDays }),
listCalendarEvents: (from?: number, to?: number) =>
invoke<CalendarEvent[]>("list_calendar_events", { from, to }),
getCalendarEvent: (eventId: string) =>
@@ -533,7 +571,7 @@ export const events = {
onRetention: (cb: (p: { meetingId: string; record: boolean }) => void): Promise<UnlistenFn> =>
listen("recording://retention", (e) => cb(e.payload as never)),
onLevel: (
cb: (p: { meetingId: string; rms: number; peak: number }) => void,
cb: (p: { meetingId: string; rms: number; peak: number; mic: boolean }) => void,
): Promise<UnlistenFn> => listen("recording://level", (e) => cb(e.payload as never)),
onDeviceChanged: (
cb: (p: { meetingId: string; recovered: boolean; message: string }) => void,
+229
View File
@@ -0,0 +1,229 @@
<script lang="ts">
// Manually add a meeting from an existing recording (feature: "add a meeting
// + upload a video URL or audio file"). Transcoding is done by the backend
// via ffmpeg (+ yt-dlp for URLs) — both external, not bundled — so this is
// just a small form: pick a local file or paste a URL, optional title, go.
import { api, errorMessage } from "../api";
import { open } from "@tauri-apps/plugin-dialog";
import { trapFocus } from "../actions/trapFocus";
import { X, FileUp, Link as LinkIcon } from "@lucide/svelte";
let { onClose, onImported }: { onClose: () => void; onImported: (id: string) => void } = $props();
// `source` is either a local file path (set via Browse) or a URL (typed).
let source = $state("");
let title = $state("");
let busy = $state(false);
let error = $state<string | null>(null);
async function browse() {
const path = await open({
multiple: false,
filters: [
{
name: "Audio / video",
extensions: ["mp3", "m4a", "wav", "aac", "ogg", "opus", "flac", "mp4", "mkv", "mov", "webm", "avi"],
},
],
});
if (typeof path === "string") {
source = path;
error = null;
}
}
async function doImport() {
if (!source.trim() || busy) return;
busy = true;
error = null;
try {
const id = await api.importMedia(source.trim(), title.trim() || undefined);
onImported(id);
onClose();
} catch (e) {
error = errorMessage(e);
} finally {
busy = false;
}
}
</script>
<div class="overlay" role="dialog" aria-modal="true" aria-label="Add a meeting from a file or URL" tabindex="-1" use:trapFocus>
<div class="panel">
<header>
<strong>Add a meeting</strong>
<button class="close" onclick={onClose} aria-label="Close" title="Close (Esc)">
<X size={18} aria-hidden="true" />
</button>
</header>
<p class="muted">
Import an existing recording — a local audio/video file, or a link
(YouTube, a streaming page, or a direct media URL). It's transcribed and
diarized just like a live recording.
</p>
<label class="wide">
File or URL
<div class="row">
<input
class="grow"
bind:value={source}
placeholder="Paste a URL, or browse for a file…"
disabled={busy}
/>
<button onclick={browse} disabled={busy} title="Choose a local file">
<FileUp size={14} aria-hidden="true" /> Browse…
</button>
</div>
</label>
<label class="wide">
Title <em>(optional)</em>
<input bind:value={title} placeholder="Defaults to the file name" disabled={busy} />
</label>
<p class="muted small">
<LinkIcon size={12} aria-hidden="true" /> Requires <code>ffmpeg</code> installed and on your
PATH (plus <code>yt-dlp</code> for URLs). WhispAssist doesn't bundle them.
</p>
{#if error}
<p class="error">{error}</p>
{/if}
<div class="actions">
<button class="primary" onclick={doImport} disabled={!source.trim() || busy}>
{busy ? "Importing… this can take a while" : "Import"}
</button>
<button class="link" onclick={onClose} disabled={busy}>Cancel</button>
</div>
</div>
</div>
<style>
.overlay {
position: fixed;
inset: 0;
background: rgba(0, 0, 0, 0.45);
display: flex;
align-items: center;
justify-content: center;
z-index: 50;
padding: 1rem;
}
.panel {
background: var(--bg);
color: var(--fg);
border: 1px solid var(--border);
border-radius: var(--radius-lg);
padding: 1.25rem;
width: min(520px, 100%);
max-height: 90vh;
overflow: auto;
box-shadow: 0 12px 40px rgba(0, 0, 0, 0.3);
}
header {
display: flex;
align-items: center;
margin-bottom: 0.75rem;
}
header strong {
font-size: 1.05rem;
}
.close {
margin-left: auto;
background: none;
border: none;
color: var(--muted);
cursor: pointer;
padding: 0.25rem;
border-radius: var(--radius-sm);
}
.close:hover {
background: var(--bg-hover);
color: var(--fg);
}
label {
display: block;
margin: 0.75rem 0 0.25rem;
font-size: 0.85rem;
font-weight: 600;
}
label em {
font-weight: 400;
color: var(--muted);
}
input {
width: 100%;
box-sizing: border-box;
padding: 0.4rem 0.55rem;
border: 1px solid var(--border);
border-radius: var(--radius-sm);
background: var(--bg);
color: var(--fg);
font: inherit;
}
input:focus-visible {
border-color: var(--accent);
outline: none;
}
.row {
display: flex;
gap: 0.4rem;
align-items: center;
}
.row .grow {
flex: 1;
}
.row button {
display: flex;
align-items: center;
gap: 0.3rem;
white-space: nowrap;
background: var(--bg);
color: var(--fg);
border: 1px solid var(--border);
border-radius: var(--radius-sm);
padding: 0.4rem 0.6rem;
cursor: pointer;
}
.muted {
color: var(--muted);
}
.small {
font-size: 0.8rem;
display: flex;
align-items: center;
gap: 0.3rem;
}
.error {
color: var(--danger, #d33);
font-size: 0.85rem;
}
.actions {
display: flex;
align-items: center;
gap: 0.6rem;
margin-top: 1rem;
}
.actions .primary {
background: var(--accent);
color: var(--accent-fg, #fff);
border: none;
border-radius: var(--radius-sm);
padding: 0.45rem 0.9rem;
font-weight: 600;
cursor: pointer;
}
.actions .primary:disabled {
opacity: 0.6;
cursor: default;
}
.actions .link {
background: none;
border: none;
color: var(--muted);
cursor: pointer;
}
</style>
+45 -13
View File
@@ -1,25 +1,45 @@
<script lang="ts">
// Live input level meter while recording (T7.3, FR-CAP-5). A simple bar
// (rms fill + peak marker) rather than a full scrolling waveform — either
// satisfies the requirement, and this is far less UI/state to get right.
let { rms, peak }: { rms: number; peak: number } = $props();
// Live input level meter while recording (T7.3, FR-CAP-5/7). One bar with the
// system/loopback level (green) and — when the mic is enabled — the microphone
// level overlaid in the accent colour, so both sides of the call are visible
// at a glance. Each stream shows an rms fill + a peak marker.
let {
rms,
peak,
micRms = 0,
micPeak = 0,
showMic = false,
}: {
rms: number;
peak: number;
micRms?: number;
micPeak?: number;
showMic?: boolean;
} = $props();
// Perceptual loudness isn't linear; sqrt gives a meter that "looks right"
// for typical speech levels instead of sitting near-empty most of the time.
let rmsPct = $derived(Math.min(1, Math.sqrt(Math.max(0, rms))) * 100);
let peakPct = $derived(Math.min(1, Math.sqrt(Math.max(0, peak))) * 100);
const pct = (v: number) => Math.min(1, Math.sqrt(Math.max(0, v))) * 100;
let rmsPct = $derived(pct(rms));
let peakPct = $derived(pct(peak));
let micRmsPct = $derived(pct(micRms));
let micPeakPct = $derived(pct(micPeak));
</script>
<div
class="meter"
role="meter"
aria-label="Input level"
aria-valuenow={Math.round(rmsPct)}
aria-label={showMic ? "System and microphone input level" : "System input level"}
aria-valuenow={Math.round(Math.max(rmsPct, showMic ? micRmsPct : 0))}
aria-valuemin={0}
aria-valuemax={100}
>
<div class="fill" style="width: {rmsPct}%"></div>
<div class="peak" style="left: {peakPct}%"></div>
<div class="fill system" style="width: {rmsPct}%"></div>
<div class="peak system" style="left: {peakPct}%"></div>
{#if showMic}
<div class="fill mic" style="width: {micRmsPct}%"></div>
<div class="peak mic" style="left: {micPeakPct}%"></div>
{/if}
</div>
<style>
@@ -34,16 +54,28 @@
.fill {
position: absolute;
inset: 0 auto 0 0;
background: var(--success);
transition: width 60ms linear;
/* Overlap is visible because the mic layer is translucent. */
opacity: 0.7;
}
.fill.system {
background: var(--success);
}
.fill.mic {
background: var(--accent);
}
.peak {
position: absolute;
top: 0;
bottom: 0;
width: 2px;
background: var(--fg);
opacity: 0.6;
transition: left 60ms linear;
}
.peak.system {
background: var(--fg);
opacity: 0.6;
}
.peak.mic {
background: var(--accent);
}
</style>
+105
View File
@@ -0,0 +1,105 @@
<script lang="ts">
// Draggable divider between two panes (FR-UX-1). Reports a delta in
// pixels via onResize as the pointer moves; the parent owns the actual
// size state and clamping. onResizeEnd fires once per drag (and per arrow
// keypress) so the parent can persist without writing on every pixel.
let {
orientation = "vertical",
onResize,
onResizeEnd,
label,
}: {
orientation?: "vertical" | "horizontal";
onResize: (deltaPx: number) => void;
onResizeEnd?: () => void;
label: string;
} = $props();
let dragging = $state(false);
let lastPos = 0;
function posOf(e: PointerEvent): number {
return orientation === "vertical" ? e.clientX : e.clientY;
}
function onPointerDown(e: PointerEvent) {
dragging = true;
lastPos = posOf(e);
(e.currentTarget as HTMLElement).setPointerCapture(e.pointerId);
}
function onPointerMove(e: PointerEvent) {
if (!dragging) return;
const pos = posOf(e);
onResize(pos - lastPos);
lastPos = pos;
}
function onPointerUp(e: PointerEvent) {
if (!dragging) return;
dragging = false;
(e.currentTarget as HTMLElement).releasePointerCapture(e.pointerId);
onResizeEnd?.();
}
function onKeydown(e: KeyboardEvent) {
const step = e.shiftKey ? 40 : 12;
const negKey = orientation === "vertical" ? "ArrowLeft" : "ArrowUp";
const posKey = orientation === "vertical" ? "ArrowRight" : "ArrowDown";
if (e.key === negKey) onResize(-step);
else if (e.key === posKey) onResize(step);
else return;
e.preventDefault();
onResizeEnd?.();
}
</script>
<!-- WAI-ARIA "window splitter" pattern: a focusable, keyboard-operable
role="separator" is the correct/standard shape for a resize handle —
the a11y linter's generic "non-interactive element" rule doesn't know
about this pattern specifically. -->
<!-- svelte-ignore a11y_no_noninteractive_tabindex -->
<!-- svelte-ignore a11y_no_noninteractive_element_interactions -->
<div
class="splitter {orientation}"
class:dragging
role="separator"
aria-orientation={orientation}
aria-label={label}
tabindex="0"
onpointerdown={onPointerDown}
onpointermove={onPointerMove}
onpointerup={onPointerUp}
onkeydown={onKeydown}
></div>
<style>
.splitter {
flex: none;
background: transparent;
position: relative;
}
.splitter.vertical {
width: 5px;
cursor: col-resize;
}
.splitter.horizontal {
height: 5px;
cursor: row-resize;
}
/* Wider invisible hit-area than the visible line — a 5px target is too
thin to reliably grab (touch-target-size / no-precision-required). */
.splitter::after {
content: "";
position: absolute;
}
.splitter.vertical::after {
inset: 0 -4px;
}
.splitter.horizontal::after {
inset: -4px 0;
}
.splitter:hover,
.splitter:focus-visible,
.splitter.dragging {
background: var(--accent-soft);
outline: none;
}
</style>
+2 -2
View File
@@ -22,12 +22,12 @@ class CalendarStore {
}
}
async importPst(path: string, password?: string) {
async importPst(path: string, password?: string, rangeDays?: number) {
this.importing = true;
this.importError = null;
this.importProgress = null;
try {
await api.importPst(path, password);
await api.importPst(path, password, rangeDays);
await this.load();
} catch (e) {
this.importError = errorMessage(e);
+77
View File
@@ -0,0 +1,77 @@
// Resizable/hideable pane preferences (FR-UX-1) — pure client-side UI
// ephemera (not app data), so localStorage is the right place for it
// rather than a round-trip through Tauri settings.
const KEY = "wa-layout-v1";
interface LayoutPrefs {
leftWidth: number;
rightWidth: number;
leftCollapsed: boolean;
rightCollapsed: boolean;
/** Width of the transcript sub-pane vs notes, 0..1. */
transcriptFraction: number;
transcriptCollapsed: boolean;
notesCollapsed: boolean;
}
const DEFAULTS: LayoutPrefs = {
leftWidth: 260,
rightWidth: 320,
leftCollapsed: false,
rightCollapsed: false,
transcriptFraction: 0.5,
transcriptCollapsed: false,
notesCollapsed: false,
};
function load(): LayoutPrefs {
try {
const raw = localStorage.getItem(KEY);
if (!raw) return { ...DEFAULTS };
return { ...DEFAULTS, ...JSON.parse(raw) };
} catch {
return { ...DEFAULTS };
}
}
export function clamp(n: number, min: number, max: number): number {
return Math.min(max, Math.max(min, n));
}
class LayoutStore {
leftWidth = $state(DEFAULTS.leftWidth);
rightWidth = $state(DEFAULTS.rightWidth);
leftCollapsed = $state(DEFAULTS.leftCollapsed);
rightCollapsed = $state(DEFAULTS.rightCollapsed);
transcriptFraction = $state(DEFAULTS.transcriptFraction);
transcriptCollapsed = $state(DEFAULTS.transcriptCollapsed);
notesCollapsed = $state(DEFAULTS.notesCollapsed);
constructor() {
Object.assign(this, load());
}
/** Call after any mutation — explicit rather than an $effect so a batch
* of drag-resize updates doesn't schedule a write per pixel. */
persist() {
try {
localStorage.setItem(
KEY,
JSON.stringify({
leftWidth: this.leftWidth,
rightWidth: this.rightWidth,
leftCollapsed: this.leftCollapsed,
rightCollapsed: this.rightCollapsed,
transcriptFraction: this.transcriptFraction,
transcriptCollapsed: this.transcriptCollapsed,
notesCollapsed: this.notesCollapsed,
} satisfies LayoutPrefs),
);
} catch {
// localStorage unavailable (private mode etc.) — layout just won't persist.
}
}
}
export const layout = new LayoutStore();
+73 -6
View File
@@ -3,6 +3,7 @@
import { api, events, type TranscriptSegment, type MeetingId } from "../api";
import { settings } from "./settings.svelte";
import { SvelteMap } from "svelte/reactivity";
class RecordingStore {
meetingId = $state<MeetingId | null>(null);
@@ -11,11 +12,20 @@ class RecordingStore {
segments = $state<TranscriptSegment[]>([]);
/** Whether audio is being retained as .wav for the in-flight meeting (ADR-0009). */
retention = $state(false);
/** Live input level for the waveform/meter (FR-CAP-5); 0 when not recording. */
/** Live system/loopback level for the waveform/meter (FR-CAP-5); 0 when not recording. */
levelRms = $state(0);
levelPeak = $state(0);
/** Live microphone level, overlaid on the meter in a different colour
* (FR-CAP-7); stays 0 when the mic is disabled or not recording. */
levelRmsMic = $state(0);
levelPeakMic = $state(0);
/** Set while a capture-device reconnect is in progress; cleared on recovery (FR-CAP-6). */
deviceNotice = $state<string | null>(null);
/** Live notes redesign: freeform text typed in the Notes pane while recording. */
notesText = $state("");
/** anchor_ms (a segment's start_ms) -> note text, for moments annotated this recording. */
segmentNotes = new SvelteMap<number, string>();
private notesSaveTimer: ReturnType<typeof setTimeout> | null = null;
async init() {
await events.onRecordingState((p) => {
@@ -29,6 +39,8 @@ class RecordingStore {
if (ended) {
this.levelRms = 0;
this.levelPeak = 0;
this.levelRmsMic = 0;
this.levelPeakMic = 0;
}
});
await events.onRetention((p) => {
@@ -40,43 +52,63 @@ class RecordingStore {
if (i >= 0) this.segments[i] = segment;
else this.segments.push(segment);
});
await events.onLevel(({ rms, peak }) => {
this.levelRms = rms;
this.levelPeak = peak;
await events.onLevel(({ rms, peak, mic }) => {
if (mic) {
this.levelRmsMic = rms;
this.levelPeakMic = peak;
} else {
this.levelRms = rms;
this.levelPeak = peak;
}
});
await events.onDeviceChanged(({ recovered, message }) => {
this.deviceNotice = recovered ? null : message;
});
}
async start(title?: string, record = false, templateId?: string) {
async start(title?: string, record = false, templateId?: string, calendarEventId?: string) {
this.segments = [];
this.retention = record;
this.deviceNotice = null;
this.notesText = "";
this.segmentNotes.clear();
// T8.7/FR-TRX-4: whatever language is currently configured in Settings
// becomes this meeting's requested language, persisted on its record.
const language = settings.settings.whisper_language ?? undefined;
this.meetingId = await api.startRecording(title, undefined, record, templateId, language);
this.meetingId = await api.startRecording(title, calendarEventId, record, templateId, language);
this.state = "recording";
}
async stop() {
// The debounced save below can lag up to 500ms behind typing — flush
// whatever's pending first so stop_recording's merge sees the latest text.
await this.flushNotes();
if (this.meetingId) await api.stopRecording(this.meetingId);
this.state = "idle";
this.levelRms = 0;
this.levelPeak = 0;
this.levelRmsMic = 0;
this.levelPeakMic = 0;
this.deviceNotice = null;
}
/** Abandon an accidental recording: stop capture and delete it entirely. */
async cancel() {
if (this.notesSaveTimer) {
clearTimeout(this.notesSaveTimer);
this.notesSaveTimer = null;
}
if (this.meetingId) await api.cancelRecording(this.meetingId);
this.meetingId = null;
this.segments = [];
this.state = "idle";
this.levelRms = 0;
this.levelPeak = 0;
this.levelRmsMic = 0;
this.levelPeakMic = 0;
this.deviceNotice = null;
this.notesText = "";
this.segmentNotes.clear();
}
/** Toggle audio retention mid-meeting (FR-REC-1); caller must have gated consent already. */
@@ -85,6 +117,41 @@ class RecordingStore {
await api.setRecordingRetention(this.meetingId, record);
this.retention = record;
}
/**
* Live notes redesign: freeform text typed in the Notes pane while
* recording. Debounced 500ms (same cadence as the post-finalize editor's
* `scheduleSave`) so every keystroke doesn't round-trip to the backend.
*/
setNotesText(text: string) {
this.notesText = text;
if (!this.meetingId) return;
const meetingId = this.meetingId;
if (this.notesSaveTimer) clearTimeout(this.notesSaveTimer);
this.notesSaveTimer = setTimeout(() => {
this.notesSaveTimer = null;
api.updateLiveNotes(meetingId, text).catch(() => {});
}, 500);
}
private async flushNotes() {
if (this.notesSaveTimer) {
clearTimeout(this.notesSaveTimer);
this.notesSaveTimer = null;
}
if (this.meetingId) {
await api.updateLiveNotes(this.meetingId, this.notesText).catch(() => {});
}
}
/** Attach (or clear, with `text: ""`) a note to a clicked transcript
* segment's moment (the "click a transcript line, add a note" feature). */
async setSegmentNote(anchorMs: number, text: string) {
if (!this.meetingId) return;
if (text.trim() === "") this.segmentNotes.delete(anchorMs);
else this.segmentNotes.set(anchorMs, text);
await api.setSegmentNote(this.meetingId, anchorMs, text);
}
}
export const recording = new RecordingStore();
+30
View File
@@ -43,6 +43,8 @@ const DEFAULT_SETTINGS: AppSettings = {
retention_max_size_gb: null,
pst_last_path: null,
pst_auto_sync: false,
pst_import_range_days: null, // full mailbox history by default
auto_record_calendar: false, // don't auto-start on calendar events by default
audio_output_device: null, // system default render device (FR-CAP-1)
microphone_enabled: true, // capture the user's mic into the transcript (FR-CAP-7)
audio_input_device: null, // system default capture device
@@ -60,6 +62,10 @@ class SettingsStore {
// Hardware + model management (Phase 3, T3.6/T3.7).
hardware = $state<HardwareStatus | null>(null);
models = $state<ModelInfo[]>([]);
// Speaker-diarization models (segmentation + embedding). Both must be
// installed before recordings separate speakers instead of labelling
// everything "S1" (T4.7, FR-MODEL-1).
diarizationModels = $state<ModelInfo[]>([]);
// Transcription language catalog for the Settings dropdown (T8.7, FR-TRX-4).
languages = $state<LanguageOption[]>([]);
audioDevices = $state<AudioDeviceInfo[]>([]);
@@ -100,6 +106,7 @@ class SettingsStore {
await this.loadAudioDevices();
await this.loadInputDevices();
await this.loadModels();
await this.loadDiarizationModels();
await this.loadLanguages();
await this.loadPrivacy();
await this.loadLlmStatus();
@@ -198,6 +205,29 @@ class SettingsStore {
}
}
async loadDiarizationModels() {
try {
this.diarizationModels = await api.listDiarizationModels();
} catch {
this.diarizationModels = [];
}
}
/** Download one diarization model. The two known ids map to the backend's
* `diar-seg`/`diar-emb` kinds; `removeModel` needs no kind (it disambiguates
* by catalog membership). */
async downloadDiarizationModel(id: string) {
this.clearProgress(id);
await api.downloadModel(id, id.startsWith("seg") ? "diar-seg" : "diar-emb");
this.clearProgress(id);
await this.loadDiarizationModels();
}
async removeDiarizationModel(id: string) {
await api.removeModel(id);
await this.loadDiarizationModels();
}
async loadLanguages() {
try {
this.languages = await api.listWhisperLanguages();
+334 -28
View File
@@ -32,6 +32,8 @@
Eye,
EyeOff,
Globe,
Download,
Trash2,
} from "@lucide/svelte";
import {
OLLAMA_OPTIONS,
@@ -295,15 +297,55 @@
const path = await open({ filters: [{ name: "Outlook data file", extensions: ["pst"] }] });
if (typeof path === "string") pstPath = path;
}
// "all" imports the full mailbox history; otherwise days of history from
// now. Persisted immediately on change so pst_auto_sync's startup
// re-import (lib.rs) applies the same range as the last manual import.
let importRange = $state<string>(
settings.settings.pst_import_range_days == null
? "all"
: String(settings.settings.pst_import_range_days),
);
function onImportRangeChange(e: Event) {
importRange = (e.target as HTMLSelectElement).value;
settings.patch({
pst_import_range_days: importRange === "all" ? null : Number(importRange),
});
}
async function doImportPst() {
if (!pstPath) return;
await calendar.importPst(pstPath, pstPassword || undefined);
const rangeDays = importRange === "all" ? undefined : Number(importRange);
await calendar.importPst(pstPath, pstPassword || undefined, rangeDays);
pstPassword = "";
if (!calendar.importError) await settings.patch({ pst_last_path: pstPath });
}
function onToggleAutoSync(e: Event) {
settings.patch({ pst_auto_sync: (e.target as HTMLInputElement).checked });
}
// ---- Calendar cleanup (dedup/bloat fix follow-up) ----
let cleanupThreshold = $state<string>("all");
let cleanupRunning = $state(false);
let cleanupResult = $state<{ deleted: number; protected: number } | null>(null);
let cleanupError = $state<string | null>(null);
async function runCleanup() {
const isDeleteAll = cleanupThreshold === "all";
const olderThanDays = isDeleteAll ? undefined : Number(cleanupThreshold);
const warning = isDeleteAll
? "Delete ALL imported calendar events? Events already attached to a recorded meeting are always kept."
: `Delete calendar events older than ${cleanupThreshold} days? Events already attached to a recorded meeting are always kept.`;
if (!confirm(warning)) return;
cleanupRunning = true;
cleanupError = null;
cleanupResult = null;
try {
cleanupResult = await api.cleanupCalendarEvents(olderThanDays);
await calendar.load();
} catch (e) {
cleanupError = errorMessage(e);
} finally {
cleanupRunning = false;
}
}
function formatEventDate(unixSecs: number | null): string {
if (!unixSecs) return "";
return new Date(unixSecs * 1000).toLocaleString(undefined, {
@@ -621,6 +663,22 @@
: "not yet acknowledged"}.
</p>
<label class="row">
<input
type="checkbox"
checked={settings.settings.auto_record_calendar}
onchange={(e) =>
settings.patch({ auto_record_calendar: (e.target as HTMLInputElement).checked })}
/>
<span>Auto-start recording when a calendar event begins</span>
</label>
<p class="muted">
Only while WhispAssist is open. When an imported calendar event's start time arrives, a
recording begins automatically (using your default retention setting above). Nothing runs
in the background — the timer is armed only while the app is running. Import events under
Settings → Calendar.
</p>
{#if showConsent}
<ConsentNotice onAccept={acceptConsent} onCancel={() => (showConsent = false)} />
{/if}
@@ -775,28 +833,124 @@
<h4>Models</h4>
<ul class="models">
{#each settings.models as m (m.id)}
<li>
<span class="name">{m.label}</span>
<span class="muted">{fmtBytes(m.size_mb)}</span>
{#if m.active}<span class="badge">active</span>{/if}
{#if m.id in settings.downloadProgress}
<span class="muted"
>downloading… {progressPct(m.id) !== null ? `${progressPct(m.id)}%` : ""}</span
>
{:else if m.installed}
<button
class="link"
disabled={m.active}
onclick={() => settings.setActiveModel(m.id)}>Use</button
>
<button
class="link danger"
disabled={m.active}
onclick={() => settings.removeModel(m.id)}>Remove</button
>
{:else}
<button class="link" onclick={() => settings.downloadModel(m.id)}>Download</button>
{/if}
{@const downloading = m.id in settings.downloadProgress}
{@const pct = progressPct(m.id)}
<li class="model" class:active={m.active}>
<div class="model-info">
<span class="name">{m.label}</span>
<span class="meta">
{fmtBytes(m.size_mb)}
{#if m.active}
<span class="badge ok"><Check size={11} aria-hidden="true" /> Active</span>
{:else if m.installed}
<span class="badge">Installed</span>
{/if}
</span>
{#if downloading}
<div
class="progress"
role="progressbar"
aria-label="Downloading {m.label}"
aria-valuenow={pct ?? undefined}
aria-valuemin={0}
aria-valuemax={100}
>
<div
class="progress-fill"
class:indeterminate={pct === null}
style={pct !== null ? `width: ${pct}%` : ""}
></div>
</div>
<span class="muted progress-label"
>Downloading… {pct !== null ? `${pct}%` : ""}</span
>
{/if}
</div>
<div class="model-actions">
{#if downloading}
<!-- no actions while a download is in flight -->
{:else if m.installed}
<button
class="icon-btn"
disabled={m.active}
onclick={() => settings.setActiveModel(m.id)}
>
<Check size={13} aria-hidden="true" /> Use
</button>
<button
class="icon-btn danger"
disabled={m.active}
title={m.active ? "Can't remove the active model" : "Remove"}
onclick={() => settings.removeModel(m.id)}
>
<Trash2 size={13} aria-hidden="true" /> Remove
</button>
{:else}
<button class="icon-btn" onclick={() => settings.downloadModel(m.id)}>
<Download size={13} aria-hidden="true" /> Download
</button>
{/if}
</div>
</li>
{/each}
</ul>
<h4>Speaker diarization</h4>
<p class="muted">
Install both models to have finished recordings separated by speaker (Speaker 1, Speaker
2, …) instead of one running transcript. Runs fully offline, once per meeting after it
stops. Without them, every line is attributed to a single speaker.
</p>
<ul class="models">
{#each settings.diarizationModels as m (m.id)}
{@const downloading = m.id in settings.downloadProgress}
{@const pct = progressPct(m.id)}
<li class="model" class:active={m.installed}>
<div class="model-info">
<span class="name">{m.label}</span>
<span class="meta">
{fmtBytes(m.size_mb)}
{#if m.installed}
<span class="badge ok"><Check size={11} aria-hidden="true" /> Installed</span>
{/if}
</span>
{#if downloading}
<div
class="progress"
role="progressbar"
aria-label="Downloading {m.label}"
aria-valuenow={pct ?? undefined}
aria-valuemin={0}
aria-valuemax={100}
>
<div
class="progress-fill"
class:indeterminate={pct === null}
style={pct !== null ? `width: ${pct}%` : ""}
></div>
</div>
<span class="muted progress-label"
>Downloading… {pct !== null ? `${pct}%` : ""}</span
>
{/if}
</div>
<div class="model-actions">
{#if downloading}
<!-- no actions while a download is in flight -->
{:else if m.installed}
<button
class="icon-btn danger"
title="Remove"
onclick={() => settings.removeDiarizationModel(m.id)}
>
<Trash2 size={13} aria-hidden="true" /> Remove
</button>
{:else}
<button class="icon-btn" onclick={() => settings.downloadDiarizationModel(m.id)}>
<Download size={13} aria-hidden="true" /> Download
</button>
{/if}
</div>
</li>
{/each}
</ul>
@@ -830,9 +984,8 @@
</select>
</label>
<p class="muted" id="language-disabled-hint">
{activeModel
? `"${activeModel.label}" is English-only.`
: "No model selected."} Switch to a multilingual model above to choose a language.
{activeModel ? `"${activeModel.label}" is English-only.` : "No model selected."} Switch to
a multilingual model above to choose a language.
</p>
{/if}
</section>
@@ -889,7 +1042,22 @@
>Password <em>(rarely needed)</em>
<input type="password" bind:value={pstPassword} placeholder="optional" />
</label>
<label
>Import range
<select value={importRange} onchange={onImportRangeChange}>
<option value="30">Last 30 days</option>
<option value="90">Last 90 days</option>
<option value="180">Last 6 months</option>
<option value="365">Last 1 year</option>
<option value="all">All time</option>
</select>
</label>
</div>
<p class="muted small">
A long-lived mailbox can hold years of recurring/holiday entries — narrowing the range
keeps the imported calendar to what's actually relevant. Applies to both this Import
button and automatic re-import on launch.
</p>
<div class="actions">
<button class="primary" onclick={doImportPst} disabled={!pstPath || calendar.importing}>
{calendar.importing ? "Importing…" : "Import"}
@@ -919,6 +1087,39 @@
</p>
{/if}
<h4>Clean up</h4>
<p class="muted small">
Events already attached to a recorded meeting are always kept, no matter which option
below is picked.
</p>
<div class="actions">
<select bind:value={cleanupThreshold} disabled={cleanupRunning}>
<option value="30">Older than 30 days</option>
<option value="90">Older than 90 days</option>
<option value="180">Older than 6 months</option>
<option value="365">Older than 1 year</option>
<option value="all">Delete all</option>
</select>
<button
class="icon-btn danger"
onclick={runCleanup}
disabled={cleanupRunning || calendar.events.length === 0}
>
<Trash2 size={13} aria-hidden="true" />
{cleanupRunning ? "Cleaning up…" : "Clean up calendar"}
</button>
</div>
{#if cleanupResult}
<p class="muted small">
Deleted {cleanupResult.deleted}{cleanupResult.protected
? `, kept ${cleanupResult.protected} (linked to a meeting)`
: ""}.
</p>
{/if}
{#if cleanupError}
<p class="error">Clean up failed: {cleanupError}</p>
{/if}
<h4>Imported events</h4>
{#if calendar.events.length === 0}
<p class="muted">No events imported yet.</p>
@@ -1888,7 +2089,6 @@
padding: 0;
margin: 0.3rem 0;
}
ul.models li,
ul.events li {
display: flex;
align-items: center;
@@ -1911,9 +2111,115 @@
input.grow {
flex: 1;
}
ul.models .name {
/* Model download list (T3.7, FR-MODEL-1) — a card row per model, not a
bare text row: clearer active/installed state, and a real progress bar
for in-flight downloads instead of a lone percentage number. */
ul.models li.model {
display: flex;
align-items: center;
justify-content: space-between;
gap: 0.75rem;
padding: 0.55rem 0.7rem;
margin-bottom: 0.4rem;
border: 1px solid var(--border);
border-left: 2px solid transparent;
border-radius: var(--radius-md, 8px);
font-size: 0.85rem;
}
ul.models li.model.active {
background: var(--accent-soft);
border-left-color: var(--accent);
}
.model-info {
display: flex;
flex-direction: column;
gap: 0.25rem;
min-width: 0;
flex: 1;
}
.model-info .name {
font-weight: 600;
}
.model-info .meta {
display: flex;
align-items: center;
gap: 0.4rem;
color: var(--muted);
font-size: 0.78rem;
}
.model-actions {
display: flex;
align-items: center;
gap: 0.4rem;
flex: none;
}
.icon-btn {
display: flex;
align-items: center;
gap: 0.3rem;
background: none;
border: 1px solid var(--border);
border-radius: var(--radius-sm, 5px);
padding: 0.3rem 0.55rem;
font-size: 0.78rem;
color: var(--fg);
cursor: pointer;
transition: background-color 120ms ease-out;
}
.icon-btn:hover:not(:disabled) {
background: var(--bg-hover);
}
.icon-btn:disabled {
opacity: 0.5;
cursor: default;
}
.icon-btn.danger {
color: var(--danger);
border-color: var(--danger);
}
.icon-btn.danger:hover:not(:disabled) {
background: color-mix(in srgb, var(--danger) 12%, transparent);
}
/* Real progress bar — a lone "NN%" string doesn't read as motion/progress
at a glance the way a filling bar does (ux: loading-states). */
.progress {
height: 6px;
border-radius: var(--radius-full, 999px);
background: var(--border);
overflow: hidden;
}
.progress-fill {
height: 100%;
background: var(--accent);
border-radius: inherit;
transition: width 200ms ease-out;
}
/* Unknown total (no content-length yet) — an animated indeterminate sweep
instead of a bar stuck at 0%, so it still reads as "in progress". */
.progress-fill.indeterminate {
width: 40%;
animation: progress-sweep 1.1s ease-in-out infinite;
}
@keyframes progress-sweep {
0% {
transform: translateX(-100%);
}
100% {
transform: translateX(250%);
}
}
@media (prefers-reduced-motion: reduce) {
.progress-fill.indeterminate {
animation: none;
width: 100%;
opacity: 0.5;
}
}
.progress-label {
font-size: 0.75rem;
}
ul.targets li {
display: flex;
align-items: center;
+30 -10
View File
@@ -473,13 +473,14 @@
{#if meetings.selected && settings.settings.sync_enabled}
<h3><UploadCloud size={14} aria-hidden="true" /> Sync</h3>
<button
class="upload-now"
class="primary upload-now"
disabled={meetings.uploading}
onclick={() => {
const m = meetings.selected;
if (m) meetings.uploadNow(m.id);
}}
>
<UploadCloud size={14} aria-hidden="true" />
{meetings.uploading ? "Uploading…" : "Upload now"}
</button>
{#if meetings.syncJobs.length}
@@ -733,7 +734,8 @@
</li>
{/each}
</ul>
<button onclick={saveActionItems} disabled={savingItems}>
<button class="primary" onclick={saveActionItems} disabled={savingItems}>
<Check size={14} aria-hidden="true" />
{savingItems ? "Saving…" : "Save action items"}
</button>
{/if}
@@ -868,10 +870,6 @@
gap: 0.5rem;
margin: -0.3rem 0 0.6rem;
}
.provider-select select {
font-size: 0.78rem;
padding: 0.25rem 0.4rem;
}
.provider-badge {
display: inline-flex;
align-items: center;
@@ -899,9 +897,34 @@
font-weight: 600;
cursor: pointer;
}
button.primary:hover {
button.primary:hover:not(:disabled) {
background: var(--accent-hover);
}
button.primary:disabled {
opacity: 0.5;
cursor: default;
}
.upload-now {
align-self: start;
}
.provider-select select {
display: inline-flex;
align-items: center;
border: 1px solid var(--border);
border-radius: var(--radius-sm);
background: var(--bg);
color: var(--fg);
font-size: 0.78rem;
padding: 0.3rem 0.5rem;
cursor: pointer;
}
.provider-select select:hover:not(:disabled) {
background: var(--bg-hover);
}
.provider-select select:disabled {
opacity: 0.6;
cursor: default;
}
.muted {
color: var(--muted);
font-size: 0.85rem;
@@ -1052,9 +1075,6 @@
font-size: 0.8rem;
padding: 0;
}
.upload-now {
align-self: start;
}
.jobs {
list-style: none;
padding: 0;
+403 -115
View File
@@ -8,6 +8,8 @@
import { api, type SpeakerInfo } from "../api";
import { renderMarkdown } from "../markdown";
import { save, open } from "@tauri-apps/plugin-dialog";
import { layout, clamp } from "../stores/layout.svelte";
import Splitter from "../components/Splitter.svelte";
import {
Bold,
Italic,
@@ -23,6 +25,10 @@
NotebookPen,
Eye,
Pencil,
PanelLeftClose,
PanelLeftOpen,
PanelRightClose,
PanelRightOpen,
} from "@lucide/svelte";
function speakerName(label: string, speakers: SpeakerInfo[] = []): string {
@@ -40,11 +46,50 @@
let notesText = $state("");
// Notes is a single pane: raw markdown ("Editor") or the rendered result
// ("Preview"), toggled by one button whose label flips to the other mode.
let notesPreview = $state(false);
// Defaults to Preview: the merged notes.md is now a properly sectioned
// document (## Notes / ## Transcript), not worth reading as raw markdown
// by default the way the old single-speaker-blob output arguably was.
let notesPreview = $state(true);
let editorEl: HTMLTextAreaElement | undefined = $state();
let saveTimer: ReturnType<typeof setTimeout> | undefined;
let loadedForId: string | null = null;
// Transcript/notes split (FR-UX-1): resizable (drag the Splitter) and
// each side independently hideable, shared across the finalized-meeting
// and live-recording views via the layout store. `splitWidth` tracks the
// container's current pixel width (bound below) so a drag delta in
// pixels can be converted to a fraction of the available space.
let splitWidth = $state(600);
function splitColumns(): string {
if (layout.transcriptCollapsed) return "auto auto 1fr";
if (layout.notesCollapsed) return "1fr auto auto";
return `${layout.transcriptFraction * 100}% auto ${(1 - layout.transcriptFraction) * 100}%`;
}
function onSplitResize(deltaPx: number) {
if (splitWidth <= 0) return;
layout.transcriptFraction = clamp(layout.transcriptFraction + deltaPx / splitWidth, 0.2, 0.8);
}
function toggleTranscript() {
const wouldCollapse = !layout.transcriptCollapsed;
if (wouldCollapse && layout.notesCollapsed) return; // never hide both
layout.transcriptCollapsed = wouldCollapse;
layout.persist();
}
function toggleNotes() {
const wouldCollapse = !layout.notesCollapsed;
if (wouldCollapse && layout.transcriptCollapsed) return;
layout.notesCollapsed = wouldCollapse;
layout.persist();
}
// Live-recording transcript: which segment (by start_ms, the note anchor —
// see recording.svelte.ts) is showing its note-entry field, if any.
let selectedSegmentMs = $state<number | null>(null);
$effect(() => {
void recording.meetingId; // dependency: reset the open note field for a new recording
selectedSegmentMs = null;
});
// Sync the editor buffer whenever a different meeting is selected.
$effect(() => {
const m = meetings.selected;
@@ -163,131 +208,287 @@
onchange={onTitleChange}
aria-label="Meeting title"
/>
<div class="split">
<div class="pane transcript">
<h4>
<MessageSquareText size={14} aria-hidden="true" /> Transcript
<span class="badge lang" title="Transcription language">{languageLabel(m.language)}</span
<div
class="split"
style="grid-template-columns: {splitColumns()};"
bind:clientWidth={splitWidth}
>
<div class="pane transcript" class:collapsed={layout.transcriptCollapsed}>
{#if layout.transcriptCollapsed}
<button
class="pane-toggle"
onclick={toggleTranscript}
title="Show transcript"
aria-label="Show transcript"
>
</h4>
{#if m.recorded && settings.models.some((mo) => mo.installed)}
{@const reprocessModelInfo = settings.models.find((mo) => mo.id === reprocessModel)}
<div class="reprocess">
<select bind:value={reprocessModel}>
<option value="">Re-transcribe with…</option>
{#each settings.models.filter((mo) => mo.installed) as mo (mo.id)}
<option value={mo.id}>{mo.label}</option>
{/each}
</select>
{#if reprocessModelInfo?.multilingual}
<select bind:value={reprocessLanguage} aria-label="Reprocess language">
<option value="">Keep current language</option>
<option value="auto">Auto-detect</option>
{#each settings.languages as l (l.code)}
<option value={l.code}>{l.label}</option>
<PanelLeftOpen size={16} aria-hidden="true" />
</button>
{:else}
<h4>
<MessageSquareText size={14} aria-hidden="true" /> Transcript
<span class="badge lang" title="Transcription language"
>{languageLabel(m.language)}</span
>
<button
class="pane-toggle inline"
onclick={toggleTranscript}
title="Hide transcript"
aria-label="Hide transcript"
>
<PanelLeftClose size={13} aria-hidden="true" />
</button>
</h4>
{#if m.recorded && settings.models.some((mo) => mo.installed)}
{@const reprocessModelInfo = settings.models.find((mo) => mo.id === reprocessModel)}
<div class="reprocess">
<select bind:value={reprocessModel}>
<option value="">Re-transcribe with…</option>
{#each settings.models.filter((mo) => mo.installed) as mo (mo.id)}
<option value={mo.id}>{mo.label}</option>
{/each}
</select>
{/if}
<button disabled={!reprocessModel || reprocessing} onclick={reprocess}>
<RefreshCw size={13} aria-hidden="true" class={reprocessing ? "spin" : ""} />
{reprocessing ? "Re-transcribing…" : "Go"}
{#if reprocessModelInfo?.multilingual}
<select bind:value={reprocessLanguage} aria-label="Reprocess language">
<option value="">Keep current language</option>
<option value="auto">Auto-detect</option>
{#each settings.languages as l (l.code)}
<option value={l.code}>{l.label}</option>
{/each}
</select>
{/if}
<button disabled={!reprocessModel || reprocessing} onclick={reprocess}>
<RefreshCw size={13} aria-hidden="true" class={reprocessing ? "spin" : ""} />
{reprocessing ? "Re-transcribing…" : "Go"}
</button>
</div>
{/if}
{#if m.segments.length === 0}
<p class="muted">No transcript for this meeting.</p>
{:else}
{#each m.segments as s (s.id)}
<p><strong>{speakerName(s.speaker, m.speakers)}:</strong> {s.text}</p>
{/each}
{/if}
{/if}
</div>
{#if !layout.transcriptCollapsed && !layout.notesCollapsed}
<Splitter
label="Resize transcript and notes"
onResize={onSplitResize}
onResizeEnd={() => layout.persist()}
/>
{:else}
<span></span>
{/if}
<div class="pane notes" class:collapsed={layout.notesCollapsed}>
{#if layout.notesCollapsed}
<button
class="pane-toggle"
onclick={toggleNotes}
title="Show notes"
aria-label="Show notes"
>
<PanelRightOpen size={16} aria-hidden="true" />
</button>
{:else}
<h4>
<NotebookPen size={14} aria-hidden="true" /> Notes
<button
class="pane-toggle inline"
onclick={toggleNotes}
title="Hide notes"
aria-label="Hide notes"
>
<PanelRightClose size={13} aria-hidden="true" />
</button>
</h4>
<div class="toolbar" role="toolbar" aria-label="Notes formatting">
<button onclick={() => wrapSelection("**")} title="Bold" aria-label="Bold">
<Bold size={14} aria-hidden="true" />
</button>
<button onclick={() => wrapSelection("_")} title="Italic" aria-label="Italic">
<Italic size={14} aria-hidden="true" />
</button>
<button onclick={() => insertLinePrefix("# ")} title="Heading 1" aria-label="Heading 1">
<Heading1 size={14} aria-hidden="true" />
</button>
<button
onclick={() => insertLinePrefix("## ")}
title="Heading 2"
aria-label="Heading 2"
>
<Heading2 size={14} aria-hidden="true" />
</button>
<button
onclick={() => insertLinePrefix("- ")}
title="Bullet list"
aria-label="Bullet list"
>
<List size={14} aria-hidden="true" />
</button>
<button
onclick={() => insertLinePrefix("- [ ] ")}
title="Checkbox"
aria-label="Checkbox list item"
>
<ListChecks size={14} aria-hidden="true" />
</button>
<button
class="toggle"
onclick={() => (notesPreview = !notesPreview)}
title={notesPreview ? "Edit the raw markdown" : "Render the markdown"}
aria-pressed={notesPreview}
>
{#if notesPreview}
<Pencil size={14} aria-hidden="true" />
Editor
{:else}
<Eye size={14} aria-hidden="true" />
Preview
{/if}
</button>
<span class="spacer"></span>
<button onclick={exportMd} title="Export notes as .md">
<FileText size={13} aria-hidden="true" />
.md
</button>
<button onclick={exportPdf} title="Export notes as .pdf">
<FileDown size={13} aria-hidden="true" />
PDF
</button>
<button onclick={exportDocx} title="Export notes as .docx">
<FileDown size={13} aria-hidden="true" />
Word
</button>
<button onclick={exportBundle} title="Export audio + transcript + notes to a folder">
<FolderOutput size={13} aria-hidden="true" />
Bundle
</button>
</div>
{/if}
{#if m.segments.length === 0}
<p class="muted">No transcript for this meeting.</p>
{:else}
{#each m.segments as s (s.id)}
<p><strong>{speakerName(s.speaker, m.speakers)}:</strong> {s.text}</p>
{/each}
{/if}
</div>
<div class="pane notes">
<h4><NotebookPen size={14} aria-hidden="true" /> Notes</h4>
<div class="toolbar" role="toolbar" aria-label="Notes formatting">
<button onclick={() => wrapSelection("**")} title="Bold" aria-label="Bold">
<Bold size={14} aria-hidden="true" />
</button>
<button onclick={() => wrapSelection("_")} title="Italic" aria-label="Italic">
<Italic size={14} aria-hidden="true" />
</button>
<button onclick={() => insertLinePrefix("# ")} title="Heading 1" aria-label="Heading 1">
<Heading1 size={14} aria-hidden="true" />
</button>
<button onclick={() => insertLinePrefix("## ")} title="Heading 2" aria-label="Heading 2">
<Heading2 size={14} aria-hidden="true" />
</button>
<button
onclick={() => insertLinePrefix("- ")}
title="Bullet list"
aria-label="Bullet list"
>
<List size={14} aria-hidden="true" />
</button>
<button
onclick={() => insertLinePrefix("- [ ] ")}
title="Checkbox"
aria-label="Checkbox list item"
>
<ListChecks size={14} aria-hidden="true" />
</button>
<button
class="toggle"
onclick={() => (notesPreview = !notesPreview)}
title={notesPreview ? "Edit the raw markdown" : "Render the markdown"}
aria-pressed={notesPreview}
>
<div class="editor-preview">
{#if notesPreview}
<Pencil size={14} aria-hidden="true" />
Editor
<!-- eslint-disable-next-line svelte/no-at-html-tags -- sanitized via renderMarkdown() -->
<div class="preview">{@html renderMarkdown(notesText)}</div>
{:else}
<Eye size={14} aria-hidden="true" />
Preview
<textarea
bind:this={editorEl}
bind:value={notesText}
oninput={scheduleSave}
placeholder="Notes…"
></textarea>
{/if}
</button>
<span class="spacer"></span>
<button onclick={exportMd} title="Export notes as .md">
<FileText size={13} aria-hidden="true" />
.md
</button>
<button onclick={exportPdf} title="Export notes as .pdf">
<FileDown size={13} aria-hidden="true" />
PDF
</button>
<button onclick={exportDocx} title="Export notes as .docx">
<FileDown size={13} aria-hidden="true" />
Word
</button>
<button onclick={exportBundle} title="Export audio + transcript + notes to a folder">
<FolderOutput size={13} aria-hidden="true" />
Bundle
</button>
</div>
<div class="editor-preview">
{#if notesPreview}
<!-- eslint-disable-next-line svelte/no-at-html-tags -- sanitized via renderMarkdown() -->
<div class="preview">{@html renderMarkdown(notesText)}</div>
{:else}
<textarea
bind:this={editorEl}
bind:value={notesText}
oninput={scheduleSave}
placeholder="Notes…"
></textarea>
{/if}
</div>
</div>
{/if}
</div>
</div>
{:else if recording.segments.length === 0}
{:else if recording.state === "idle"}
<p class="muted pad">Transcript will appear here as you record.</p>
{:else}
<div class="pad">
{#each recording.segments as s (s.id)}
<p class:interim={s.interim}>
<strong>{speakerName(s.speaker)}:</strong>
{s.text}
</p>
{/each}
<!-- Granola-style redesign: the Notes pane is open and typable while
recording, and clicking a transcript line attaches a note to that
moment — both merged into notes.md with the transcript at stop. -->
<div
class="split"
style="grid-template-columns: {splitColumns()};"
bind:clientWidth={splitWidth}
>
<div class="pane transcript" class:collapsed={layout.transcriptCollapsed}>
{#if layout.transcriptCollapsed}
<button
class="pane-toggle"
onclick={toggleTranscript}
title="Show transcript"
aria-label="Show transcript"
>
<PanelLeftOpen size={16} aria-hidden="true" />
</button>
{:else}
<h4>
<MessageSquareText size={14} aria-hidden="true" /> Transcript
<button
class="pane-toggle inline"
onclick={toggleTranscript}
title="Hide transcript"
aria-label="Hide transcript"
>
<PanelLeftClose size={13} aria-hidden="true" />
</button>
</h4>
{#if recording.segments.length === 0}
<p class="muted">Transcript will appear here as you record.</p>
{:else}
{#each recording.segments as s (s.id)}
{@const hasNote = recording.segmentNotes.has(s.start_ms)}
{@const open = selectedSegmentMs === s.start_ms}
<div class="segment">
<button
type="button"
class="segment-line"
class:interim={s.interim}
class:active={open}
onclick={() => (selectedSegmentMs = open ? null : s.start_ms)}
>
<strong>{speakerName(s.speaker)}:</strong>
{s.text}
{#if hasNote}
<span class="note-badge" title="Has a note">📝</span>
{/if}
</button>
{#if open}
<input
type="text"
class="segment-note-input"
placeholder="Add a note for this moment…"
value={recording.segmentNotes.get(s.start_ms) ?? ""}
onchange={(e) =>
recording.setSegmentNote(s.start_ms, (e.target as HTMLInputElement).value)}
aria-label="Note for this transcript line"
/>
{/if}
</div>
{/each}
{/if}
{/if}
</div>
{#if !layout.transcriptCollapsed && !layout.notesCollapsed}
<Splitter
label="Resize transcript and notes"
onResize={onSplitResize}
onResizeEnd={() => layout.persist()}
/>
{:else}
<span></span>
{/if}
<div class="pane notes" class:collapsed={layout.notesCollapsed}>
{#if layout.notesCollapsed}
<button
class="pane-toggle"
onclick={toggleNotes}
title="Show notes"
aria-label="Show notes"
>
<PanelRightOpen size={16} aria-hidden="true" />
</button>
{:else}
<h4>
<NotebookPen size={14} aria-hidden="true" /> Notes
<button
class="pane-toggle inline"
onclick={toggleNotes}
title="Hide notes"
aria-label="Hide notes"
>
<PanelRightClose size={13} aria-hidden="true" />
</button>
</h4>
<textarea
class="live-notes"
value={recording.notesText}
oninput={(e) => recording.setNotesText((e.target as HTMLTextAreaElement).value)}
placeholder="Type notes while you talk…"
></textarea>
{/if}
</div>
</div>
{/if}
</div>
@@ -328,9 +529,71 @@
line-height: 1.5;
}
.segment {
margin: 0.2rem 0;
}
.segment-line {
display: block;
width: 100%;
text-align: left;
background: transparent;
border: none;
border-left: 2px solid transparent;
color: inherit;
font: inherit;
line-height: 1.5;
padding: 0.15rem 0.4rem;
border-radius: var(--radius-sm);
cursor: pointer;
transition: background-color 120ms ease-out;
}
.segment-line:hover {
background: var(--bg-hover);
}
.segment-line.active {
background: var(--accent-soft);
border-left-color: var(--accent);
}
.segment-line.interim {
opacity: 0.55;
font-style: italic;
}
.note-badge {
margin-left: 0.3rem;
}
.segment-note-input {
display: block;
width: 100%;
box-sizing: border-box;
margin: 0.2rem 0 0.4rem;
padding: 0.3rem 0.5rem;
border: 1px solid var(--accent);
border-radius: var(--radius-sm);
background: var(--bg);
color: var(--fg);
font: inherit;
font-size: 0.85rem;
}
.live-notes {
resize: none;
width: 100%;
height: calc(100% - 2rem);
box-sizing: border-box;
padding: 0.6rem;
border: 1px solid var(--border);
border-radius: var(--radius-md);
background: var(--bg);
color: var(--fg);
font: inherit;
line-height: 1.5;
}
.live-notes:focus-visible {
border-color: var(--accent);
}
.split {
display: grid;
grid-template-columns: 1fr 1fr;
/* grid-template-columns set inline — depends on resize/collapse state (FR-UX-1). */
flex: 1;
min-height: 0;
}
@@ -342,6 +605,12 @@
.pane.transcript {
border-right: 1px solid var(--border);
}
.pane.collapsed {
display: flex;
align-items: flex-start;
justify-content: center;
padding: 0.4rem;
}
.pane h4 {
display: flex;
align-items: center;
@@ -353,6 +622,25 @@
letter-spacing: 0.04em;
color: var(--muted);
}
.pane-toggle {
display: flex;
align-items: center;
justify-content: center;
flex: none;
background: none;
border: none;
color: var(--muted);
cursor: pointer;
padding: 0.3rem;
border-radius: var(--radius-sm);
}
.pane-toggle:hover {
background: var(--bg-hover);
color: var(--fg);
}
.pane-toggle.inline {
margin-left: auto;
}
/* Transcription language (T8.7, FR-TRX-4) — a quiet pill, not a status
color, since "which language" isn't a good/bad state to flag. */
.badge.lang {
+5
View File
@@ -1,6 +1,11 @@
import App from "./App.svelte";
import { mount } from "svelte";
// The WebView2 default right-click menu (Back/Forward/Reload/Inspect) doesn't
// belong in a native-feeling desktop app — disabled app-wide until/unless a
// WhispAssist-specific context menu replaces it (see project memory).
document.addEventListener("contextmenu", (e) => e.preventDefault());
const app = mount(App, { target: document.getElementById("app")! });
export default app;