diff --git a/RELEASE_NOTES_0.7.1.md b/RELEASE_NOTES_0.7.1.md new file mode 100644 index 0000000..584ab52 --- /dev/null +++ b/RELEASE_NOTES_0.7.1.md @@ -0,0 +1,94 @@ +# WhispAssist v0.7.1 + +**Privacy-first, Windows-native meeting assistant — everything on-device, nothing leaves unless you say so.** + +A quality-of-life release focused on **importing meetings, taking notes, and living in the +background**. Adding a meeting from a file or link now runs without freezing the app and shows you +exactly where it's up to; notes gained AI cleanup and quick formatting; and WhispAssist can now mute +your mic mid-meeting and tuck itself into the system tray. No feature here changes the privacy +posture: everything optional stays off-by-default and local-first. + +One universal installer (**MSI** and **NSIS**) covers every machine: **Vulkan** for all GPUs +(NVIDIA/AMD/Intel), the Intel **NPU** (OpenVINO), a **DirectML** fallback, and **CPU**. + +--- + +## ✨ New + +### Add a meeting — now in the background, with a progress tracker +Importing a recording (a local audio/video file or a YouTube/streaming/direct URL) no longer blocks +the app while it works. Click **Import** and the meeting appears in your list immediately with a +**four-step progress tracker** — *Transcode → Transcribe → Identify speakers → Finalize* — where the +current step pulses and finished steps show how long they took. A 25-minute video that used to lock +the window for ~13 minutes now transcribes quietly in the background. + +- **Pick the transcription model** right in the dialog, and see **"Transcribed with …"** on the + finished meeting so you always know how it was produced. +- **One-click links** to download `ffmpeg` and `yt-dlp` (still external, not bundled). +- A failed import stays in your list marked **error** instead of vanishing. + +### Better notes +- **AI-enhance** (✨): turn rough notes into clean, structured notes using your local LLM, with a + one-step **Undo**. Off unless you have a local model configured. +- **Slash commands & a formatting toolbar**: type `/todo`, `/h1`, `/quote`, … or use the toolbar for + headings, lists, checkboxes, quotes, and dividers. +- **Fix:** notes no longer show stale text after re-transcribing a meeting — the pane updates in + place, no restart needed. + +### Mute your microphone — press **M** +Mute/unmute the mic mid-meeting with the **M** key or the new mic button by the level meter. The mic +channel goes silent (recording, live transcript, and meter) while system/loopback audio keeps +capturing. + +### Close to system tray +Closing the window now **keeps WhispAssist running in the background** instead of quitting. Reopen it +from the tray icon; the tray's **Quit** exits fully. On by default — toggle it in +**Settings ▸ Recording ▸ Close to system tray**. (This release also fixes a bug that showed **two** +WhispAssist icons in the tray — there's now just one.) + +### Privacy & hardware odds and ends +- **Vault lock card** in **Settings ▸ Privacy**: lock/unlock the encrypted store and change its + password at a glance. +- **Test your audio devices**: a live level meter for your mic and system audio, plus a test tone. +- **Quick hardware stress test**: benchmark the available backends against your installed models and + apply the fastest real-time combination. +- Tidier recording header. + +--- + +## 📦 Install + +**Requirements:** Windows 10 or 11 (x64). WhispAssist needs **WebView2** (preinstalled on Windows 11; +the installer fetches it on Windows 10). Importing from a file/URL additionally needs **`ffmpeg`** +(and **`yt-dlp`** for URLs) on your PATH — the Import dialog now links to both. + +1. Download **`WhispAssist_0.7.1_x64_en-US.msi`** (or the NSIS **`WhispAssist_0.7.1_x64-setup.exe`**). +2. Run it and accept the UAC prompt. If SmartScreen appears, choose **More info → Run anyway**. +3. Launch **WhispAssist** from the Start menu. + +On first run WA picks the best transcription backend (**NPU → NVIDIA → AMD → Intel → CPU**). It runs +without admin rights, does **not** add itself to startup unless you opt in, and keeps all data under +`%LOCALAPPDATA%\WhispAssist`. + +Deploying to many machines? See [`docs/enterprise-deployment.md`](docs/enterprise-deployment.md). + +## 🔐 Checksums (SHA-256) + +``` +bd96a059db3658a9bee81161edc709cfc4741ed99d6c66a12194403b339f9369 WhispAssist_0.7.1_x64_en-US.msi +711fa7df618c8fc3f03c18d543abad2288d56f750b532288204e0ec0f9426595 WhispAssist_0.7.1_x64-setup.exe +``` + +Verify after download: + +```powershell +Get-FileHash .\WhispAssist_0.7.1_x64_en-US.msi -Algorithm SHA256 +``` + +--- + +## Privacy, unchanged +Everything optional is **off by default**. With nothing configured, WhispAssist makes **no content +egress at all**. Recording is opt-in; sync/AI credentials live only in the OS credential store; the +MCP server is loopback-only and adds no egress; the deployment file never carries secrets. The +reachable-host allowlist is derived from your settings and enforced in the core. diff --git a/docs/03-data-model.md b/docs/03-data-model.md index 4380e74..5a1f966 100644 --- a/docs/03-data-model.md +++ b/docs/03-data-model.md @@ -386,6 +386,9 @@ segment (the M1 grounding invariant, asserted by the golden-transcript test). // via `set_auto_start` also writes a per-user `HKCU\...\Run` entry (no admin); // startup reconciles the OS entry to this flag (e.g. after a reinstall). "auto_start": false, + // Closing the window hides WhispAssist to the system tray (keep running in background) instead of + // quitting; ON by default. Tray "Quit" is the real exit. Enforced in the Rust on_window_event handler. + "close_to_tray": true, // Optional MS Graph calendar source (M4.4, T8.9, FR-CAL-6). Opt-in, explicit consent via OAuth // PKCE — OFF by default. `credential_ref` points into the OS credential store; the token itself // is never written here (same invariant as sync credentials, FR-SYNC-6). diff --git a/docs/04-api-contracts.md b/docs/04-api-contracts.md index 4cad933..2cf9769 100644 --- a/docs/04-api-contracts.md +++ b/docs/04-api-contracts.md @@ -23,6 +23,9 @@ start_recording(input: { meetingTitle?: string; calendarEventId?: string; record stop_recording(input: { meetingId: MeetingId }): MeetingSummaryRef pause_recording(input: { meetingId: MeetingId }): void resume_recording(input: { meetingId: MeetingId }): void +// Mute/unmute the mic mid-meeting (FR-CAP-7): mic channel goes silent (recording + transcript + meter), +// loopback keeps capturing. Returns the new muted state; emits recording://mic. Errs if the mic is off. +toggle_microphone_mute(input: { meetingId: MeetingId }): boolean set_recording_retention(input: { meetingId: MeetingId; record: boolean }): void // toggle mid-meeting (FR-REC-1) acknowledge_recording_consent(): void // one-time (FR-REC-2) @@ -40,6 +43,10 @@ hardware_status(): { backends: BackendInfo[]; active: BackendId; modelSize: stri set_preferred_backend(input: { backend: BackendId | "auto" }): void // Launch-at-login (NFR-RES-4). Writes/removes a per-user OS Run entry (no admin) and persists auto_start. Opt-in, off by default. set_auto_start(input: { enabled: boolean }): void +// Device test: opens a mic ("input") or the render device in loopback ("loopback") for a few seconds and streams device://level (no recording, no retained audio). Refused while recording. +monitor_audio_level(input: { kind: "input" | "loopback"; deviceId?: string; durationMs?: number }): void +// Quick stress test: benchmarks each available backend × installed model (≤3 sizes) on a fixed sample, returns per-pair real-time factor + the most-accurate real-time-capable recommendation. Emits stress://progress. Refused while recording. +stress_test_hardware(): { results: { backend: string; model: string; rtf: number; realtime: boolean }[]; recommended: { backend: string; model: string } | null } // ---- Transcription / models ---- // language (T8.7, M4.2): omitted reuses the meeting's current language rather than resetting it. @@ -81,6 +88,8 @@ export_meeting(input: { meetingId: MeetingId; dest: string; format: "md" | "pdf" // they enqueue for finalize-trigger targets and pump in the background. SHA-256 dedup means an // edit that didn't alter a file uploads nothing. update_notes(input: { meetingId: MeetingId; markdown: string }): void +// AI-enhance rough notes into structured Markdown grounded in the transcript (Granola-style), via the configured LlmProvider (no new egress). Takes the live buffer, returns the enhanced text WITHOUT persisting — the UI keeps or undoes it. Refused while recording; errors with no provider. +enhance_notes(input: { meetingId: MeetingId; notes: string }): string // SearchHit = MeetingListItem fields (id, title, started_at, duration_secs, status, tags) + snippet: string search(input: { query: string }): SearchHit[] // FTS (FR-SEARCH-1) set_tags(input: { meetingId: MeetingId; tags: string[] }): void @@ -95,6 +104,13 @@ bulk_export_meetings(input: { destDir: string; format: "md" | "pdf" | "docx" | " // folder of them (from a bulk export). Each is reconstructed under a fresh meeting id (original // title/date/duration/speakers/tags/action items preserved). Returns the count imported. import_meeting_bundle(input: { dir: string }): number +// Add a meeting from an existing recording: a local audio/video file path or a URL (YouTube/ +// streaming page or direct media URL). Needs ffmpeg (+ yt-dlp for URLs) on PATH; neither bundled. +// `model` overrides the Settings whisper model for this import (recorded as meeting.model_used). +// Returns the new meeting id IMMEDIATELY (status "transcribing"); transcode→transcribe→diarize→ +// finalize run in the background, streaming import://progress and ending with transcript://finalized. +// A failed import is left in the list with status "error" (not deleted). +import_media(input: { source: string; title?: string; model?: string }): MeetingId // ---- LLM / AI provider (ADR-0007/0011) ---- // provider ∈ ollama | custom | anthropic | openai | off. Hosted-provider API keys are passed to @@ -185,8 +201,10 @@ privacy_self_check(): { "recording://state" { meetingId, state: "recording"|"paused"|"stopped"|"cancelled", elapsedMs } "recording://level" { meetingId, rms: number, peak: number } // waveform (FR-CAP-5) "recording://device" { meetingId, recovered: boolean, message: string } // capture device change (FR-CAP-6) +"recording://mic" { meetingId, muted: boolean } // mic mute toggled (FR-CAP-7) "transcript://segment" { meetingId, segment: TranscriptSegment } // live segments (FR-TRX-2); may re-emit a committed segment with a refined `speaker` — replace by `segment.id` "transcript://finalized" { meetingId, segmentCount } +"import://progress" { meetingId, phase: "prepare"|"transcribe"|"diarize"|"finalize", state: "active"|"done"|"error", elapsedMs: number|null, error: string|null } // background import_media tracker "diarization://updated" { meetingId, speakers: SpeakerInfo[] } // post-pass AND live 15s provisional passes (FR-SPK); carries "You" once the mic voiceprint matches "llm://token" { meetingId, text } // streamed summary (FR-LLM-4) "llm://done" { meetingId, summary: SummaryFile } // full summary.json contents, not just a pointer @@ -195,6 +213,8 @@ privacy_self_check(): { "calendar://linked" { ok: boolean, error?: string } // MS Graph OAuth handshake settled (M4.4) "calendar://progress" { processed, total } // MS Graph import (M4.4) "hardware://changed" { active: BackendId, reason: string } // fallback occurred (FR-HW-4) +"device://level" { kind: "input"|"loopback", rms?, peak?, done?: boolean } // Settings device test meter; done=window ended +"stress://progress" { backend: string, model: string } // quick stress test, per pairing benchmarked "recording://retention" { meetingId, record: boolean } // retention toggled (FR-REC-1/3) "sync://job" { jobId, meetingId, targetId, artifact, status, bytesSent, bytesTotal } // FR-SYNC-5 "sync://done" { meetingId, targetId, uploaded: number, failed: number } diff --git a/package.json b/package.json index 0aa7b3b..5de12bc 100644 --- a/package.json +++ b/package.json @@ -1,7 +1,7 @@ { "name": "whispassist", "private": true, - "version": "0.7.0", + "version": "0.7.2", "type": "module", "description": "Privacy-first, fully local Windows meeting assistant.", "license": "MIT OR Apache-2.0", diff --git a/src-tauri/Cargo.lock b/src-tauri/Cargo.lock index 3dfc2b3..b036b3d 100644 --- a/src-tauri/Cargo.lock +++ b/src-tauri/Cargo.lock @@ -6088,7 +6088,7 @@ dependencies = [ [[package]] name = "whispassist" -version = "0.7.0" +version = "0.7.2" dependencies = [ "argon2", "async-trait", diff --git a/src-tauri/Cargo.toml b/src-tauri/Cargo.toml index 748dab4..3db30ed 100644 --- a/src-tauri/Cargo.toml +++ b/src-tauri/Cargo.toml @@ -1,6 +1,6 @@ [package] name = "whispassist" -version = "0.7.0" +version = "0.7.2" description = "Privacy-first, fully local Windows meeting assistant" authors = ["WhispAssist contributors"] license = "MIT OR Apache-2.0" diff --git a/src-tauri/src/audio/mod.rs b/src-tauri/src/audio/mod.rs index 0765842..0d71224 100644 --- a/src-tauri/src/audio/mod.rs +++ b/src-tauri/src/audio/mod.rs @@ -45,9 +45,26 @@ pub enum AudioError { pub struct CaptureHandle { running: Arc, paused: Arc, + /// Mic-only (FR-CAP-7): when set, the microphone stream emits silence instead + /// of real samples — the recording's mic-left channel and the live transcript + /// go quiet, the meter drops to zero, while loopback keeps recording. Toggled + /// live via `set_muted` (the "press M to mute" control). + muted: Arc, thread: JoinHandle>, } +impl CaptureHandle { + /// Mute/unmute this stream live. Only meaningful for the microphone capture. + pub fn set_muted(&self, muted: bool) { + self.muted.store(muted, Ordering::SeqCst); + } + + /// Whether this stream is currently muted. + pub fn is_muted(&self) -> bool { + self.muted.load(Ordering::SeqCst) + } +} + /// Where captured frames are delivered for live transcription: mono f32 @ 16kHz, /// bounded so a slow/absent consumer can never stall the capture thread. pub type FrameSink = SyncSender>; @@ -317,8 +334,10 @@ impl WasapiCapture { ) -> Result { let running = Arc::new(AtomicBool::new(true)); let paused = Arc::new(AtomicBool::new(false)); + let muted = Arc::new(AtomicBool::new(false)); let running_th = running.clone(); let paused_th = paused.clone(); + let muted_th = muted.clone(); let wav_path = wav_path.map(Path::to_path_buf); let device_id = device_id.map(str::to_string); @@ -333,6 +352,7 @@ impl WasapiCapture { &event_sink, &running_th, &paused_th, + &muted_th, bridge.as_ref(), voice_sample.as_ref(), split, @@ -344,6 +364,7 @@ impl WasapiCapture { Ok(CaptureHandle { running, paused, + muted, thread, }) } @@ -610,6 +631,9 @@ fn capture_loop( event_sink: &EventSink, running: &AtomicBool, paused: &AtomicBool, + // Mic-only live mute (FR-CAP-7): zeroes the decoded mic samples so the + // recording, transcript, and meter all go silent while loopback continues. + muted: &AtomicBool, bridge: Option<&Arc>, voice_sample: Option<&Arc>, // FR-SPK: when true, the loopback WAV is stereo L=mic / R=loopback (the mic @@ -762,7 +786,14 @@ fn capture_loop( write_wav_bytes(w, &bytes, &session.format, &mic)? }; } - let mono = decode_mono_f32(&bytes, &session.format)?; + let mut mono = decode_mono_f32(&bytes, &session.format)?; + // Mic muted: replace the decoded samples with silence before anything + // downstream sees them — the recording's mic channel, the bridge, the + // transcript feed, the meter, and the voiceprint sample all go quiet. + // Loopback (`is_loopback`) is never muted this way. + if !is_loopback && muted.load(Ordering::Relaxed) { + mono.iter_mut().for_each(|s| *s = 0.0); + } // Mic: feed the shared bridge (resampled to the loopback's rate) so the // loopback thread can fold it into the recording. diff --git a/src-tauri/src/commands.rs b/src-tauri/src/commands.rs index 0340017..11867ee 100644 --- a/src-tauri/src/commands.rs +++ b/src-tauri/src/commands.rs @@ -88,6 +88,7 @@ pub(crate) fn default_settings() -> Settings { mcp_expose: "none".into(), mcp_expose_recordings: false, auto_start: false, + close_to_tray: true, } } @@ -119,6 +120,32 @@ fn model_id_for(settings: &Settings) -> String { } } +/// The model id to *report* for a session on `backend`: the ONNX engine +/// (NPU/DirectML) ignores the configured ggml model and always runs its own +/// ONNX artifacts, so recording the ggml id would be a lie — e.g. a meeting +/// shown as "medium.en-q5_0 · npu" actually transcribed with ONNX base.en. +/// Mirrors the exact routing condition in `load_transcriber`; if the engine +/// fails to load at runtime the worker falls back to whisper.cpp and the +/// backend is corrected via `hardware://changed`, but this reported model +/// isn't — acceptable for that rare failure path. +fn effective_model_id(backend: BackendId, requested: &str) -> String { + #[cfg(feature = "npu")] + { + use crate::hardware::{resolve_accel, AccelPath}; + use crate::transcription::onnx_models; + if matches!( + resolve_accel(backend), + AccelPath::OnnxOpenVino | AccelPath::OnnxDirectML + ) && onnx_models::is_installed(onnx_models::DEFAULT_ONNX_MODEL) + { + return format!("{} (onnx)", onnx_models::DEFAULT_ONNX_MODEL); + } + } + #[cfg(not(feature = "npu"))] + let _ = backend; + requested.to_string() +} + /// Picks the backend to transcribe with: "low overhead" always forces CPU; /// otherwise resolve the user's preferred backend (or auto-detect) against /// what's actually available (T3.2, T3.5). @@ -165,7 +192,22 @@ fn load_transcriber( if onnx_models::is_installed(onnx_models::DEFAULT_ONNX_MODEL) { let dir = onnx_models::model_dir(onnx_models::DEFAULT_ONNX_MODEL); match OnnxTranscriber::load(&dir, backend, language) { - Ok(t) => return Ok((Box::new(t), backend)), + Ok(t) => { + // The definitive "is the accelerator actually engaged" line + // (visible with `npm run tauri dev`): the encoder runs on + // the EP below; the autoregressive decoder always runs on + // the CPU EP, which is why Task Manager shows only short + // periodic NPU spikes alongside sustained CPU load. + tracing::info!( + "transcription engine: ONNX {} (encoder EP: {}, decoder: CPU)", + onnx_models::DEFAULT_ONNX_MODEL, + match path { + AccelPath::OnnxOpenVino => "OpenVINO/NPU", + _ => "DirectML/GPU", + } + ); + return Ok((Box::new(t), backend)); + } Err(e) => tracing::warn!("ONNX engine load failed ({e}); falling back to CPU"), } } else { @@ -256,7 +298,8 @@ fn diarizer_from_installed_models() -> Option { /// Split attribution (FR-SPK, supersedes `phase3_attribute`): diarize the far /// side (right/loopback channel) into `Speaker N`, take "You" straight from -/// left-channel (mic) voice activity, merge + assign + name. Shared by +/// left-channel (mic) voice activity, attribute per channel +/// (`diarization::assign_split`) + name. Shared by /// `stop_recording` and `reprocess_transcript`; both read it back from the file /// so they agree. `None` if the far-side pass fails (caller falls back). async fn attribute_split( @@ -267,16 +310,19 @@ async fn attribute_split( let diarizer_for_task = diarizer.clone(); let result = tauri::async_runtime::spawn_blocking(move || { // Right channel = loopback/far side; diarize it alone (mic never in it). + // Its VAD is the "was the far side talking at all" evidence assign_split + // weighs against the mic channel. let far = crate::audio::read_wav_channel_16k(&wav_path, 1).map_err(|e| e.to_string())?; + let far_vad = crate::audio::vad_spans(&far); let far_spans = diarizer_for_task .diarize_samples(far) .map_err(|e| e.to_string())?; // Left channel = mic; its voice activity is "You". let mic = crate::audio::read_wav_channel_16k(&wav_path, 0).map_err(|e| e.to_string())?; - Ok::<_, String>((far_spans, crate::audio::vad_spans(&mic))) + Ok::<_, String>((far_spans, far_vad, crate::audio::vad_spans(&mic))) }) .await; - let (far_spans, you_spans) = match result { + let (far_spans, far_vad, you_spans) = match result { Ok(Ok(v)) => v, Ok(Err(e)) => { tracing::warn!("split attribution failed: {e}"); @@ -287,6 +333,8 @@ async fn attribute_split( return None; } }; + crate::diarization::assign_split(segments, &you_spans, &far_vad, &far_spans); + // Merged span timeline, only for first-appearance naming order below. let mut spans: Vec = you_spans .iter() .map(|&(start_ms, end_ms)| SpeakerSpan { @@ -297,7 +345,6 @@ async fn attribute_split( .collect(); spans.extend(far_spans); spans.sort_by_key(|s| s.start_ms); - diarizer.assign(segments, &spans); // Uniform naming: "You" -> "You", far speakers -> "Speaker 2", "Speaker 3"…. let labels = crate::diarization::voiceprint::first_appearance_order(&spans); Some(crate::diarization::voiceprint::build_name_map(&labels, "You")) @@ -573,7 +620,8 @@ pub async fn start_recording( let wav_path_for_diar = wav_path.clone(); let segments_for_diar = segments.clone(); let names_for_diar = speaker_names.clone(); - let voice_sample_for_diar = mic_voice_sample.clone(); + // Same condition as `audio_layout` below: mic on → split stereo file. + let split_layout = settings.microphone_enabled; tauri::async_runtime::spawn(async move { // ponytail: reprocesses the whole recording-so-far each tick // rather than incremental/windowed segmentation — sherpa-onnx's @@ -596,56 +644,83 @@ pub async fn start_recording( break; // recording stopped (or a new one started) — nothing left to do } - let diarizer_for_pass = diarizer.clone(); - let wav_path = wav_path_for_diar.clone(); - let spans = tauri::async_runtime::spawn_blocking(move || { - diarizer_for_pass.diarize(&wav_path) - }) - .await; - let spans = match spans { - Ok(Ok(spans)) => spans, - Ok(Err(e)) => { - tracing::warn!("live diarization pass failed: {e}"); - continue; + // Split recording: the exact same channel-based attribution as + // the post-stop pass — mic (left) is always "You", the far side + // (right) is diarized alone. The old whole-mix pass + voiceprint + // cosine match never reliably showed "You" live (clusters over + // the summed mix reshuffle every tick and the match often missed + // its threshold), so "You" only appeared after stop. + let (speakers, changed) = if split_layout { + let mut snapshot = segments_for_diar + .lock() + .unwrap_or_else(|e| e.into_inner()) + .clone(); + if attribute_split( + diarizer.clone(), + wav_path_for_diar.clone(), + &mut snapshot, + ) + .await + .map(|auto_names| { + let mut names = names_for_diar.lock().unwrap_or_else(|e| e.into_inner()); + // Never overwrite a name already set (user rename or a + // prior pass) — same guard as the post-stop pass. + for (label, name) in auto_names { + names.entry(label).or_insert(name); + } + }) + .is_none() + { + continue; // pass failed (already logged); retry next tick } - Err(e) => { - tracing::warn!("live diarization task failed: {e}"); - continue; - } - }; - - let (speakers, changed) = { + let relabeled: HashMap = snapshot + .into_iter() + .map(|s| (s.id, s.speaker)) + .collect(); let mut segs = segments_for_diar.lock().unwrap_or_else(|e| e.into_inner()); // Snapshot prior labels so only segments whose speaker // actually changed this pass get re-emitted (ids are stable; - // the frontend replaces by id). + // the frontend replaces by id). Segments committed while the + // pass ran keep their placeholder until the next tick. + let before: HashMap = + segs.iter().map(|s| (s.id, s.speaker.clone())).collect(); + for seg in segs.iter_mut() { + if let Some(speaker) = relabeled.get(&seg.id) { + seg.speaker = speaker.clone(); + } + } + let names = names_for_diar.lock().unwrap_or_else(|e| e.into_inner()); + let changed: Vec = segs + .iter() + .filter(|s| before.get(&s.id) != Some(&s.speaker)) + .cloned() + .collect(); + (speaker_infos_from_segments(&segs, &names), changed) + } else { + // Summed recording (mic off): whole-signal pass. No mic + // channel → no live "You" (the mic voice sample doesn't + // exist either), matching the post-stop behavior. + let diarizer_for_pass = diarizer.clone(); + let wav_path = wav_path_for_diar.clone(); + let spans = tauri::async_runtime::spawn_blocking(move || { + diarizer_for_pass.diarize(&wav_path) + }) + .await; + let spans = match spans { + Ok(Ok(spans)) => spans, + Ok(Err(e)) => { + tracing::warn!("live diarization pass failed: {e}"); + continue; + } + Err(e) => { + tracing::warn!("live diarization task failed: {e}"); + continue; + } + }; + let mut segs = segments_for_diar.lock().unwrap_or_else(|e| e.into_inner()); let before: HashMap = segs.iter().map(|s| (s.id, s.speaker.clone())).collect(); diarizer.assign(&mut segs, &spans); - - // Live "You": match the mic voiceprint against this pass's - // clusters. Clusters re-shuffle every tick so match every - // tick; never overwrite a name already set (user rename or a - // prior pass) — same guard as the post-stop pass. - if let Some(voice_sample) = &voice_sample_for_diar { - let mic_samples = voice_sample.samples(); - match crate::diarization::voiceprint::match_mic_speaker( - &diarization_embedding_model_file(), - &mic_samples, - &wav_path_for_diar, - &spans, - ) { - Ok(auto_names) => { - let mut names = - names_for_diar.lock().unwrap_or_else(|e| e.into_inner()); - for (label, name) in auto_names { - names.entry(label).or_insert(name); - } - } - Err(e) => tracing::warn!("live voiceprint match failed: {e}"), - } - } - let names = names_for_diar.lock().unwrap_or_else(|e| e.into_inner()); let changed: Vec = segs .iter() @@ -681,7 +756,10 @@ pub async fn start_recording( transcription_worker, segments, active_backend, - model_id, + // Report the model the routed engine will actually run (the ONNX + // engine ignores the configured ggml model), so the meeting's + // "transcribed with … · npu" line is truthful (T3.4/T3.5). + model_id: effective_model_id(backend, &model_id), language: language_state, diarizer, speaker_names, @@ -760,9 +838,17 @@ pub async fn stop_recording( let mut attributed = false; if session.audio_layout == "split" { if let Some(diarizer) = session.diarizer.clone() { - if let Some(names) = + if let Some(mut names) = attribute_split(diarizer, session.wav_path.clone(), &mut segments).await { + // Renames made *during* the recording are user-authored — they + // win over the automatic "You"/"Speaker N" defaults instead of + // being silently dropped at finalize. + if let Ok(user_names) = session.speaker_names.lock() { + for (label, name) in user_names.iter() { + names.insert(label.clone(), name.clone()); + } + } speaker_names = names; attributed = true; } @@ -1048,6 +1134,43 @@ pub async fn resume_recording( Ok(()) } +/// Toggle the microphone mute state for the active recording (FR-CAP-7): the +/// mic channel goes silent (recording + live transcript + meter) while loopback +/// keeps capturing. Bound to the "M" key in the UI. Returns the new muted state. +/// Errors if this meeting was started with the mic off (nothing to mute). +#[tauri::command] +pub async fn toggle_microphone_mute( + app: AppHandle, + state: State<'_, AppState>, + meeting_id: MeetingId, +) -> WaResult { + let guard = state.session.lock().await; + let session = guard + .as_ref() + .filter(|s| s.meeting_id == meeting_id) + .ok_or_else(|| WaError::new("recording", "no matching active recording"))?; + let mic = session + .mic_capture + .as_ref() + .ok_or_else(|| WaError::new("audio", "the microphone is off for this meeting"))?; + let muted = !mic.is_muted(); + mic.set_muted(muted); + drop(guard); + let _ = app.emit( + "recording://mic", + serde_json::json!({ "meetingId": meeting_id, "muted": muted }), + ); + crate::update_tray_tooltip( + &app, + if muted { + "WhispAssist — recording (mic muted)" + } else { + "WhispAssist — recording" + }, + ); + Ok(muted) +} + /// Toggle audio retention mid-meeting (ADR-0009, FR-REC-1). #[tauri::command] pub async fn set_recording_retention( @@ -1222,6 +1345,55 @@ async fn refresh_notes_and_notify( Ok(meeting.speakers) } +/// `notes.md` bakes display names into its `**Name:**` dialogue tags at +/// finalize, so a post-meeting rename must rewrite them or the Notes pane +/// (and every export) keeps the old name forever. Targeted tag replace, not +/// a regenerate, so the user's own edits to notes.md are preserved. +async fn rename_speaker_in_notes( + state: &State<'_, AppState>, + meeting_id: &MeetingId, + old_name: &str, + new_name: &str, +) -> WaResult<()> { + if old_name == new_name || new_name.is_empty() { + return Ok(()); + } + let meeting = state + .store + .get_meeting(meeting_id) + .await + .map_err(|e| WaError::new("storage", e.to_string()))?; + let old_tag = format!("**{old_name}:**"); + if meeting.notes_markdown.contains(&old_tag) { + let updated = meeting + .notes_markdown + .replace(&old_tag, &format!("**{new_name}:**")); + state + .store + .update_notes(meeting_id, &updated) + .await + .map_err(|e| WaError::new("storage", e.to_string()))?; + } + Ok(()) +} + +/// A speaker's current name as notes.md renders it: display name, else the +/// raw label. `None` if the meeting/speaker can't be read (nothing to rewrite). +async fn current_speaker_name( + state: &State<'_, AppState>, + meeting_id: &MeetingId, + label: &str, +) -> Option { + let meeting = state.store.get_meeting(meeting_id).await.ok()?; + let speaker = meeting.speakers.iter().find(|s| s.label == label)?; + Some( + speaker + .display_name + .clone() + .unwrap_or_else(|| label.to_string()), + ) +} + /// Name a speaker; applies to that speaker's past & future segments (T4.4, /// FR-SPK-2). Segments only ever carry the internal label ("S1"…) — never /// rewritten — so persisting the label→name mapping here is enough to cover @@ -1236,6 +1408,9 @@ pub async fn rename_speaker( label: String, name: String, ) -> WaResult<()> { + // Resolve the name notes.md currently shows *before* the rename lands, so + // the finalized-meeting branch below can rewrite its dialogue tags. + let old_name = current_speaker_name(&state, &meeting_id, &label).await; state .store .rename_speaker(&meeting_id, &label, &name) @@ -1271,6 +1446,9 @@ pub async fn rename_speaker( } None => { drop(guard); + if let Some(old_name) = old_name { + rename_speaker_in_notes(&state, &meeting_id, &old_name, &name).await?; + } refresh_notes_and_notify(&app, &state, &meeting_id).await?; } } @@ -1333,12 +1511,22 @@ pub async fn map_speaker_to_participant( } drop(guard); + let old_name = current_speaker_name(&state, &meeting_id, &label).await; state .store .map_speaker_to_participant(&meeting_id, &label, &participant_id) .await .map_err(|e| WaError::new("storage", e.to_string()))?; - refresh_notes_and_notify(&app, &state, &meeting_id).await?; + let speakers = refresh_notes_and_notify(&app, &state, &meeting_id).await?; + // Rewrite notes.md's baked-in dialogue tags to the participant's name, + // same as a free-text rename (the Notes pane must follow the Speakers pane). + let new_name = speakers + .iter() + .find(|s| s.label == label) + .and_then(|s| s.display_name.clone()); + if let (Some(old_name), Some(new_name)) = (old_name, new_name) { + rename_speaker_in_notes(&state, &meeting_id, &old_name, &new_name).await?; + } Ok(()) } @@ -1465,6 +1653,218 @@ pub async fn set_auto_start(app: AppHandle, enabled: bool) -> WaResult<()> { save_settings(&settings) } +/// Live level meter for a device, without starting a recording (FR-CAP-5): open +/// the selected mic (`"input"`) or the render device in loopback (`"loopback"`) +/// for a few seconds and stream `device://level` events so Settings ▸ Hardware +/// can show whether audio is coming through. Reuses the normal capture path; +/// discards frames (no transcription, no retained audio). One capture at a time, +/// so it refuses while a recording is active. +/// ponytail: transient monitor handle; ceiling = one monitor at a time. +#[tauri::command] +pub async fn monitor_audio_level( + app: AppHandle, + state: State<'_, AppState>, + kind: String, + device_id: Option, + duration_ms: Option, +) -> WaResult<()> { + use crate::audio::{AudioCapture, CaptureEvent, WasapiCapture}; + if state.session.lock().await.is_some() { + return Err(WaError::new( + "audio", + "stop the current recording before testing a device", + )); + } + let duration = + std::time::Duration::from_millis(duration_ms.unwrap_or(6000).clamp(1000, 20_000)); + + // Bounded channels so a slow consumer can't stall capture; frames are dropped. + let (frame_tx, frame_rx) = std::sync::mpsc::sync_channel::>(8); + let (event_tx, event_rx) = std::sync::mpsc::sync_channel::(64); + std::thread::spawn(move || frame_rx.into_iter().for_each(drop)); + + let app_ev = app.clone(); + let kind_ev = kind.clone(); + std::thread::spawn(move || { + for event in event_rx { + if let CaptureEvent::Level(level) = event { + let _ = app_ev.emit( + "device://level", + serde_json::json!({ "kind": kind_ev, "rms": level.rms, "peak": level.peak }), + ); + } + } + }); + + // `start` (loopback) needs a WAV path; write to a temp file and delete it after. + let tmp_wav = std::env::temp_dir().join(format!("wa-monitor-{}.wav", now_unix())); + let handle = match kind.as_str() { + "input" => WasapiCapture.start_microphone(device_id.as_deref(), frame_tx, event_tx), + "loopback" => WasapiCapture.start(&tmp_wav, device_id.as_deref(), frame_tx, event_tx), + _ => return Err(WaError::new("audio", "kind must be 'input' or 'loopback'")), + } + .map_err(|e| WaError::new("audio", e.to_string()))?; + + tauri::async_runtime::spawn_blocking(move || { + std::thread::sleep(duration); + let _ = WasapiCapture.stop(handle); + }) + .await + .map_err(|e| WaError::new("audio", e.to_string()))?; + + if kind == "loopback" { + let _ = std::fs::remove_file(&tmp_wav); + } + let _ = app.emit( + "device://level", + serde_json::json!({ "kind": kind, "done": true }), + ); + Ok(()) +} + +/// One row of the quick hardware stress test. +#[derive(Debug, Clone, serde::Serialize)] +pub struct StressResult { + pub backend: String, + pub model: String, + pub rtf: f64, + pub realtime: bool, +} + +#[derive(Debug, Clone, serde::Serialize)] +pub struct StressRecommendation { + pub backend: String, + pub model: String, +} + +#[derive(Debug, Clone, serde::Serialize)] +pub struct StressTestResult { + pub results: Vec, + pub recommended: Option, +} + +/// Real-time recommendation: among rows that keep up with live speech +/// (`rtf < 1`), prefer the **largest** model (most accurate), breaking ties by +/// the **lowest** rtf (most headroom). `sizes` maps model id → size_mb (an +/// accuracy proxy). `None` when nothing runs in real time. +fn pick_realtime_recommendation( + results: &[StressResult], + sizes: &HashMap, +) -> Option { + results + .iter() + .filter(|r| r.realtime) + .max_by(|a, b| { + let sa = sizes.get(&a.model).copied().unwrap_or(0); + let sb = sizes.get(&b.model).copied().unwrap_or(0); + sa.cmp(&sb) + .then(b.rtf.partial_cmp(&a.rtf).unwrap_or(std::cmp::Ordering::Equal)) + }) + .map(|r| StressRecommendation { + backend: r.backend.clone(), + model: r.model.clone(), + }) +} + +/// Quick hardware stress test (FR-HW): benchmark each available backend against +/// the installed whisper models on a fixed sample, measure real-time factor +/// (elapsed / audio seconds), and recommend the most accurate model that still +/// keeps up with live speech. Heavy (loads + runs each model) but explicit and +/// progress-reported; runs off the async thread. +/// ponytail: benchmarks installed models only, capped to 3 sizes. +#[tauri::command] +pub async fn stress_test_hardware( + app: AppHandle, + state: State<'_, AppState>, +) -> WaResult { + if state.session.lock().await.is_some() { + return Err(WaError::new( + "hardware", + "stop the current recording before running the stress test", + )); + } + + // Fixed ~10s 16kHz mono sample. RTF timing is ~content-independent, so a + // synthetic quiet tone is enough — no bundled speech clip needed. + const SAMPLE_SECS: f64 = 10.0; + let n = (16_000.0 * SAMPLE_SECS) as usize; + let samples: Vec = (0..n).map(|i| (i as f32 * 0.05).sin() * 0.1).collect(); + let wav = std::env::temp_dir().join("wa-stress-sample.wav"); + crate::audio::write_wav_mono_16k(&wav, &samples) + .map_err(|e| WaError::new("hardware", e.to_string()))?; + + let backends: Vec = WinHardwareDetector + .detect() + .into_iter() + .filter(|b| b.available) + .map(|b| b.id) + .collect(); + + // Installed models, smallest → largest, capped to bound runtime. + let mut models: Vec<(String, u32)> = model_catalog::list("") + .into_iter() + .filter(|m| m.installed) + .map(|m| (m.id, m.size_mb)) + .collect(); + models.sort_by_key(|(_, sz)| *sz); + models.truncate(3); + let sizes: HashMap = models.iter().cloned().collect(); + + if backends.is_empty() || models.is_empty() { + let _ = std::fs::remove_file(&wav); + return Err(WaError::new( + "hardware", + "no installed whisper model to benchmark — download one in Settings first", + )); + } + + let wav_for_task = wav.clone(); + let app_for_task = app.clone(); + let results = tauri::async_runtime::spawn_blocking(move || { + let mut out: Vec = Vec::new(); + for backend in &backends { + for (model_id, _size) in &models { + let path = whisper_model_file(model_id); + if !path.exists() { + continue; + } + let _ = app_for_task.emit( + "stress://progress", + serde_json::json!({ "backend": backend.as_str(), "model": model_id }), + ); + let start = std::time::Instant::now(); + let ok = match load_transcriber(*backend, &path, None) { + Ok((t, _used)) => t.transcribe_file(&wav_for_task).is_ok(), + Err(e) => { + tracing::warn!("stress test load failed ({backend:?}/{model_id}): {e}"); + false + } + }; + if !ok { + continue; + } + let rtf = start.elapsed().as_secs_f64() / SAMPLE_SECS; + out.push(StressResult { + backend: backend.as_str().to_string(), + model: model_id.clone(), + rtf, + realtime: rtf < 1.0, + }); + } + } + out + }) + .await + .map_err(|e| WaError::new("hardware", e.to_string()))?; + + let _ = std::fs::remove_file(&wav); + let recommended = pick_realtime_recommendation(&results, &sizes); + Ok(StressTestResult { + results, + recommended, + }) +} + /// True when the NPU ONNX Whisper model is downloaded (false on non-NPU builds). fn npu_model_installed() -> bool { #[cfg(feature = "npu")] @@ -1959,7 +2359,7 @@ pub async fn reprocess_transcript( recorded: meeting.recorded, language: resolved_language, backend_used: Some(backend.as_str().to_string()), - model_used: Some(model), + model_used: Some(effective_model_id(backend, &model)), audio_layout: None, // reprocess preserves the recorded layout }, ) @@ -1999,21 +2399,49 @@ fn default_title_from_source(source: &str) -> Option { .filter(|s| !s.trim().is_empty()) } +/// Emit one `import://progress` tick for the Domino's-tracker UI. `state` is +/// `active` (this phase is running), `done` (finished; `elapsed_ms` set), or +/// `error` (`message` set). `phase` is one of prepare/transcribe/diarize/finalize. +fn emit_import_progress( + app: &AppHandle, + meeting_id: &str, + phase: &str, + state: &str, + elapsed_ms: Option, + message: Option<&str>, +) { + let _ = app.emit( + "import://progress", + serde_json::json!({ + "meetingId": meeting_id, + "phase": phase, + "state": state, + "elapsedMs": elapsed_ms.map(|m| m as u64), + "error": message, + }), + ); +} + /// Manually add a meeting from an existing recording: a local audio/video file -/// or a URL (YouTube/streaming page, or a direct media URL). Shells out to -/// `ffmpeg` (transcode) and — for URLs — `yt-dlp` (both external, not bundled; -/// a missing tool is a clear error). The produced 16kHz-mono WAV becomes the -/// meeting's retained `audio.wav`, then goes through the same -/// transcription + diarization + finalize path as a live recording. +/// or a URL (YouTube/streaming page, or a direct media URL). Creates the meeting +/// row (status `transcribing`) and returns its id **immediately**; the heavy +/// transcode → transcribe → diarize → finalize work runs detached so the UI is +/// never blocked (a 25-min video can take 10+ min). Progress streams via +/// `import://progress` and completion via `transcript://finalized`. `model` +/// overrides the Settings whisper model for this one import (so the user picks +/// and can see what it was transcribed with); omit to use the Settings default. #[tauri::command] pub async fn import_media( app: AppHandle, state: State<'_, AppState>, source: String, title: Option, + model: Option, ) -> WaResult { let settings = load_settings(); - let model_id = model_id_for(&settings); + let model_id = model + .filter(|m| !m.trim().is_empty()) + .unwrap_or_else(|| model_id_for(&settings)); let backend = backend_for(&settings); let model_path = whisper_model_file(&model_id); if !model_path.exists() { @@ -2042,12 +2470,47 @@ pub async fn import_media( }) .await .map_err(|e| WaError::new("storage", e.to_string()))?; + // `create_meeting` starts rows as `recording`; mark this one `transcribing` + // so the meetings-list badge reads as an import in flight, not a live mic. + let _ = state + .store + .set_meeting_status(&meeting_id, "transcribing") + .await; + + // Run the pipeline detached and return now — the dialog closes and the + // meeting appears in the list with a live tracker fed by the events below. + let store = state.store.clone(); let dir = meeting_dir(&meeting_id); let wav_path = dir.join("audio.wav"); + let mid = meeting_id.clone(); + tauri::async_runtime::spawn(run_import_pipeline( + app, store, mid, source, wav_path, dir, backend, model_id, model_path, language, + )); + Ok(meeting_id) +} - // Transcode into the meeting's audio.wav off the async runtime (shells out - // to ffmpeg/yt-dlp). On any failure, drop the empty meeting so a bad import - // doesn't leave a husk row behind. +/// The detached body of `import_media`: transcode, transcribe, diarize, finalize, +/// emitting an `import://progress` tick at the start and end of each phase. On +/// the first failure it marks the meeting `error` (kept in the list, not +/// deleted, so the user sees the failed import) and stops. +#[allow(clippy::too_many_arguments)] +async fn run_import_pipeline( + app: AppHandle, + store: Arc, + meeting_id: MeetingId, + source: String, + wav_path: std::path::PathBuf, + dir: std::path::PathBuf, + backend: BackendId, + model_id: String, + model_path: std::path::PathBuf, + language: Option, +) { + use std::time::Instant; + + // Phase 1: prepare — transcode (and, for URLs, yt-dlp download) to audio.wav. + emit_import_progress(&app, &meeting_id, "prepare", "active", None, None); + let t = Instant::now(); let transcode = tauri::async_runtime::spawn_blocking({ let source = source.clone(); let wav_path = wav_path.clone(); @@ -2060,15 +2523,19 @@ pub async fn import_media( r } }) - .await - .map_err(|e| WaError::new("import", e.to_string()))?; - if let Err(e) = transcode { - let _ = state.store.delete_meeting(&meeting_id).await; - return Err(WaError::new("import", e.to_string())); + .await; + match transcode { + Ok(Ok(())) => { + emit_import_progress(&app, &meeting_id, "prepare", "done", Some(t.elapsed().as_millis()), None) + } + Ok(Err(e)) => return fail_import(&app, &store, &meeting_id, "prepare", &e.to_string()).await, + Err(e) => return fail_import(&app, &store, &meeting_id, "prepare", &e.to_string()).await, } - // Transcribe the produced WAV (same batch path as reprocess_transcript). - let (mut segments, resolved_language) = tauri::async_runtime::spawn_blocking({ + // Phase 2: transcribe (same batch path as reprocess_transcript). + emit_import_progress(&app, &meeting_id, "transcribe", "active", None, None); + let t = Instant::now(); + let transcribed = tauri::async_runtime::spawn_blocking({ let wav_path = wav_path.clone(); let model_path = model_path.clone(); let requested_language = language.clone(); @@ -2082,12 +2549,20 @@ pub async fn import_media( Ok::<_, crate::transcription::TrxError>((segments, resolved)) } }) - .await - .map_err(|e| WaError::new("transcription", e.to_string()))? - .map_err(|e| WaError::new("transcription", e.to_string()))?; + .await; + let (mut segments, resolved_language) = match transcribed { + Ok(Ok(v)) => { + emit_import_progress(&app, &meeting_id, "transcribe", "done", Some(t.elapsed().as_millis()), None); + v + } + Ok(Err(e)) => return fail_import(&app, &store, &meeting_id, "transcribe", &e.to_string()).await, + Err(e) => return fail_import(&app, &store, &meeting_id, "transcribe", &e.to_string()).await, + }; - // One diarization pass if the models are installed, exactly like - // stop_recording — otherwise every line stays the single "S1" placeholder. + // Phase 3: diarize — one pass if the models are installed, else every line + // stays the single "S1" placeholder (same as stop_recording). + emit_import_progress(&app, &meeting_id, "diarize", "active", None, None); + let t = Instant::now(); let diarizer: Option> = tauri::async_runtime::spawn_blocking(diarizer_from_installed_models) .await @@ -2103,15 +2578,18 @@ pub async fn import_media( Err(e) => tracing::warn!("import diarization task failed: {e}"), } } + emit_import_progress(&app, &meeting_id, "diarize", "done", Some(t.elapsed().as_millis()), None); + // Phase 4: finalize — persist segments, notes, seal audio. + emit_import_progress(&app, &meeting_id, "finalize", "active", None, None); + let t = Instant::now(); let speakers = speaker_infos_from_segments(&segments, &HashMap::new()); let duration_secs = segments .last() .map(|s| (s.end_ms / 1000) as i64) .unwrap_or(0); - state - .store + if let Err(e) = store .finalize_meeting( &meeting_id, FinalizeMeeting { @@ -2121,12 +2599,14 @@ pub async fn import_media( recorded: true, // the imported WAV is the recording — keep it language: resolved_language, backend_used: Some(backend.as_str().to_string()), - model_used: Some(model_id.clone()), + model_used: Some(effective_model_id(backend, &model_id)), audio_layout: Some("summed".to_string()), // single-source import }, ) .await - .map_err(|e| WaError::new("storage", e.to_string()))?; + { + return fail_import(&app, &store, &meeting_id, "finalize", &e.to_string()).await; + } let notes_md = crate::notes::MarkdownNotes.merge( &segments, @@ -2135,7 +2615,7 @@ pub async fn import_media( None, None, ); - let _ = state.store.update_notes(&meeting_id, ¬es_md).await; + let _ = store.update_notes(&meeting_id, ¬es_md).await; // Seal the retained recording at rest when the vault is unlocked (T8.8), // matching stop_recording so imports aren't left as plaintext outliers. @@ -2146,12 +2626,25 @@ pub async fn import_media( } } } + emit_import_progress(&app, &meeting_id, "finalize", "done", Some(t.elapsed().as_millis()), None); let _ = app.emit( "transcript://finalized", serde_json::json!({ "meetingId": meeting_id, "segmentCount": segments.len() }), ); - Ok(meeting_id) +} + +/// Mark a failed background import `error` (kept in the list) and emit the error +/// tick for the phase that failed. +async fn fail_import( + app: &AppHandle, + store: &Arc, + meeting_id: &str, + phase: &str, + message: &str, +) { + let _ = store.set_meeting_status(&meeting_id.to_string(), "error").await; + emit_import_progress(app, meeting_id, phase, "error", None, Some(message)); } /// Re-run transcription from a `recovering` meeting's working `audio.wav` @@ -3440,6 +3933,60 @@ pub async fn generate_tags( .map_err(|e| WaError::new("llm", e.to_string())) } +/// AI-enhance the user's rough notes into structured Markdown grounded in the +/// transcript (Granola-style). Reuses the configured `LlmProvider` (local by +/// default → no new egress). The caller passes the live editor buffer so we +/// enhance exactly what the user sees, not a possibly-stale saved copy; we +/// return the enhanced Markdown without persisting it — the UI decides to keep +/// or undo it. Refuses a still-recording meeting; errors clearly with no +/// provider configured. Invents nothing beyond the notes + transcript. +#[tauri::command] +pub async fn enhance_notes( + state: State<'_, AppState>, + meeting_id: MeetingId, + notes: String, +) -> WaResult { + let guard = state.session.lock().await; + if guard.as_ref().is_some_and(|s| s.meeting_id == meeting_id) { + return Err(WaError::new( + "llm", + "cannot enhance notes while this meeting is still recording — wait until it's stopped", + )); + } + drop(guard); + + let settings = load_settings(); + let provider = llm_provider_from_settings(&settings).ok_or_else(|| { + WaError::new( + "llm", + "no LLM provider is configured — enable one in Settings first", + ) + })?; + + let meeting = state + .store + .get_meeting(&meeting_id) + .await + .map_err(|e| WaError::new("storage", e.to_string()))?; + let prompt = build_prompt(&meeting, None); + + let system = "You expand a user's rough meeting notes into clear, well-structured Markdown. \ +Use ONLY facts stated in the transcript and the user's own notes — never invent details, names, \ +numbers, or decisions. Preserve the user's intent and any structure they started. Output Markdown \ +only, with no preamble or commentary."; + let notes = notes.trim(); + let user = format!( + "My rough notes:\n{}\n\nTranscript:\n{}", + if notes.is_empty() { "(none yet)" } else { notes }, + prompt.transcript + ); + provider + .complete(system, &user) + .await + .map(|s| s.trim().to_string()) + .map_err(|e| WaError::new("llm", e.to_string())) +} + // ---- Calendar / .pst (Phase 6) ---- /// Import events + attendees from a `.pst` (T6.1/T6.2, FR-CAL-1). The @@ -4929,6 +5476,49 @@ pub async fn privacy_self_check(state: State<'_, AppState>) -> WaResult StressResult { + StressResult { + backend: backend.into(), + model: model.into(), + rtf, + realtime: rtf < 1.0, + } + } + + #[test] + fn recommendation_picks_largest_realtime_model() { + let sizes = HashMap::from([ + ("tiny".to_string(), 32), + ("base".to_string(), 60), + ("small".to_string(), 190), + ]); + let results = vec![ + row("cpu", "tiny", 0.4), + row("cpu", "base", 0.9), + row("cpu", "small", 1.4), // too slow — excluded + row("vulkan", "small", 0.6), + ]; + let rec = pick_realtime_recommendation(&results, &sizes).unwrap(); + // small is the largest model that still runs in real time (on vulkan). + assert_eq!(rec.model, "small"); + assert_eq!(rec.backend, "vulkan"); + } + + #[test] + fn recommendation_breaks_size_ties_by_lowest_rtf() { + let sizes = HashMap::from([("base".to_string(), 60)]); + let results = vec![row("cpu", "base", 0.8), row("vulkan", "base", 0.3)]; + let rec = pick_realtime_recommendation(&results, &sizes).unwrap(); + assert_eq!(rec.backend, "vulkan"); // same model, more headroom + } + + #[test] + fn recommendation_is_none_when_nothing_is_realtime() { + let sizes = HashMap::from([("small".to_string(), 190)]); + let results = vec![row("cpu", "small", 1.2)]; + assert!(pick_realtime_recommendation(&results, &sizes).is_none()); + } + /// Extracts the real hosted DirectML bundle (LZMA2+BCJ 7z) and checks the /// DLL lands flat. Skips if the binary isn't present (e.g. a lean checkout). #[cfg(feature = "npu")] diff --git a/src-tauri/src/diarization/mod.rs b/src-tauri/src/diarization/mod.rs index bb06d6b..3f8e3cf 100644 --- a/src-tauri/src/diarization/mod.rs +++ b/src-tauri/src/diarization/mod.rs @@ -58,6 +58,55 @@ pub fn assign_by_overlap(segments: &mut [TranscriptSegment], spans: &[SpeakerSpa } } +/// Split-layout attribution (FR-SPK): decides per segment between "You" (mic +/// channel voice activity) and the far side's diarized speakers by comparing +/// the *total* voiced overlap on each channel, not by picking the single +/// longest span — a long far-side diarizer span could otherwise swallow a +/// segment the user spoke most of, showing their words under "Speaker N". +/// The mic channel is physically the user's voice alone, so channel evidence +/// outranks cluster evidence; ties go to "You" (mislabeling the user's own +/// words as someone else is the worse failure). A segment with no voiced +/// overlap on either channel keeps its prior label rather than guessing. +// ponytail: whole-segment labels — a segment genuinely containing both sides +// still gets one speaker; the upgrade path is transcribing each channel +// separately so segments can never mix voices. +pub fn assign_split( + segments: &mut [TranscriptSegment], + you_spans: &[(u64, u64)], + far_vad: &[(u64, u64)], + far_spans: &[SpeakerSpan], +) { + fn overlap(a0: u64, a1: u64, b0: u64, b1: u64) -> u64 { + a1.min(b1).saturating_sub(a0.max(b0)) + } + for seg in segments.iter_mut() { + let mic_ms: u64 = you_spans + .iter() + .map(|&(s, e)| overlap(seg.start_ms, seg.end_ms, s, e)) + .sum(); + let far_ms: u64 = far_vad + .iter() + .map(|&(s, e)| overlap(seg.start_ms, seg.end_ms, s, e)) + .sum(); + if mic_ms == 0 && far_ms == 0 { + continue; + } + if mic_ms >= far_ms { + seg.speaker = "You".to_string(); + } else if let Some(span) = far_spans + .iter() + .map(|sp| (overlap(seg.start_ms, seg.end_ms, sp.start_ms, sp.end_ms), sp)) + .filter(|(o, _)| *o > 0) + .max_by_key(|(o, _)| *o) + .map(|(_, sp)| sp) + { + seg.speaker = span.speaker.clone(); + } + // Far side voiced but no diarizer span overlaps (e.g. a sub-700ms span + // was filtered): keep the prior label rather than guess. + } +} + /// sherpa-onnx-backed diarizer: pyannote segmentation + speaker-embedding + /// fast clustering (ADR-0005, T4.1). `Diarize::compute` needs `&mut self`; it's /// wrapped in a `Mutex` to satisfy `Diarizer: Sync` — diarization is a @@ -210,6 +259,58 @@ mod overlap_tests { assign_by_overlap(&mut segments, &[]); assert_eq!(segments[0].speaker, "S1"); } + + #[test] + fn split_labels_a_mic_dominant_segment_you_even_against_a_longer_far_span() { + // The user spoke 0-4000ms; the far side 4000-6000ms — but the far + // cluster span covers the whole window, so the old merged max-overlap + // pick handed the entire segment (the user's words included) to the + // far speaker. Channel totals must side with the mic instead. + let mut segments = vec![segment(0, 6000)]; + let you = vec![(0u64, 4000u64)]; + let far_vad = vec![(4000u64, 6000u64)]; + let far_spans = vec![span(0, 6000, "S1")]; // long far cluster span + assign_split(&mut segments, &you, &far_vad, &far_spans); + assert_eq!(segments[0].speaker, "You"); + } + + #[test] + fn split_ties_go_to_you() { + let mut segments = vec![segment(0, 2000)]; + let you = vec![(0u64, 1000u64)]; + let far_vad = vec![(1000u64, 2000u64)]; + let far_spans = vec![span(1000, 2000, "S1")]; + assign_split(&mut segments, &you, &far_vad, &far_spans); + assert_eq!(segments[0].speaker, "You"); + } + + #[test] + fn split_assigns_the_best_far_span_when_the_far_side_dominates() { + let mut segments = vec![segment(0, 3000)]; + let you = vec![(0u64, 500u64)]; + let far_vad = vec![(500u64, 3000u64)]; + let far_spans = vec![span(500, 1000, "S1"), span(1000, 3000, "S2")]; + assign_split(&mut segments, &you, &far_vad, &far_spans); + assert_eq!(segments[0].speaker, "S2"); + } + + #[test] + fn split_keeps_the_prior_label_when_both_channels_are_silent() { + let mut segments = vec![segment(5000, 6000)]; + segments[0].speaker = "S9".to_string(); + assign_split(&mut segments, &[(0, 1000)], &[(0, 1000)], &[span(0, 1000, "S1")]); + assert_eq!(segments[0].speaker, "S9"); + } + + #[test] + fn split_keeps_the_prior_label_when_far_is_voiced_but_no_far_span_overlaps() { + // Far VAD hears speech but every diarizer span was filtered (sub-700ms): + // don't guess a label. + let mut segments = vec![segment(0, 1000)]; + segments[0].speaker = "S3".to_string(); + assign_split(&mut segments, &[], &[(0, 1000)], &[span(2000, 3000, "S1")]); + assert_eq!(segments[0].speaker, "S3"); + } } #[cfg(all(test, feature = "diarization"))] diff --git a/src-tauri/src/lib.rs b/src-tauri/src/lib.rs index 403cdd4..2840cca 100644 --- a/src-tauri/src/lib.rs +++ b/src-tauri/src/lib.rs @@ -28,7 +28,8 @@ pub mod vault; use std::path::PathBuf; use std::sync::{Arc, Mutex as StdMutex}; use std::thread::JoinHandle; -use tauri::tray::TrayIcon; +use tauri::menu::{Menu, MenuItem}; +use tauri::tray::{MouseButton, MouseButtonState, TrayIcon, TrayIconBuilder, TrayIconEvent}; use tauri::Manager; use tokio::sync::Mutex; @@ -174,10 +175,34 @@ pub fn run() { session: Mutex::new(None), }) .setup(move |app| { + // Single tray icon (the `trayIcon` in tauri.conf.json was removed so + // this is the only one). It carries a Show/Quit menu and, on + // left-click, restores the window — the always-available way back + // from "close to tray". let icon = tauri::image::Image::from_bytes(include_bytes!("../icons/tray.png"))?; - let tray = tauri::tray::TrayIconBuilder::new() + let show_item = MenuItem::with_id(app, "show", "Show WhispAssist", true, None::<&str>)?; + let quit_item = MenuItem::with_id(app, "quit", "Quit", true, None::<&str>)?; + let menu = Menu::with_items(app, &[&show_item, &quit_item])?; + let tray = TrayIconBuilder::new() .icon(icon) .tooltip("WhispAssist — idle") + .menu(&menu) + .show_menu_on_left_click(false) + .on_menu_event(|app, event| match event.id.as_ref() { + "show" => show_main_window(app), + "quit" => app.exit(0), + _ => {} + }) + .on_tray_icon_event(|tray, event| { + if let TrayIconEvent::Click { + button: MouseButton::Left, + button_state: MouseButtonState::Up, + .. + } = event + { + show_main_window(tray.app_handle()); + } + }) .build(app)?; app.manage(TrayHandle(tray)); @@ -321,6 +346,17 @@ pub fn run() { }); Ok(()) }) + // Close to tray (keep running in background): when the setting is on, + // the window X hides instead of quitting; the tray "Quit" is the real + // exit. Off → default behavior (closing the window quits the app). + .on_window_event(|window, event| { + if let tauri::WindowEvent::CloseRequested { api, .. } = event { + if commands::load_settings().close_to_tray { + api.prevent_close(); + let _ = window.hide(); + } + } + }) .invoke_handler(tauri::generate_handler![ commands::start_recording, commands::stop_recording, @@ -328,6 +364,7 @@ pub fn run() { commands::recording_playback_path, commands::pause_recording, commands::resume_recording, + commands::toggle_microphone_mute, commands::set_recording_retention, commands::acknowledge_recording_consent, commands::update_live_notes, @@ -340,6 +377,8 @@ pub fn run() { commands::list_input_devices, commands::set_preferred_backend, commands::set_auto_start, + commands::monitor_audio_level, + commands::stress_test_hardware, commands::list_models, commands::list_whisper_languages, commands::download_npu_package, @@ -368,6 +407,7 @@ pub fn run() { commands::generate_summary, commands::confirm_action_items, commands::generate_tags, + commands::enhance_notes, commands::llm_setup_suggestions, commands::pull_ollama_model, commands::import_pst, @@ -412,6 +452,16 @@ pub fn run() { .expect("error while running WhispAssist"); } +/// Restore the main window from the tray (show + unminimize + focus). Shared by +/// the tray left-click and the "Show WhispAssist" menu item. +fn show_main_window(app: &tauri::AppHandle) { + if let Some(w) = app.get_webview_window("main") { + let _ = w.show(); + let _ = w.unminimize(); + let _ = w.set_focus(); + } +} + /// Used by `commands.rs` to keep the tray tooltip honest about capture state (FR-CAP-4). pub(crate) fn update_tray_tooltip(app: &tauri::AppHandle, text: &str) { if let Some(tray) = app.try_state::() { diff --git a/src-tauri/src/models.rs b/src-tauri/src/models.rs index b3debe4..a83e959 100644 --- a/src-tauri/src/models.rs +++ b/src-tauri/src/models.rs @@ -360,6 +360,11 @@ pub struct Settings { /// enterprise deploy file may set this to `true` (see `deploy.rs`). #[serde(default)] pub auto_start: bool, + /// Closing the window hides WhispAssist to the system tray instead of + /// quitting, so it keeps running in the background (tray "Quit" really + /// exits). ON by default; the tray icon is the always-available way back. + #[serde(default = "default_true")] + pub close_to_tray: bool, } fn default_mcp_transport() -> String { diff --git a/src-tauri/src/storage/mod.rs b/src-tauri/src/storage/mod.rs index f7e8087..c9c93f1 100644 --- a/src-tauri/src/storage/mod.rs +++ b/src-tauri/src/storage/mod.rs @@ -276,6 +276,10 @@ pub trait Store: Send + Sync { ) -> Result, StoreError>; async fn get_meeting(&self, id: &MeetingId) -> Result; async fn delete_meeting(&self, id: &MeetingId) -> Result<(), StoreError>; + /// Overwrite a meeting's lifecycle `status` (e.g. mark a background import + /// `transcribing` while it runs, or `error` if it fails). `finalize_meeting` + /// is still the only path to `ready`. + async fn set_meeting_status(&self, id: &MeetingId, status: &str) -> Result<(), StoreError>; async fn update_notes(&self, id: &MeetingId, markdown: &str) -> Result<(), StoreError>; /// (Re)builds this meeting's FTS index row from the current title and /// whatever's on disk/in the DB for transcript/notes/summary/tags (Phase @@ -976,6 +980,16 @@ impl Store for SqliteStore { Ok(()) } + async fn set_meeting_status(&self, id: &MeetingId, status: &str) -> Result<(), StoreError> { + sqlx::query("UPDATE meetings SET status = ?, updated_at = ? WHERE id = ?") + .bind(status) + .bind(now_unix()) + .bind(id) + .execute(&self.pool) + .await?; + Ok(()) + } + async fn update_notes(&self, id: &MeetingId, markdown: &str) -> Result<(), StoreError> { write_artifact( &paths::meeting_dir(id).join("notes.md"), diff --git a/src-tauri/tauri.conf.json b/src-tauri/tauri.conf.json index 2508a86..98af576 100644 --- a/src-tauri/tauri.conf.json +++ b/src-tauri/tauri.conf.json @@ -1,7 +1,7 @@ { "$schema": "https://schema.tauri.app/config/2", "productName": "WhispAssist", - "version": "0.7.0", + "version": "0.7.2", "identifier": "bet.dou.whispassist", "build": { "frontendDist": "../dist", @@ -22,10 +22,6 @@ ], "security": { "csp": "default-src 'self'; connect-src 'self' http://localhost:* http://127.0.0.1:*; img-src 'self' data:; media-src 'self' http://waaudio.localhost; style-src 'self' 'unsafe-inline'" - }, - "trayIcon": { - "iconPath": "icons/tray.png", - "tooltip": "WhispAssist" } }, "bundle": { diff --git a/src/App.svelte b/src/App.svelte index 8baa9e3..127aa55 100644 --- a/src/App.svelte +++ b/src/App.svelte @@ -12,6 +12,7 @@ import { recording } from "./lib/stores/recording.svelte"; import { settings } from "./lib/stores/settings.svelte"; import { meetings } from "./lib/stores/meetings.svelte"; + import { imports } from "./lib/stores/imports.svelte"; import { calendar } from "./lib/stores/calendar.svelte"; import { api, type NoteTemplate, type CalendarEvent } from "./lib/api"; import { onMount } from "svelte"; @@ -24,6 +25,8 @@ Square, Trash2, FilePlus, + Mic, + MicOff, Settings as SettingsIcon, AlertTriangle, PanelLeftClose, @@ -131,6 +134,7 @@ recording.init(); settings.load(); meetings.init(); + imports.init(); // live background-import progress for the tracker calendar.load(); // events power the auto-record timer above checkVault(); api @@ -207,6 +211,18 @@ } else if (e.ctrlKey && e.key === ",") { e.preventDefault(); showSettings = !showSettings; + } else if ( + // Press "M" to mute/unmute the mic mid-meeting (FR-CAP-7). Bare key (no + // modifiers) and only while recording with the mic on. + e.key.toLowerCase() === "m" && + !e.ctrlKey && + !e.metaKey && + !e.altKey && + recording.state !== "idle" && + settings.settings.microphone_enabled + ) { + e.preventDefault(); + recording.toggleMute(); } } @@ -232,10 +248,16 @@
{recordingAnnouncement}
- WhispAssist - {t("app.tagline")} -
{#if recording.state === "idle"} + - - {:else} + {/if} +
+ {#if recording.state === "idle"} + + {:else} {t("app.recording")} @@ -289,6 +306,23 @@ micPeak={recording.levelPeakMic} showMic={settings.settings.microphone_enabled} /> + {#if settings.settings.microphone_enabled} + + {/if} {#if settings.hardware} {settings.hardware.active} {/if} @@ -588,10 +622,6 @@ border-bottom: 1px solid var(--border); background: var(--bg-elevated); } - .bar strong { - font-size: 0.95rem; - letter-spacing: -0.01em; - } .spacer { flex: 1; } @@ -718,6 +748,26 @@ border-radius: var(--radius-full); padding: 0.15rem 0.5rem; } + .mute-btn { + display: inline-flex; + align-items: center; + justify-content: center; + width: 30px; + height: 30px; + color: var(--fg); + background: var(--bg); + border: 1px solid var(--border); + border-radius: var(--radius-full); + cursor: pointer; + } + .mute-btn:hover { + background: var(--bg-hover); + } + .mute-btn.muted { + color: var(--danger, #d33); + border-color: var(--danger, #d33); + background: color-mix(in srgb, var(--danger, #d33) 12%, transparent); + } .retention { display: flex; align-items: center; diff --git a/src/lib/api.ts b/src/lib/api.ts index b4bf86e..f4bc49f 100644 --- a/src/lib/api.ts +++ b/src/lib/api.ts @@ -56,6 +56,19 @@ export interface AudioDeviceInfo { name: string; } +// Quick hardware stress test (Settings ▸ Hardware): per-(backend, model) +// real-time factor, plus the recommended real-time-capable pairing. +export interface StressResult { + backend: string; + model: string; + rtf: number; + realtime: boolean; +} +export interface StressTestResult { + results: StressResult[]; + recommended: { backend: string; model: string } | null; +} + export interface LlmStatus { provider: string; // ollama|custom|anthropic|off (ADR-0011; "openai" not yet wired) reachable: boolean; @@ -83,6 +96,19 @@ export interface LanguageOption { export type MeetingStatus = "recording" | "transcribing" | "ready" | "recovering" | "error"; +// The four ordered phases of a background media import (import://progress). +export type ImportPhase = "prepare" | "transcribe" | "diarize" | "finalize"; + +// One `import://progress` tick. `state` is active (running), done (finished, +// `elapsedMs` set) or error (`error` message set) for the given `phase`. +export interface ImportProgress { + meetingId: MeetingId; + phase: ImportPhase; + state: "active" | "done" | "error"; + elapsedMs: number | null; + error: string | null; +} + export interface MeetingListItem { id: MeetingId; title: string; @@ -339,6 +365,9 @@ export interface AppSettings { /** Launch WhispAssist at login (opt-in, off by default; NFR-RES-4). Toggled * via setAutoStart, which writes a per-user Run entry (no admin). */ auto_start: boolean; + /** Closing the window hides to the tray (keep running in background) instead + * of quitting; on by default. Tray "Quit" is the real exit. */ + close_to_tray: boolean; } // Feature brief — agent-ready spec distilled from a meeting (ADR-0011). @@ -401,6 +430,10 @@ export const api = { invoke("recording_playback_path", { meetingId }), pauseRecording: (meetingId: MeetingId) => invoke("pause_recording", { meetingId }), resumeRecording: (meetingId: MeetingId) => invoke("resume_recording", { meetingId }), + // Toggle mic mute for the active recording (FR-CAP-7); returns the new muted + // state. Errors if the meeting was started with the mic off. + toggleMicrophoneMute: (meetingId: MeetingId) => + invoke("toggle_microphone_mute", { meetingId }), setRecordingRetention: (meetingId: MeetingId, record: boolean) => invoke("set_recording_retention", { meetingId, record }), acknowledgeRecordingConsent: () => invoke("acknowledge_recording_consent"), @@ -421,6 +454,10 @@ export const api = { setPreferredBackend: (backend: BackendId | "auto") => invoke("set_preferred_backend", { args: { backend } }), setAutoStart: (enabled: boolean) => invoke("set_auto_start", { enabled }), + // Test a device: stream device://level for a few seconds. Resolves when done. + monitorAudioLevel: (kind: "input" | "loopback", deviceId: string | null, durationMs = 6000) => + invoke("monitor_audio_level", { kind, deviceId, durationMs }), + stressTestHardware: () => invoke("stress_test_hardware"), downloadNpuPackage: () => invoke("download_npu_package"), downloadDirectmlPackage: () => invoke("download_directml_package"), listModels: () => invoke("list_models"), @@ -441,10 +478,12 @@ export const api = { invoke("reprocess_transcript", { meetingId, model, language }), // Manually add a meeting from an existing recording — a local audio/video // file path or a URL (YouTube/streaming page or direct media URL). Requires - // ffmpeg (and yt-dlp for URLs) on PATH; neither is bundled. Returns the new - // meeting's id once transcription + diarization have finished. - importMedia: (source: string, title?: string) => - invoke("import_media", { source, title }), + // ffmpeg (and yt-dlp for URLs) on PATH; neither is bundled. `model` overrides + // the Settings whisper model for this one import. Returns the new meeting's id + // *immediately*; transcode/transcribe/diarize run in the background and stream + // `import://progress` ticks, finishing with `transcript://finalized`. + importMedia: (source: string, title?: string, model?: string) => + invoke("import_media", { source, title, model }), resumeTranscription: (meetingId: MeetingId) => invoke("resume_transcription", { meetingId }), listMeetings: (filter?: MeetingFilter) => @@ -464,6 +503,10 @@ export const api = { deleteMeeting: (meetingId: MeetingId) => invoke("delete_meeting", { meetingId }), updateNotes: (meetingId: MeetingId, markdown: string) => invoke("update_notes", { meetingId, markdown }), + // AI-enhance rough notes into structured Markdown grounded in the transcript + // (Granola-style). Returns the enhanced text; the caller decides to keep it. + enhanceNotes: (meetingId: MeetingId, notes: string) => + invoke("enhance_notes", { meetingId, notes }), // dest is a file path for md/pdf/docx/obsidian, a folder for bundle. // "obsidian" writes one self-contained vault note (no audio) — FR-STORE-4. exportMeeting: ( @@ -592,12 +635,19 @@ export const events = { onDeviceChanged: ( cb: (p: { meetingId: string; recovered: boolean; message: string }) => void, ): Promise => listen("recording://device", (e) => cb(e.payload as never)), + // Mic mute toggled for the active recording (FR-CAP-7). + onMicMuted: ( + cb: (p: { meetingId: string; muted: boolean }) => void, + ): Promise => listen("recording://mic", (e) => cb(e.payload as never)), onSegment: ( cb: (p: { meetingId: string; segment: TranscriptSegment }) => void, ): Promise => listen("transcript://segment", (e) => cb(e.payload as never)), onFinalized: ( cb: (p: { meetingId: string; segmentCount: number }) => void, ): Promise => listen("transcript://finalized", (e) => cb(e.payload as never)), + // Per-phase progress of a background media import (feeds the import tracker). + onImportProgress: (cb: (p: ImportProgress) => void): Promise => + listen("import://progress", (e) => cb(e.payload as never)), // Live diarization refined the speaker list mid-recording (FR-SPK): updated // labels/display names, including the mic speaker resolved to "You". onDiarizationUpdated: ( @@ -613,6 +663,15 @@ export const events = { onHardwareChanged: ( cb: (p: { active: BackendId; reason: string }) => void, ): Promise => listen("hardware://changed", (e) => cb(e.payload as never)), + // Live level meter for a device test (Settings ▸ Hardware). `done` marks the + // end of the monitor window. + onDeviceLevel: ( + cb: (p: { kind: string; rms?: number; peak?: number; done?: boolean }) => void, + ): Promise => listen("device://level", (e) => cb(e.payload as never)), + // Per-(backend, model) progress ticks during the quick stress test. + onStressProgress: ( + cb: (p: { backend: string; model: string }) => void, + ): Promise => listen("stress://progress", (e) => cb(e.payload as never)), onNpuDownload: ( cb: (p: { stage: "model" | "runtime" | "done"; diff --git a/src/lib/components/ImportMeeting.svelte b/src/lib/components/ImportMeeting.svelte index b298062..5779ed7 100644 --- a/src/lib/components/ImportMeeting.svelte +++ b/src/lib/components/ImportMeeting.svelte @@ -1,22 +1,43 @@ + +{#if run} +
+
+ {#if run.error} + {t("import.tracker.failed")} + {:else if run.done} + {t("import.tracker.done")} + {:else} + {t("import.tracker.running")} + {/if} +
+ +
    + {#each run.phases as p (p.phase)} + {@const Icon = ICONS[p.phase]} +
  1. +
    + {#if p.state === "done"} +
    +
    + {t(`import.phase.${p.phase}`)} + + {#if p.state === "done"}{fmtDur(p.elapsedMs)} + {:else if p.state === "active"}{t("import.tracker.working")} + {:else if p.state === "error"}{t("import.tracker.stopped")} + {/if} + +
    +
    2. + {/each} +
+ + {#if run.error} +

{run.error}

+ {/if} +
+{/if} + + diff --git a/src/lib/i18n/en.json b/src/lib/i18n/en.json index c95c339..f9a4bb0 100644 --- a/src/lib/i18n/en.json +++ b/src/lib/i18n/en.json @@ -25,6 +25,24 @@ "settings.hardware.title": "Hardware", "settings.hardware.refresh": "Refresh", + "settings.hardware.test_output": "Test", + "settings.hardware.test_mic": "Test", + "settings.hardware.testing": "Listening…", + "settings.hardware.play_tone": "Play tone", + "settings.hardware.stress_title": "Quick stress test", + "settings.hardware.stress_hint": "Benchmarks your installed models on each available backend and recommends the most accurate one that still keeps up with live speech. Takes a moment.", + "settings.hardware.stress_run": "Run stress test", + "settings.hardware.stress_running": "Running…", + "settings.hardware.stress_progress": "Benchmarking {pair}…", + "settings.hardware.stress_recommend": "Recommended: {backend} + {model}", + "settings.hardware.stress_apply": "Apply", + "settings.hardware.stress_none": "No installed model keeps up with live speech on this hardware — try a smaller model.", + "settings.hardware.stress_backend": "Backend", + "settings.hardware.stress_model": "Model", + "settings.hardware.stress_rtf": "Speed (×real-time)", + "settings.hardware.stress_realtime": "Live?", + "settings.hardware.stress_yes": "Yes", + "settings.hardware.stress_no": "No", "settings.hardware.active_backend": "Active backend", "settings.hardware.model_meta": "· model {size}", "settings.hardware.preferred_backend": "Preferred backend", @@ -279,6 +297,7 @@ "settings.privacy.locked_word": "locked", "settings.privacy.vault_locked_2": ". Unlock to read encrypted meetings.", "settings.privacy.password": "Password", + "settings.privacy.show_password": "Show password", "settings.privacy.unlock": "Unlock", "settings.privacy.vault_unlocked_1": "Vault is ", "settings.privacy.unlocked_word": "unlocked", @@ -321,12 +340,23 @@ "import.optional": "optional", "import.title_placeholder": "Defaults to the file name", "import.filter_av": "Audio / video", - "import.requires_1": "Requires", - "import.requires_2": "installed and on your PATH (plus", - "import.requires_3": "for URLs). WhispAssist doesn't bundle them.", - "import.importing": "Importing… this can take a while", + "import.model_label": "Transcription model", + "import.model_hint": "Recorded with the meeting so you can see how it was transcribed.", + "import.requires": "Needs these on your PATH (not bundled):", + "import.background_note": "Runs in the background — track it in the list.", + "import.importing": "Starting…", "import.import": "Import", "import.cancel": "Cancel", + "import.tracker.label": "Import progress", + "import.tracker.running": "Importing…", + "import.tracker.done": "Import complete", + "import.tracker.failed": "Import failed", + "import.tracker.working": "working…", + "import.tracker.stopped": "stopped", + "import.phase.prepare": "Transcode", + "import.phase.transcribe": "Transcribe", + "import.phase.diarize": "Identify speakers", + "import.phase.finalize": "Finalize", "tagchip.filter": "Filter meetings tagged \"{tag}\"", "tagchip.remove": "Remove tag {tag}", @@ -346,6 +376,10 @@ "app.cancel": "Cancel", "app.cancel_title": "Discard this recording and delete it", "app.recording": "Recording…", + "app.mute": "Mute microphone", + "app.unmute": "Unmute microphone", + "app.mute_title": "Mute microphone (M)", + "app.unmute_title": "Unmute microphone (M)", "app.backend_title": "Active transcription backend", "app.retention_title": "Save audio as .wav for this meeting", "app.saving": "saving", @@ -398,6 +432,7 @@ "transcript.heading": "Transcript", "transcript.title_aria": "Meeting title", + "transcript.transcribed_with": "Transcribed with", "transcript.lang_title": "Transcription language", "transcript.lang_auto": "auto-detecting…", "transcript.show": "Show transcript", @@ -429,9 +464,18 @@ "notes.italic": "Italic", "notes.h1": "Heading 1", "notes.h2": "Heading 2", + "notes.h3": "Heading 3", "notes.bullet": "Bullet list", + "notes.numbered": "Numbered list", + "notes.quote": "Quote", + "notes.divider": "Divider", "notes.checkbox_title": "Checkbox", "notes.checkbox_aria": "Checkbox list item", + "notes.enhance": "Enhance", + "notes.enhancing": "Enhancing…", + "notes.enhance_title": "Expand these notes into structured Markdown using the transcript (AI)", + "notes.enhanced_note": "Notes enhanced from the transcript.", + "notes.undo_enhance": "Undo", "notes.edit_raw": "Edit the raw markdown", "notes.render": "Render the markdown", "notes.editor": "Editor", @@ -530,5 +574,7 @@ "settings.recording.auto_label": "Auto-start recording when a calendar event begins", "settings.recording.auto_hint": "Only while WhispAssist is open. When an imported calendar event's start time arrives, a recording begins automatically (using your default retention setting above). Nothing runs in the background — the timer is armed only while the app is running. Import events under Settings → Calendar.", "settings.recording.autostart_label": "Launch WhispAssist at login", - "settings.recording.autostart_hint": "Starts WhispAssist automatically when you sign in to Windows. Off by default; installs a per-user startup entry (no admin required) and does not begin recording on its own." + "settings.recording.autostart_hint": "Starts WhispAssist automatically when you sign in to Windows. Off by default; installs a per-user startup entry (no admin required) and does not begin recording on its own.", + "settings.recording.close_tray_label": "Close to system tray", + "settings.recording.close_tray_hint": "Closing the window keeps WhispAssist running in the background instead of quitting. Reopen it from the tray icon; use the tray's Quit to exit fully. On by default." } diff --git a/src/lib/stores/imports.svelte.ts b/src/lib/stores/imports.svelte.ts new file mode 100644 index 0000000..56c9e55 --- /dev/null +++ b/src/lib/stores/imports.svelte.ts @@ -0,0 +1,63 @@ +// Live per-meeting progress of background media imports (feeds ImportTracker). +// Fed entirely by `import://progress` events emitted by `import_media`; kept in +// memory only (the meeting's `status` badge is the persistent story after a +// restart). See commands.rs `run_import_pipeline`. + +import { events, type ImportPhase, type ImportProgress, type MeetingId } from "../api"; + +export type PhaseState = "pending" | "active" | "done" | "error"; + +// The four phases in the order the backend runs (and the tracker renders) them. +export const IMPORT_PHASES: ImportPhase[] = ["prepare", "transcribe", "diarize", "finalize"]; + +export interface PhaseInfo { + phase: ImportPhase; + state: PhaseState; + elapsedMs: number | null; +} + +export interface ImportRun { + meetingId: MeetingId; + phases: PhaseInfo[]; + error: string | null; + done: boolean; +} + +function freshRun(meetingId: MeetingId): ImportRun { + return { + meetingId, + phases: IMPORT_PHASES.map((phase) => ({ phase, state: "pending", elapsedMs: null })), + error: null, + done: false, + }; +} + +class ImportsStore { + runs = $state>({}); + + get(meetingId: MeetingId): ImportRun | undefined { + return this.runs[meetingId]; + } + + async init() { + await events.onImportProgress((p) => this.apply(p)); + } + + private apply(p: ImportProgress) { + // Re-read through the record after inserting so we mutate the $state proxy, + // not the raw object (Svelte 5 deep reactivity only tracks the proxy). + if (!this.runs[p.meetingId]) this.runs[p.meetingId] = freshRun(p.meetingId); + const run = this.runs[p.meetingId]; + const info = run.phases.find((x) => x.phase === p.phase); + if (!info) return; + info.state = p.state; + if (p.state === "done") info.elapsedMs = p.elapsedMs; + if (p.state === "error") { + run.error = p.error; + run.done = true; + } + if (p.phase === "finalize" && p.state === "done") run.done = true; + } +} + +export const imports = new ImportsStore(); diff --git a/src/lib/stores/recording.svelte.ts b/src/lib/stores/recording.svelte.ts index a888432..d2bd8ed 100644 --- a/src/lib/stores/recording.svelte.ts +++ b/src/lib/stores/recording.svelte.ts @@ -22,6 +22,9 @@ class RecordingStore { * (FR-CAP-7); stays 0 when the mic is disabled or not recording. */ levelRmsMic = $state(0); levelPeakMic = $state(0); + /** Mic muted for the in-flight meeting (FR-CAP-7): mic channel goes silent + * while loopback keeps recording. Toggled by the "M" key / mute button. */ + micMuted = $state(false); /** Set while a capture-device reconnect is in progress; cleared on recovery (FR-CAP-6). */ deviceNotice = $state(null); /** Live notes redesign: freeform text typed in the Notes pane while recording. */ @@ -72,6 +75,22 @@ class RecordingStore { await events.onDeviceChanged(({ recovered, message }) => { this.deviceNotice = recovered ? null : message; }); + // Keep mute state in sync even if it was toggled elsewhere (e.g. a future + // tray control), not just from this store's toggleMute(). + await events.onMicMuted(({ muted }) => { + this.micMuted = muted; + }); + } + + /** Toggle mic mute for the active recording (FR-CAP-7); no-op if not + * recording. Optimistically flips, then reconciles with the backend result. */ + async toggleMute() { + if (!this.meetingId || this.state === "idle") return; + try { + this.micMuted = await api.toggleMicrophoneMute(this.meetingId); + } catch { + // Mic off for this meeting (or capture gone) — nothing to mute. + } } async start(title?: string, record = false, templateId?: string, calendarEventId?: string) { @@ -79,6 +98,7 @@ class RecordingStore { this.speakers = []; this.retention = record; this.deviceNotice = null; + this.micMuted = false; this.notesText = ""; this.segmentNotes.clear(); // T8.7/FR-TRX-4: whatever language is currently configured in Settings @@ -98,6 +118,7 @@ class RecordingStore { this.levelPeak = 0; this.levelRmsMic = 0; this.levelPeakMic = 0; + this.micMuted = false; this.deviceNotice = null; } @@ -116,6 +137,7 @@ class RecordingStore { this.levelPeak = 0; this.levelRmsMic = 0; this.levelPeakMic = 0; + this.micMuted = false; this.deviceNotice = null; this.notesText = ""; this.segmentNotes.clear(); diff --git a/src/lib/stores/settings.svelte.ts b/src/lib/stores/settings.svelte.ts index 47c3fe5..ea50e99 100644 --- a/src/lib/stores/settings.svelte.ts +++ b/src/lib/stores/settings.svelte.ts @@ -49,6 +49,7 @@ const DEFAULT_SETTINGS: AppSettings = { microphone_enabled: true, // capture the user's mic into the transcript (FR-CAP-7) audio_input_device: null, // system default capture device auto_start: false, // launch at login — opt-in, off by default (NFR-RES-4) + close_to_tray: true, // closing the window hides to tray; on by default }; class SettingsStore { diff --git a/src/lib/views/Settings.svelte b/src/lib/views/Settings.svelte index 1b7c311..360716b 100644 --- a/src/lib/views/Settings.svelte +++ b/src/lib/views/Settings.svelte @@ -9,9 +9,16 @@ import { t, i18n, LOCALES } from "../i18n/index.svelte"; import ConsentNotice from "../components/ConsentNotice.svelte"; import HostedAiBanner from "../components/HostedAiBanner.svelte"; + import LevelMeter from "../components/LevelMeter.svelte"; import { open } from "@tauri-apps/plugin-dialog"; import { api, errorMessage, events } from "../api"; - import type { BackendId, SyncKind, SyncTargetConfig, SyncTargetInfo } from "../api"; + import type { + BackendId, + StressTestResult, + SyncKind, + SyncTargetConfig, + SyncTargetInfo, + } from "../api"; import { trapFocus } from "../actions/trapFocus"; import { X, @@ -23,7 +30,11 @@ CalendarDays, UploadCloud, ShieldCheck, + Lock, + LockOpen, Sparkles, + Volume2, + Zap, RefreshCw, ChevronRight, RotateCcw, @@ -381,11 +392,98 @@ let showConsent = $state(false); let testResult = $state<{ ok: boolean; message: string } | null>(null); + // ---- Audio device test (live level meter) ---- + let monitorKind = $state<"input" | "loopback" | null>(null); + let monitorRms = $state(0); + let monitorPeak = $state(0); + let monitorUnlisten: (() => void) | null = null; + function stopMonitor() { + monitorUnlisten?.(); + monitorUnlisten = null; + monitorKind = null; + monitorRms = 0; + monitorPeak = 0; + } + async function testDevice(kind: "input" | "loopback") { + if (monitorKind) return; + monitorKind = kind; + monitorRms = 0; + monitorPeak = 0; + monitorUnlisten = await events.onDeviceLevel((p) => { + if (p.kind !== kind) return; + if (p.done) { + stopMonitor(); + return; + } + monitorRms = p.rms ?? 0; + monitorPeak = p.peak ?? 0; + }); + const deviceId = + kind === "input" + ? (settings.settings.audio_input_device ?? null) + : (settings.settings.audio_output_device ?? null); + try { + await api.monitorAudioLevel(kind, deviceId, 6000); + } catch (e) { + testResult = { ok: false, message: errorMessage(e) }; + } finally { + stopMonitor(); + } + } + // A 440Hz beep to the default output so the user can confirm speakers work. + function playTone() { + try { + const ctx = new AudioContext(); + const osc = ctx.createOscillator(); + const gain = ctx.createGain(); + osc.frequency.value = 440; + gain.gain.value = 0.15; + osc.connect(gain).connect(ctx.destination); + osc.start(); + osc.stop(ctx.currentTime + 0.5); + osc.onended = () => ctx.close(); + } catch { + /* no Web Audio available */ + } + } + + // ---- Quick hardware stress test ---- + let stressRunning = $state(false); + let stressProgress = $state(null); + let stressResult = $state(null); + let stressError = $state(null); + async function runStressTest() { + if (stressRunning) return; + stressRunning = true; + stressError = null; + stressResult = null; + const un = await events.onStressProgress((p) => { + stressProgress = `${p.backend} · ${p.model}`; + }); + try { + stressResult = await api.stressTestHardware(); + } catch (e) { + stressError = errorMessage(e); + } finally { + un(); + stressProgress = null; + stressRunning = false; + } + } + async function applyRecommendation() { + const r = stressResult?.recommended; + if (!r) return; + await settings.setPreferredBackend(r.backend as BackendId | "auto"); + await settings.patch({ whisper_model: r.model }); + } + // ---- At-rest encryption vault (T8.8, FR-SEC-3) ---- let vault = $state<{ enabled: boolean; unlocked: boolean } | null>(null); let vaultPw = $state(""); let vaultPw2 = $state(""); let vaultMsg = $state(null); + let vaultMsgError = $state(false); + let showVaultPw = $state(false); async function loadVault() { try { vault = await api.vaultStatus(); @@ -394,42 +492,46 @@ } } onMount(loadVault); + function setVaultMsg(msg: string | null, isError = false) { + vaultMsg = msg; + vaultMsgError = isError; + } async function enableVault() { - vaultMsg = null; + setVaultMsg(null); try { await api.enableVault(vaultPw); vaultPw = ""; - vaultMsg = "Vault enabled and unlocked."; + setVaultMsg("Vault enabled and unlocked."); await loadVault(); } catch (e) { - vaultMsg = errorMessage(e); + setVaultMsg(errorMessage(e), true); } } async function unlockVault() { - vaultMsg = null; + setVaultMsg(null); try { await api.unlockVault(vaultPw); vaultPw = ""; - vaultMsg = "Unlocked."; + setVaultMsg("Unlocked."); await loadVault(); } catch (e) { - vaultMsg = errorMessage(e); + setVaultMsg(errorMessage(e), true); } } async function lockVault() { await api.lockVault(); - vaultMsg = "Locked."; + setVaultMsg("Locked."); await loadVault(); } async function changeVaultPassword() { - vaultMsg = null; + setVaultMsg(null); try { await api.changeVaultPassword(vaultPw, vaultPw2); vaultPw = ""; vaultPw2 = ""; - vaultMsg = "Password changed."; + setVaultMsg("Password changed."); } catch (e) { - vaultMsg = errorMessage(e); + setVaultMsg(errorMessage(e), true); } } @@ -764,6 +866,17 @@

{t("settings.recording.autostart_hint")}

+ +

{t("settings.recording.close_tray_hint")}

+ {#if showConsent} (showConsent = false)} /> {/if} @@ -820,6 +933,20 @@

{t("settings.hardware.recording_device_hint")}

+
+ + + {#if monitorKind === "loopback"} + + {/if} +

{t("settings.hardware.mic_hint")}

+
+ + {#if monitorKind === "input"} + + {/if} +
+ +

{t("settings.hardware.stress_title")}

+

{t("settings.hardware.stress_hint")}

+ + {#if stressProgress} +

{t("settings.hardware.stress_progress", { pair: stressProgress })}

+ {/if} + {#if stressError}

{stressError}

{/if} + {#if stressResult} + {#if stressResult.recommended} +
+
+ {:else} +

{t("settings.hardware.stress_none")}

+ {/if} + + + + + + + + + + + {#each stressResult.results as r (r.backend + r.model)} + + + + + + + {/each} + +
{t("settings.hardware.stress_backend")}{t("settings.hardware.stress_model")}{t("settings.hardware.stress_rtf")}{t("settings.hardware.stress_realtime")}
{r.backend}{r.model}{r.rtf.toFixed(2)}× + {#if r.realtime} +
+ {/if} {#if settings.hardware.npu?.present} {@const npu = settings.hardware.npu} @@ -1934,67 +2130,118 @@ {/if} {#if vault} -

{t("settings.privacy.vault_title")}

- {#if !vault.enabled} -

{t("settings.privacy.vault_intro")}

-
- +
+
+ {#if !vault.enabled} +
- -

{t("settings.privacy.vault_pw_hint")}

- {:else if !vault.unlocked} -

- {t("settings.privacy.vault_locked_1")}{t("settings.privacy.locked_word")}{t("settings.privacy.vault_locked_2")} -

-
-
{/if} {:else if section === "language"} @@ -2445,6 +2692,115 @@ color: var(--accent, #2563eb); border-color: currentColor; } + .vault-card { + margin-top: 0.6rem; + padding: 0.85rem 1rem; + border: 1px solid var(--border); + border-radius: var(--radius-md, 8px); + background: var(--bg-elevated); + display: flex; + flex-direction: column; + gap: 0.6rem; + } + .vault-card.locked { + border-color: color-mix(in srgb, var(--accent) 45%, var(--border)); + } + .vault-card.unlocked { + border-color: color-mix(in srgb, var(--success, #16a34a) 45%, var(--border)); + } + .vault-head { + display: flex; + align-items: center; + gap: 0.5rem; + } + .vault-head h4 { + margin: 0; + } + .vault-head :global(svg) { + color: var(--muted); + } + .vault-card.locked .vault-head :global(svg) { + color: var(--accent); + } + .vault-card.unlocked .vault-head :global(svg) { + color: var(--success, #16a34a); + } + .vault-head .badge { + margin-left: auto; + } + .pw-row { + display: flex; + align-items: center; + gap: 0.4rem; + max-width: 22rem; + } + .pw-row input { + flex: 1; + } + .pw-toggle { + flex: none; + } + .vault-card .hint { + margin: 0; + } + .vault-msg { + margin: 0; + font-size: 0.85rem; + color: var(--success, #16a34a); + } + .vault-msg.error { + color: var(--danger); + } + .device-test { + display: flex; + align-items: center; + gap: 0.5rem; + flex-wrap: wrap; + margin: 0.35rem 0 0.6rem; + } + .device-test :global(.meter) { + flex: 1; + min-width: 8rem; + } + .stress-rec { + display: flex; + align-items: center; + gap: 0.5rem; + margin: 0.6rem 0; + padding: 0.55rem 0.75rem; + border: 1px solid color-mix(in srgb, var(--accent) 40%, var(--border)); + border-radius: var(--radius-md, 8px); + background: color-mix(in srgb, var(--accent) 10%, var(--bg)); + } + .stress-rec :global(svg) { + color: var(--accent); + } + .stress-rec span { + flex: 1; + } + .stress-table { + width: 100%; + border-collapse: collapse; + margin-top: 0.5rem; + font-size: 0.82rem; + } + .stress-table th, + .stress-table td { + text-align: left; + padding: 0.3rem 0.5rem; + border-bottom: 1px solid var(--border); + } + .stress-table th { + color: var(--muted); + font-weight: 600; + } + .stress-table .num { + font-variant-numeric: tabular-nums; + text-align: right; + } + .err { + color: var(--danger); + } .npu-package { margin-top: 0.6rem; padding: 0.6rem 0.75rem; diff --git a/src/lib/views/SummaryPanel.svelte b/src/lib/views/SummaryPanel.svelte index a95f381..83429b3 100644 --- a/src/lib/views/SummaryPanel.svelte +++ b/src/lib/views/SummaryPanel.svelte @@ -64,7 +64,7 @@ // nonce so re-clicking the same segment still jumps). Reading seekNonce is // what makes this effect re-run. $effect(() => { - player.seekNonce; + void player.seekNonce; const ms = player.seekMs; if (ms == null || !audioEl) return; audioEl.currentTime = ms / 1000; @@ -770,47 +770,54 @@ {#if editableItems.length === 0}

{t("summary.ai_empty")}

{:else} +
    {#each editableItems as item, i (i)}
  • - - - (item.owner = (e.target as HTMLInputElement).value || null)} - placeholder={t("summary.owner")} - aria-label={t("summary.owner")} - /> - onDueDateChange(item, (e.target as HTMLInputElement).value)} - /> - - +
    + + + +
    +
    + (item.owner = (e.target as HTMLInputElement).value || null)} + placeholder={t("summary.owner")} + aria-label={t("summary.owner")} + /> + onDueDateChange(item, (e.target as HTMLInputElement).value)} + /> + +
    • {/each}
@@ -1080,9 +1087,9 @@ } ul.action-items li { display: flex; - align-items: center; + flex-direction: column; gap: 0.35rem; - padding: 0.25rem 0; + padding: 0.45rem 0; border-bottom: 1px solid var(--border); } ul.action-items label { @@ -1090,14 +1097,28 @@ align-items: center; gap: 0.4rem; } + .ai-main { + display: flex; + align-items: center; + gap: 0.35rem; + } .ai-text { flex: 1; min-width: 0; - font-size: 0.82rem; + font-size: 0.85rem; + } + /* Meta row indented under the text (past the confirm checkbox), wrapping + rather than crushing its inputs when the pane is narrow. */ + .ai-meta { + display: flex; + align-items: center; + flex-wrap: wrap; + gap: 0.35rem; + padding-left: 1.4rem; } .ai-owner { - flex: none; - width: 5rem; + flex: 1; + min-width: 5rem; font-size: 0.75rem; } .ai-del { diff --git a/src/lib/views/TranscriptNotes.svelte b/src/lib/views/TranscriptNotes.svelte index e34fa3d..7e33d31 100644 --- a/src/lib/views/TranscriptNotes.svelte +++ b/src/lib/views/TranscriptNotes.svelte @@ -6,19 +6,27 @@ import { meetings } from "../stores/meetings.svelte"; import { settings } from "../stores/settings.svelte"; import { player } from "../stores/player.svelte"; - import { api, type SpeakerInfo } from "../api"; + import { api, errorMessage, type SpeakerInfo } from "../api"; import { t } from "../i18n/index.svelte"; import { renderMarkdown } from "../markdown"; import { save, open } from "@tauri-apps/plugin-dialog"; import { layout, clamp } from "../stores/layout.svelte"; + import { imports } from "../stores/imports.svelte"; import Splitter from "../components/Splitter.svelte"; + import ImportTracker from "../components/ImportTracker.svelte"; import { Bold, Italic, Heading1, Heading2, + Heading3, List, + ListOrdered, ListChecks, + Quote, + Minus, + Sparkles, + Undo2, FileDown, FileText, FolderOutput, @@ -66,6 +74,10 @@ let editorEl: HTMLTextAreaElement | undefined = $state(); let saveTimer: ReturnType | undefined; let loadedForId: string | null = null; + // The server copy the buffer was last synced against — lets the effect below + // tell a server-side notes change (speaker rename, reprocess) apart from the + // user's own unsaved edits. + let lastServerNotes: string | null = null; // Transcript/notes split (FR-UX-1): resizable (drag the Splitter) and // each side independently hideable, shared across the finalized-meeting @@ -103,14 +115,23 @@ selectedSegmentMs = null; }); - // Sync the editor buffer whenever a different meeting is selected. + // Sync the editor buffer whenever a different meeting is selected — and when + // the *server* copy of the same meeting's notes changes underneath us (a + // speaker rename rewrites notes.md's dialogue tags, reprocess regenerates it). + // A buffer with unsaved local edits is never clobbered: it only adopts the + // server copy when it still equals the last-synced one. $effect(() => { const m = meetings.selected; if (m && m.id !== loadedForId) { notesText = m.notes_markdown; + lastServerNotes = m.notes_markdown; loadedForId = m.id; + } else if (m && m.id === loadedForId && m.notes_markdown !== lastServerNotes) { + if (notesText === lastServerNotes) notesText = m.notes_markdown; + lastServerNotes = m.notes_markdown; } else if (!m) { loadedForId = null; + lastServerNotes = null; } }); @@ -158,6 +179,70 @@ scheduleSave(); } + // Slash commands: typing "/todo" (etc.) at the start of a line and pressing + // Space/Enter swaps it for the matching Markdown prefix. Reuses the same + // line-prefix model as the toolbar buttons — no rich inline menu. + // ponytail: line-prefix slash only; add a picker popover if users ask. + const SLASH_COMMANDS: Record = { + h1: "# ", + h2: "## ", + h3: "### ", + todo: "- [ ] ", + bullet: "- ", + num: "1. ", + quote: "> ", + divider: "---\n", + }; + function handleNotesKeydown(e: KeyboardEvent) { + if (e.key !== "Enter" && e.key !== " ") return; + const el = editorEl; + if (!el) return; + const { selectionStart: s, value } = el; + const lineStart = value.lastIndexOf("\n", s - 1) + 1; + const match = /^\/(\w+)$/.exec(value.slice(lineStart, s)); + if (!match) return; + const prefix = SLASH_COMMANDS[match[1].toLowerCase()]; + if (prefix === undefined) return; + e.preventDefault(); + const head = value.slice(0, lineStart) + prefix; + notesText = head + value.slice(s); + queueMicrotask(() => { + el.focus(); + el.selectionStart = el.selectionEnd = head.length; + }); + scheduleSave(); + } + + // AI-enhance (Granola-style): expand the user's rough notes into structured + // Markdown grounded in the transcript, via the configured LlmProvider (local + // by default, no new egress). Keeps a one-step Undo so we never silently lose + // what the user typed. + let enhancing = $state(false); + let enhanceError = $state(null); + let notesBeforeEnhance = $state(null); + async function enhanceNotes() { + const m = meetings.selected; + if (!m || enhancing) return; + enhancing = true; + enhanceError = null; + try { + const enhanced = await api.enhanceNotes(m.id, notesText); + notesBeforeEnhance = notesText; + notesText = enhanced; + scheduleSave(); + } catch (e) { + enhanceError = errorMessage(e); + } finally { + enhancing = false; + } + } + function undoEnhance() { + if (notesBeforeEnhance === null) return; + notesText = notesBeforeEnhance; + notesBeforeEnhance = null; + scheduleSave(); + } + async function exportMd() { const m = meetings.selected; if (!m) return; @@ -247,6 +332,15 @@ reprocessing = true; try { await meetings.reprocess(m.id, reprocessModel, reprocessLanguage || undefined); + // Re-transcribe rebuilds notes.md server-side (merging saved manual notes), + // but the meeting stays selected (same id), so the buffer-sync $effect — + // which only fires on an id change — won't pick it up. Resync explicitly so + // the notes pane updates in place instead of only after a restart. + const updated = meetings.selected; + if (updated && updated.id === m.id) { + notesText = updated.notes_markdown; + loadedForId = updated.id; + } } finally { reprocessing = false; } @@ -262,14 +356,23 @@ onchange={onTitleChange} aria-label={t("transcript.title_aria")} /> + {#if m.status === "transcribing" || imports.get(m.id)} +
+ {:else if m.model_used} +

+ {t("transcript.transcribed_with")} + {m.model_used}{#if m.backend_used} + · {m.backend_used}{/if} +

+ {/if}
- + +
+ {#if enhanceError} + + {:else if notesBeforeEnhance !== null} +
+ {t("notes.enhanced_note")} + +
+ {/if}
{#if notesPreview} @@ -468,6 +619,7 @@ bind:this={editorEl} bind:value={notesText} oninput={scheduleSave} + onkeydown={handleNotesKeydown} placeholder={t("notes.placeholder")} > {/if} @@ -610,6 +762,21 @@ background: var(--border); outline: none; } + .import-strip { + flex: none; + padding: 0 1rem 0.5rem; + } + .engine-meta { + flex: none; + margin: 0; + padding: 0 1rem 0.4rem; + font-size: 0.75rem; + color: var(--muted); + } + .engine-meta strong { + font-weight: 600; + color: var(--fg); + } .pad { padding: 1rem; max-width: 760px; @@ -842,6 +1009,40 @@ .toolbar .spacer { flex: 1; } + .toolbar .enhance { + color: var(--accent); + border-color: color-mix(in srgb, var(--accent) 40%, var(--border)); + font-weight: 600; + } + .toolbar .enhance:hover:not(:disabled) { + background: color-mix(in srgb, var(--accent) 12%, var(--bg)); + } + .toolbar .enhance:disabled { + opacity: 0.6; + cursor: default; + } + .enhance-bar { + display: flex; + align-items: center; + gap: 0.5rem; + margin-bottom: 0.5rem; + font-size: 0.8rem; + color: var(--muted); + } + .enhance-bar.error { + color: var(--danger); + } + .enhance-bar .link { + display: inline-flex; + align-items: center; + gap: 0.25rem; + background: none; + border: none; + color: var(--accent); + cursor: pointer; + font-size: 0.8rem; + padding: 0; + } .editor-preview { height: calc(100% - 2.5rem);