WhispAssist/docs/00-overview.md

00 — Overview

Vision

WhispAssist (WA) is a privacy-first, Windows-native meeting assistant that runs entirely on the local machine. It captures system audio, transcribes it with Whisper-class models using on-device acceleration (NPU → GPU → CPU), structures the result into Markdown notes, and optionally augments those notes with a locally hosted LLM. It combines Meetily's local-only architecture with Granola's calendar-aware, role-aware note workflows, and leans harder into Windows hardware acceleration and Outlook .pst integration than either.

Product goals

1. High-quality, real-time or near-real-time transcription, fully on-device.
2. Strict local-only data handling — recordings and transcripts never leave the machine except by explicit user export.
3. Intelligent note-taking and summarization via a locally configured LLM endpoint.
4. Calendar and local Outlook .pst integration for meeting context and speaker mapping.
5. A modern, responsive desktop experience: light/dark themes, fast startup, low-latency interaction, and near-zero idle footprint.

Non-goals (for v1)

• Cloud sync, multi-device, or team sharing of recordings/transcripts.
• Joining meetings as a bot.
• Mobile apps (Windows-on-ARM is a "later", not a v1 target).
• Cloud AI for transcription or summarization (a user may point the LLM at a remote OpenAI-compatible endpoint, but that is off by default and clearly labeled).

How to read this plan

Doc	Answers
00-overview.md (this)	What and why, at a glance; glossary
01-requirements.md	Exactly what it must do (FR) and how well (NFR), with IDs
02-architecture.md	How the pieces fit; data flow; threading
03-data-model.md	Database schema, file layout, transcript JSON
04-api-contracts.md	Tauri commands/events + internal Rust traits
05-roadmap.md	Build order: 8 phases, tasks, acceptance criteria
06-test-strategy.md	How each phase is proven correct
07-research-findings.md	Evidence the stack works (with sources)
adr/*	The big decisions and their trade-offs

Requirement IDs (e.g. FR-CAP-1) are referenced throughout the roadmap, tests, and code commits so any line of work traces back to a requirement.

Glossary

• System / loopback audio — the audio the OS plays back (what you hear), captured without a microphone or meeting bot, via WASAPI loopback.
• Backend — the hardware execution path for inference (NPU, NVIDIA, AMD, Intel, CPU).
• Segment — a timestamped chunk of transcript text with an associated speaker ID.
• Speaker ID — an internal label (S1, S2, …) assigned by diarization, later mapped to a human name or calendar participant.
• Diarization — partitioning audio by "who spoke when".
• Real-time factor (RTF) — processing time ÷ audio duration; < 1.0 means faster than real time.
• Provider — a configured local LLM endpoint (Ollama by default).
• Meeting — the top-level record: audio + transcript + notes + metadata + speakers.

One-paragraph architecture

A Svelte frontend in a WebView2 window talks over Tauri IPC to a Rust core split into independent services — audio (WASAPI capture), hardware (backend detection), transcription (whisper.cpp / ONNX), diarization (sherpa-onnx), storage (SQLite + files), llm (Ollama HTTP), calendar (.pst/future Graph), and notes. Capture writes audio to disk first (source of truth), streams it to transcription, which emits segments to the UI live; on stop, diarization and (optionally) the LLM refine and summarize, and everything is persisted locally. See 02-architecture.md.