69 lines
3.6 KiB
Markdown
69 lines
3.6 KiB
Markdown
# 00 — Overview
|
|
|
|
## Vision
|
|
|
|
WhispAssist (WA) is a privacy-first, Windows-native meeting assistant that runs **entirely on the
|
|
local machine**. It captures system audio, transcribes it with Whisper-class models using
|
|
on-device acceleration (NPU → GPU → CPU), structures the result into Markdown notes, and
|
|
optionally augments those notes with a locally hosted LLM. It combines **Meetily's** local-only
|
|
architecture with **Granola's** calendar-aware, role-aware note workflows, and leans harder into
|
|
Windows hardware acceleration and Outlook `.pst` integration than either.
|
|
|
|
## Product goals
|
|
|
|
1. High-quality, real-time or near-real-time transcription, fully on-device.
|
|
2. Strict local-only data handling — recordings and transcripts never leave the machine except by
|
|
explicit user export.
|
|
3. Intelligent note-taking and summarization via a locally configured LLM endpoint.
|
|
4. Calendar and local Outlook `.pst` integration for meeting context and speaker mapping.
|
|
5. A modern, responsive desktop experience: light/dark themes, fast startup, low-latency
|
|
interaction, and near-zero idle footprint.
|
|
|
|
## Non-goals (for v1)
|
|
|
|
- Cloud sync, multi-device, or team sharing of recordings/transcripts.
|
|
- Joining meetings as a bot.
|
|
- Mobile apps (Windows-on-ARM is a "later", not a v1 target).
|
|
- Cloud AI for transcription or summarization (a user *may* point the LLM at a remote
|
|
OpenAI-compatible endpoint, but that is off by default and clearly labeled).
|
|
|
|
## How to read this plan
|
|
|
|
| Doc | Answers |
|
|
|---|---|
|
|
| `00-overview.md` (this) | What and why, at a glance; glossary |
|
|
| `01-requirements.md` | Exactly what it must do (FR) and how well (NFR), with IDs |
|
|
| `02-architecture.md` | How the pieces fit; data flow; threading |
|
|
| `03-data-model.md` | Database schema, file layout, transcript JSON |
|
|
| `04-api-contracts.md` | Tauri commands/events + internal Rust traits |
|
|
| `05-roadmap.md` | Build order: 8 phases, tasks, acceptance criteria |
|
|
| `06-test-strategy.md` | How each phase is proven correct |
|
|
| `07-research-findings.md` | Evidence the stack works (with sources) |
|
|
| `adr/*` | The big decisions and their trade-offs |
|
|
|
|
Requirement IDs (e.g. `FR-CAP-1`) are referenced throughout the roadmap, tests, and code commits
|
|
so any line of work traces back to a requirement.
|
|
|
|
## Glossary
|
|
|
|
- **System / loopback audio** — the audio the OS plays back (what you hear), captured without a
|
|
microphone or meeting bot, via WASAPI loopback.
|
|
- **Backend** — the hardware execution path for inference (NPU, NVIDIA, AMD, Intel, CPU).
|
|
- **Segment** — a timestamped chunk of transcript text with an associated speaker ID.
|
|
- **Speaker ID** — an internal label (`S1`, `S2`, …) assigned by diarization, later mapped to a
|
|
human name or calendar participant.
|
|
- **Diarization** — partitioning audio by "who spoke when".
|
|
- **Real-time factor (RTF)** — processing time ÷ audio duration; < 1.0 means faster than real time.
|
|
- **Provider** — a configured local LLM endpoint (Ollama by default).
|
|
- **Meeting** — the top-level record: audio + transcript + notes + metadata + speakers.
|
|
|
|
## One-paragraph architecture
|
|
|
|
A **Svelte** frontend in a **WebView2** window talks over Tauri IPC to a **Rust core** split into
|
|
independent services — `audio` (WASAPI capture), `hardware` (backend detection), `transcription`
|
|
(whisper.cpp / ONNX), `diarization` (sherpa-onnx), `storage` (SQLite + files), `llm` (Ollama
|
|
HTTP), `calendar` (`.pst`/future Graph), and `notes`. Capture writes audio to disk first (source
|
|
of truth), streams it to transcription, which emits segments to the UI live; on stop, diarization
|
|
and (optionally) the LLM refine and summarize, and everything is persisted locally. See
|
|
`02-architecture.md`.
|