92 lines
5.2 KiB
Markdown
92 lines
5.2 KiB
Markdown
# WhispAssist (WA)
|
||
|
||
**A privacy-first, Windows-native meeting assistant that runs entirely on-device.**
|
||
|
||
WhispAssist captures system audio, transcribes it locally with Whisper-class models using
|
||
on-device acceleration (NPU → GPU → CPU), structures the result into Markdown notes, and
|
||
optionally augments them with a locally hosted LLM (Ollama). Audio and transcripts **never
|
||
leave the machine** unless the user explicitly exports them.
|
||
|
||
> Status: **planning + scaffold**. This repository currently contains the full engineering
|
||
> plan (`docs/`) and a compiling-intent skeleton (`src-tauri/`, `src/`). No feature code is
|
||
> implemented yet. See [`docs/05-roadmap.md`](docs/05-roadmap.md) for the build order.
|
||
|
||
## Why WhispAssist
|
||
|
||
| | Granola | Meetily | **WhispAssist** |
|
||
|---|---|---|---|
|
||
| Local transcription | ❌ cloud | ✅ | ✅ |
|
||
| Notes/summaries stay local | ❌ cloud AI | ⚠️ optional | ✅ local-only by design |
|
||
| NPU/GPU auto-acceleration | n/a | partial | ✅ NPU→GPU→CPU ladder |
|
||
| Calendar + Outlook `.pst` context | ✅ (cloud) | ❌ | ✅ local |
|
||
| Bot-free system-audio capture | ✅ | ✅ | ✅ |
|
||
| License | proprietary | MIT | open source |
|
||
|
||
## Technology
|
||
|
||
WhispAssist is built as a **Tauri 2** application: a small Rust core with a compiled
|
||
**Svelte** web frontend rendered through the OS WebView2 (no bundled browser → low idle
|
||
memory). The choice and its alternatives are recorded in [`docs/adr/`](docs/adr/).
|
||
|
||
- **Shell / IPC:** Tauri 2 (Rust ⇄ WebView2)
|
||
- **Audio capture:** WASAPI loopback (`wasapi` crate)
|
||
- **Transcription:** `whisper-rs` (whisper.cpp: CPU/Vulkan/CUDA) + ONNX Runtime (`ort`) DirectML for the NPU path
|
||
- **Diarization:** `sherpa-onnx` (pyannote segmentation + speaker-embedding clustering), fully offline
|
||
- **Storage:** SQLite (`sqlx`/`rusqlite`) + on-disk audio/transcript files
|
||
- **Local LLM:** Ollama HTTP API on `localhost:11434`
|
||
- **Calendar / Outlook:** `outlook-pst` crate for `.pst`, OS notifications for reminders
|
||
- **Optional recording:** opt-in (default off), saved as `.wav`, with a consent reminder (ADR-0009)
|
||
- **Optional sync:** upload artifacts to your own server — WebDAV covers **Nextcloud, ownCloud, Cloudreve, Seafile, Synology**; **OneDrive/Dropbox/Box** via OAuth. Off by default (ADR-0010)
|
||
- **Optional AI/agent integration:** hosted summary providers (Anthropic, OpenAI-compatible) behind the same provider model, **and** a local **MCP server** so your own coding agents (**Claude, Codex, Copilot, OpenCode, …**) can pull meeting context and "feature briefs" to start coding. Off by default; the MCP server is inbound/loopback only (ADR-0011)
|
||
|
||
## Repository layout
|
||
|
||
```
|
||
WhispAssist/
|
||
├── docs/ # The engineering plan (read this first)
|
||
│ ├── 00-overview.md Vision, goals, glossary
|
||
│ ├── 01-requirements.md Functional + non-functional requirements (traceable IDs)
|
||
│ ├── 02-architecture.md Components, data flow, threading model
|
||
│ ├── 03-data-model.md SQLite schema, file layout, transcript JSON
|
||
│ ├── 04-api-contracts.md Tauri commands/events + internal Rust service traits
|
||
│ ├── 05-roadmap.md 8 phases, task breakdown, acceptance criteria
|
||
│ ├── 06-test-strategy.md Test plan per phase + quality gates
|
||
│ ├── 07-research-findings.md Validated stack with sources
|
||
│ └── adr/ Architecture Decision Records (0001–0010)
|
||
├── src-tauri/ # Rust core (service module skeletons)
|
||
├── src/ # Svelte frontend skeleton
|
||
├── scripts/ # Dev/model-download helper scripts
|
||
└── tests/ # Test fixtures + integration test scaffolding
|
||
```
|
||
|
||
## Getting started (for builders)
|
||
|
||
Prerequisites once implementation begins: Rust (stable), Node.js 20+, the Tauri CLI, and
|
||
WebView2 runtime (preinstalled on Windows 11). Then:
|
||
|
||
```bash
|
||
npm install
|
||
npm run tauri dev # once src-tauri/Cargo.toml dependencies are filled in
|
||
```
|
||
|
||
The current skeleton intentionally does **not** compile end-to-end — modules contain typed
|
||
stubs and `todo!()` markers that map 1:1 to roadmap tasks. Start at Phase 1 in
|
||
[`docs/05-roadmap.md`](docs/05-roadmap.md).
|
||
|
||
## Privacy guarantee
|
||
|
||
WhispAssist originates **no outbound connection for audio or transcript content** except to
|
||
destinations **you explicitly configure** — an LLM endpoint (local Ollama by default, or a hosted AI
|
||
provider if you choose one) and any sync targets you enable — plus explicit model downloads.
|
||
Everything optional is **off by default**; with nothing configured, WA makes no content egress at
|
||
all. The local **MCP server** (for handing meetings to your coding agents) is **inbound on loopback
|
||
and adds no egress** — data only leaves via the agent's own provider, which WA discloses. The set of
|
||
reachable hosts is an allowlist derived from your settings and enforced in the core (and verified by
|
||
a CI network test). Recording is opt-in; sync/AI credentials live in the OS credential store, never
|
||
in config files. See the privacy requirements (`FR-SEC-*`, `NFR-SEC-*`, `FR-SYNC-*`, `FR-MCP-*`) in
|
||
[`docs/01-requirements.md`](docs/01-requirements.md) and ADRs 0009–0011.
|
||
|
||
## License
|
||
|
||
Dual-licensed under MIT or Apache-2.0, at your option. See [`LICENSE`](LICENSE).
|