WhispAssist (WA)

A privacy-first, Windows-native meeting assistant that runs entirely on-device.

WhispAssist captures system audio, transcribes it locally with Whisper-class models using on-device acceleration (NPU → GPU → CPU), structures the result into Markdown notes, and optionally augments them with a locally hosted LLM (Ollama). Audio and transcripts never leave the machine unless the user explicitly exports them.

Status: planning + scaffold. This repository currently contains the full engineering plan (docs/) and a compiling-intent skeleton (src-tauri/, src/). No feature code is implemented yet. See docs/05-roadmap.md for the build order.

Why WhispAssist

	Granola	Meetily	WhispAssist
Local transcription	❌ cloud	✅	✅
Notes/summaries stay local	❌ cloud AI	⚠️ optional	✅ local-only by design
NPU/GPU auto-acceleration	n/a	partial	✅ NPU→GPU→CPU ladder
Calendar + Outlook .pst context	✅ (cloud)	❌	✅ local
Bot-free system-audio capture	✅	✅	✅
License	proprietary	MIT	open source

Technology

WhispAssist is built as a Tauri 2 application: a small Rust core with a compiled Svelte web frontend rendered through the OS WebView2 (no bundled browser → low idle memory). The choice and its alternatives are recorded in docs/adr/.

• Shell / IPC: Tauri 2 (Rust ⇄ WebView2)
• Audio capture: WASAPI loopback (wasapi crate)
• Transcription: whisper-rs (whisper.cpp: CPU/Vulkan/CUDA) + ONNX Runtime (ort) DirectML for the NPU path
• Diarization: sherpa-onnx (pyannote segmentation + speaker-embedding clustering), fully offline
• Storage: SQLite (sqlx/rusqlite) + on-disk audio/transcript files
• Local LLM: Ollama HTTP API on localhost:11434
• Calendar / Outlook: outlook-pst crate for .pst, OS notifications for reminders
• Optional recording: opt-in (default off), saved as .wav, with a consent reminder (ADR-0009)
• Optional sync: upload artifacts to your own server — WebDAV covers Nextcloud, ownCloud, Cloudreve, Seafile, Synology; OneDrive/Dropbox/Box via OAuth. Off by default (ADR-0010)
• Optional AI/agent integration: hosted summary providers (Anthropic, OpenAI-compatible) behind the same provider model, and a local MCP server so your own coding agents (Claude, Codex, Copilot, OpenCode, …) can pull meeting context and "feature briefs" to start coding. Off by default; the MCP server is inbound/loopback only (ADR-0011)

Repository layout

WhispAssist/
├── docs/                 # The engineering plan (read this first)
│   ├── 00-overview.md        Vision, goals, glossary
│   ├── 01-requirements.md    Functional + non-functional requirements (traceable IDs)
│   ├── 02-architecture.md    Components, data flow, threading model
│   ├── 03-data-model.md      SQLite schema, file layout, transcript JSON
│   ├── 04-api-contracts.md   Tauri commands/events + internal Rust service traits
│   ├── 05-roadmap.md         8 phases, task breakdown, acceptance criteria
│   ├── 06-test-strategy.md   Test plan per phase + quality gates
│   ├── 07-research-findings.md  Validated stack with sources
│   └── adr/                  Architecture Decision Records (0001–0010)
├── src-tauri/            # Rust core (service module skeletons)
├── src/                  # Svelte frontend skeleton
├── scripts/              # Dev/model-download helper scripts
└── tests/                # Test fixtures + integration test scaffolding

Getting started (for builders)

Prerequisites once implementation begins: Rust (stable), Node.js 20+, the Tauri CLI, and WebView2 runtime (preinstalled on Windows 11). Then:

npm install
npm run tauri dev      # once src-tauri/Cargo.toml dependencies are filled in

The current skeleton intentionally does not compile end-to-end — modules contain typed stubs and todo!() markers that map 1:1 to roadmap tasks. Start at Phase 1 in docs/05-roadmap.md.

Privacy guarantee

WhispAssist originates no outbound connection for audio or transcript content except to destinations you explicitly configure — an LLM endpoint (local Ollama by default, or a hosted AI provider if you choose one) and any sync targets you enable — plus explicit model downloads. Everything optional is off by default; with nothing configured, WA makes no content egress at all. The local MCP server (for handing meetings to your coding agents) is inbound on loopback and adds no egress — data only leaves via the agent's own provider, which WA discloses. The set of reachable hosts is an allowlist derived from your settings and enforced in the core (and verified by a CI network test). Recording is opt-in; sync/AI credentials live in the OS credential store, never in config files. See the privacy requirements (FR-SEC-*, NFR-SEC-*, FR-SYNC-*, FR-MCP-*) in docs/01-requirements.md and ADRs 0009–0011.

License

Dual-licensed under MIT or Apache-2.0, at your option. See LICENSE.