docs(readme): reflect working v0.1.4 (CPU/NPU/Vulkan, vault, sync) + refresh Granola/Meetily comparison
This commit is contained in:
iamdoubz committed
1 parent
fa96a16be4
commit
c24feeb542
1 file changed
+117
-53
@@ -3,88 +3,152 @@
|
||||
**A privacy-first, Windows-native meeting assistant that runs entirely on-device.**
|
||||
|
||||
WhispAssist captures system audio, transcribes it locally with Whisper-class models using
|
||||
on-device acceleration (NPU → GPU → CPU), structures the result into Markdown notes, and
|
||||
optionally augments them with a locally hosted LLM (Ollama). Audio and transcripts **never
|
||||
leave the machine** unless the user explicitly exports them.
|
||||
on-device acceleration (**NPU → GPU → CPU**), labels speakers, structures the result into
|
||||
Markdown notes, and optionally augments them with a locally hosted LLM (Ollama). Audio and
|
||||
transcripts **never leave the machine** unless you explicitly configure a destination.
|
||||
|
||||
> Status: **planning + scaffold**. This repository currently contains the full engineering
|
||||
> plan (`docs/`) and a compiling-intent skeleton (`src-tauri/`, `src/`). No feature code is
|
||||
> implemented yet. See [`docs/05-roadmap.md`](docs/05-roadmap.md) for the build order.
|
||||
> **Status: working application (v0.1.4).** Capture, transcription (CPU / Intel NPU / Vulkan
|
||||
> GPU), speaker diarization, storage + crash recovery, local-LLM summaries, opt-in recording,
|
||||
> at-rest encryption, and self-hosted sync are implemented and ship as signed **MSI + NSIS**
|
||||
> installers. Outlook `.pst`/calendar context and the coding-agent (MCP) handoff are in
|
||||
> progress. Build order and remaining tasks are in [`docs/05-roadmap.md`](docs/05-roadmap.md).
|
||||
|
||||
## Why WhispAssist
|
||||
## Why WhispAssist — Granola vs Meetily vs WhispAssist
|
||||
|
||||
| | Granola | Meetily | **WhispAssist** |
|
||||
| | **Granola** | **Meetily** | **WhispAssist** |
|
||||
|---|---|---|---|
|
||||
| Local transcription | ❌ cloud | ✅ | ✅ |
|
||||
| Notes/summaries stay local | ❌ cloud AI | ⚠️ optional | ✅ local-only by design |
|
||||
| NPU/GPU auto-acceleration | n/a | partial | ✅ NPU→GPU→CPU ladder |
|
||||
| Calendar + Outlook `.pst` context | ✅ (cloud) | ❌ | ✅ local |
|
||||
| Bot-free system-audio capture | ✅ | ✅ | ✅ |
|
||||
| License | proprietary | MIT | open source |
|
||||
| **Positioning** | Cloud AI notepad | Open-source self-hosted assistant | Local, Windows-native assistant |
|
||||
| **Platform** | macOS, Windows | macOS, Windows, Linux | Windows 10/11 |
|
||||
| **Bot-free system-audio capture** | ✅ | ✅ | ✅ WASAPI loopback |
|
||||
| **Transcription** | ☁️ cloud | ✅ local (Whisper) | ✅ local (whisper.cpp) |
|
||||
| **NPU / GPU auto-acceleration** | n/a (cloud) | ⚠️ CPU/GPU, manual | ✅ **NPU→NVIDIA→AMD→Intel→CPU** ladder, one binary |
|
||||
| **Speaker diarization** | ✅ cloud | ⚠️ limited | ✅ offline (sherpa-onnx) |
|
||||
| **Summaries / notes AI** | ☁️ cloud LLM | ✅ local (Ollama) / BYO | ✅ local (Ollama, localhost **or LAN**) + optional hosted |
|
||||
| **Default data egress** | ☁️ audio + notes to cloud | 🔒 local (cloud optional) | 🔒 **none** — everything off by default, allowlist-enforced |
|
||||
| **Calendar / Outlook `.pst` context** | ✅ cloud calendar | ❌ | ⚙️ local `.pst` — in progress |
|
||||
| **Opt-in recording + consent notice** | ⚠️ partial | ❌ | ✅ off by default, one-time consent |
|
||||
| **At-rest encryption** | ☁️ server-side | ❌ | ✅ vault: Argon2id + XChaCha20-Poly1305 |
|
||||
| **Self-hosted sync w/ client-side encryption** | ❌ | ⚠️ | ✅ WebDAV + OAuth, **encrypt-before-upload** |
|
||||
| **Coding-agent (MCP) handoff** | ❌ | ❌ | ⚙️ local MCP server — in progress |
|
||||
| **License** | Proprietary | Open source (MIT) | Open source (MIT / Apache-2.0) |
|
||||
| **Cost** | Subscription | Free | Free |
|
||||
|
||||
<sub>Comparison reflects each project's public positioning as of mid-2026. Granola and Meetily
|
||||
are independent products and their capabilities evolve — verify current details before relying
|
||||
on any row.</sub>
|
||||
|
||||
**The short version:** Granola is the polished cloud option (your audio and notes are processed
|
||||
on their servers). Meetily is the closest peer — open-source and self-hosted — but is
|
||||
cross-platform-generic and leans on manual setup. WhispAssist is the **Windows-native, hardware-
|
||||
accelerated, zero-egress-by-default** option: it exploits the NPU/GPU in modern laptops, keeps
|
||||
everything on the device unless you opt in, and adds Windows-specific context (Outlook) and a
|
||||
coding-agent handoff.
|
||||
|
||||
## What's built (v0.1.4)
|
||||
|
||||
- **Bot-free capture** — WASAPI loopback records the system mix (all participants) with no
|
||||
meeting bot and no per-app plumbing.
|
||||
- **Local transcription with a hardware ladder** — whisper.cpp via `whisper-rs` on CPU; the
|
||||
**Intel NPU** via ONNX Runtime + OpenVINO; **GPU via Vulkan** (a single binary that runs on
|
||||
NVIDIA, AMD, and Intel). WA detects the hardware, picks the best backend
|
||||
(**NPU → NVIDIA → AMD → Intel → CPU**), streams partial transcripts live, and shows the active
|
||||
backend in the UI.
|
||||
- **Speaker diarization** — `sherpa-onnx` (pyannote segmentation + speaker-embedding
|
||||
clustering), fully offline.
|
||||
- **Notes & summaries** — Markdown notes; local-LLM summaries via **Ollama** on `localhost`
|
||||
**or a private LAN endpoint** (RFC-1918), with a full advanced-parameter panel (system prompt,
|
||||
`think`, `keep_alive`, `num_ctx`, sampling/repetition/mirostat, etc.).
|
||||
- **Storage & crash recovery** — SQLite + on-disk audio/transcripts under
|
||||
`%LOCALAPPDATA%\WhispAssist`. Audio is the source of truth; notes and transcripts regenerate
|
||||
after a crash.
|
||||
- **Opt-in recording** — off by default; `.wav` retained only when you turn it on, after a
|
||||
one-time consent notice.
|
||||
- **At-rest encryption vault** — Argon2id key derivation + XChaCha20-Poly1305; transcripts,
|
||||
notes, summaries, and recordings sealed on disk; startup unlock gate; keys zeroized on lock.
|
||||
- **Self-hosted sync (optional, off by default)** — WebDAV (Nextcloud, ownCloud, Cloudreve,
|
||||
Seafile, Synology) plus OneDrive/Dropbox/Box (OAuth 2.0 PKCE); durable retry queue with
|
||||
backoff; **client-side encryption before upload** so the destination holds only ciphertext.
|
||||
Credentials live only in the OS credential store.
|
||||
- **Optional hosted AI** — Anthropic and OpenAI-compatible providers behind the same
|
||||
`LlmProvider` interface, off by default (third-party egress, keys in the OS credential store).
|
||||
- **Installers** — signed MSI and NSIS `-setup.exe`.
|
||||
|
||||
**In progress:** Outlook `.pst` + calendar context, the local **MCP server** that hands meeting
|
||||
context to your coding agents (Claude, Codex, Copilot, OpenCode), and MS Graph calendar.
|
||||
|
||||
## Technology
|
||||
|
||||
WhispAssist is built as a **Tauri 2** application: a small Rust core with a compiled
|
||||
**Svelte** web frontend rendered through the OS WebView2 (no bundled browser → low idle
|
||||
memory). The choice and its alternatives are recorded in [`docs/adr/`](docs/adr/).
|
||||
WhispAssist is a **Tauri 2** application: a small Rust core with a compiled **Svelte + TypeScript**
|
||||
frontend rendered through the OS WebView2 (no bundled browser → low idle memory). Every decision
|
||||
and its alternatives are recorded in [`docs/adr/`](docs/adr/) (ADR-0001–0011).
|
||||
|
||||
- **Shell / IPC:** Tauri 2 (Rust ⇄ WebView2)
|
||||
- **Audio capture:** WASAPI loopback (`wasapi` crate)
|
||||
- **Transcription:** `whisper-rs` (whisper.cpp: CPU/Vulkan/CUDA) + ONNX Runtime (`ort`) DirectML for the NPU path
|
||||
- **Diarization:** `sherpa-onnx` (pyannote segmentation + speaker-embedding clustering), fully offline
|
||||
- **Storage:** SQLite (`sqlx`/`rusqlite`) + on-disk audio/transcript files
|
||||
- **Local LLM:** Ollama HTTP API on `localhost:11434`
|
||||
- **Calendar / Outlook:** `outlook-pst` crate for `.pst`, OS notifications for reminders
|
||||
- **Optional recording:** opt-in (default off), saved as `.wav`, with a consent reminder (ADR-0009)
|
||||
- **Optional sync:** upload artifacts to your own server — WebDAV covers **Nextcloud, ownCloud, Cloudreve, Seafile, Synology**; **OneDrive/Dropbox/Box** via OAuth. Off by default (ADR-0010)
|
||||
- **Optional AI/agent integration:** hosted summary providers (Anthropic, OpenAI-compatible) behind the same provider model, **and** a local **MCP server** so your own coding agents (**Claude, Codex, Copilot, OpenCode, …**) can pull meeting context and "feature briefs" to start coding. Off by default; the MCP server is inbound/loopback only (ADR-0011)
|
||||
- **Audio capture:** WASAPI loopback
|
||||
- **Transcription:** `whisper-rs` (whisper.cpp: CPU / **Vulkan** / CUDA) + ONNX Runtime (`ort`)
|
||||
with the **OpenVINO** execution provider for the Intel NPU path
|
||||
- **Diarization:** `sherpa-onnx`, fully offline
|
||||
- **Storage:** SQLite + on-disk audio/transcript files
|
||||
- **Local LLM:** Ollama HTTP API (localhost or a private LAN endpoint)
|
||||
- **Encryption:** Argon2id + XChaCha20-Poly1305 envelope vault; secrets in the OS credential store
|
||||
- **Calendar / Outlook:** `outlook-pst` for `.pst`, OS notifications for reminders *(in progress)*
|
||||
- **Sync:** WebDAV primary set + OneDrive/Dropbox/Box via OAuth — off by default (ADR-0010)
|
||||
- **External AI / agents:** hosted providers behind `LlmProvider`; a loopback-only **MCP server**
|
||||
for coding-agent handoff — off by default (ADR-0011)
|
||||
|
||||
## Repository layout
|
||||
|
||||
```
|
||||
WhispAssist/
|
||||
├── docs/ # The engineering plan (read this first)
|
||||
│ ├── 00-overview.md Vision, goals, glossary
|
||||
│ ├── 01-requirements.md Functional + non-functional requirements (traceable IDs)
|
||||
│ ├── 02-architecture.md Components, data flow, threading model
|
||||
│ ├── 03-data-model.md SQLite schema, file layout, transcript JSON
|
||||
│ ├── 04-api-contracts.md Tauri commands/events + internal Rust service traits
|
||||
│ ├── 05-roadmap.md 8 phases, task breakdown, acceptance criteria
|
||||
│ ├── 06-test-strategy.md Test plan per phase + quality gates
|
||||
│ ├── 07-research-findings.md Validated stack with sources
|
||||
│ └── adr/ Architecture Decision Records (0001–0010)
|
||||
├── src-tauri/ # Rust core (service module skeletons)
|
||||
├── src/ # Svelte frontend skeleton
|
||||
├── scripts/ # Dev/model-download helper scripts
|
||||
└── tests/ # Test fixtures + integration test scaffolding
|
||||
│ ├── 00-overview.md … 07-research-findings.md
|
||||
│ └── adr/ Architecture Decision Records (0001–0011)
|
||||
├── src-tauri/ # Rust core — implemented service modules:
|
||||
│ └── src/{audio,transcription,diarization,storage,llm,calendar,
|
||||
│ hardware,notes,sync,mcp,vault}
|
||||
├── src/ # Svelte + TypeScript frontend (views, stores, API bindings)
|
||||
├── packaging/ # NPU/OpenVINO runtime bundle + release assets
|
||||
├── scripts/ # Dev / model-download helpers
|
||||
└── tests/ # Fixtures + cross-service integration tests
|
||||
```
|
||||
|
||||
## Getting started (for builders)
|
||||
|
||||
Prerequisites once implementation begins: Rust (stable), Node.js 20+, the Tauri CLI, and
|
||||
WebView2 runtime (preinstalled on Windows 11). Then:
|
||||
Prerequisites: **Rust** (stable), **Node.js 20+**, the **Tauri CLI**, and the WebView2 runtime
|
||||
(preinstalled on Windows 11). Native builds also need the **VS 2022 Build Tools** (load
|
||||
`vcvars64.bat` first).
|
||||
|
||||
```bash
|
||||
npm install
|
||||
npm run tauri dev # once src-tauri/Cargo.toml dependencies are filled in
|
||||
npm run tauri dev # CPU/NPU build
|
||||
```
|
||||
|
||||
The current skeleton intentionally does **not** compile end-to-end — modules contain typed
|
||||
stubs and `todo!()` markers that map 1:1 to roadmap tasks. Start at Phase 1 in
|
||||
[`docs/05-roadmap.md`](docs/05-roadmap.md).
|
||||
**GPU (Vulkan) build.** whisper.cpp's GPU backends are compiled in (not downloaded at runtime),
|
||||
so a GPU build needs a one-time toolchain setup — the **Vulkan SDK**, a **Ninja** generator, and
|
||||
a short target dir (to dodge Windows' 260-char path limit in the shader build):
|
||||
|
||||
```bash
|
||||
# after: Vulkan SDK installed, ninja.exe on PATH, vcvars64 loaded
|
||||
set VULKAN_SDK=C:\VulkanSDK\1.4.350.0
|
||||
set CMAKE_GENERATOR=Ninja
|
||||
set CARGO_TARGET_DIR=C:\wt
|
||||
npm run tauri build -- --features vulkan
|
||||
```
|
||||
|
||||
CUDA (NVIDIA-only, faster) is planned as an optional variant. The full, gotcha-annotated build
|
||||
recipe lives in the project notes.
|
||||
|
||||
## Privacy guarantee
|
||||
|
||||
WhispAssist originates **no outbound connection for audio or transcript content** except to
|
||||
destinations **you explicitly configure** — an LLM endpoint (local Ollama by default, or a hosted AI
|
||||
provider if you choose one) and any sync targets you enable — plus explicit model downloads.
|
||||
destinations **you explicitly configure** — an LLM endpoint (local Ollama by default, or a hosted
|
||||
AI provider if you choose one) and any sync targets you enable — plus explicit model downloads.
|
||||
Everything optional is **off by default**; with nothing configured, WA makes no content egress at
|
||||
all. The local **MCP server** (for handing meetings to your coding agents) is **inbound on loopback
|
||||
and adds no egress** — data only leaves via the agent's own provider, which WA discloses. The set of
|
||||
reachable hosts is an allowlist derived from your settings and enforced in the core (and verified by
|
||||
a CI network test). Recording is opt-in; sync/AI credentials live in the OS credential store, never
|
||||
in config files. See the privacy requirements (`FR-SEC-*`, `NFR-SEC-*`, `FR-SYNC-*`, `FR-MCP-*`) in
|
||||
[`docs/01-requirements.md`](docs/01-requirements.md) and ADRs 0009–0011.
|
||||
all. The local **MCP server** (for handing meetings to your coding agents) is **inbound on
|
||||
loopback and adds no egress** — data only leaves via the agent's own provider, which WA discloses.
|
||||
The set of reachable hosts is an allowlist derived from your settings and enforced in the core
|
||||
(and verified by a CI network test). Recording is opt-in; sync/AI credentials live in the OS
|
||||
credential store, never in config files. See the privacy requirements (`FR-SEC-*`, `NFR-SEC-*`,
|
||||
`FR-SYNC-*`, `FR-MCP-*`) in [`docs/01-requirements.md`](docs/01-requirements.md) and ADRs 0009–0011.
|
||||
|
||||
## License
|
||||
|
||||
|
||||
Reference in new issue
Block a user