diff --git a/README.md b/README.md index 70be6be..4d70195 100644 --- a/README.md +++ b/README.md @@ -7,13 +7,14 @@ on-device acceleration (**NPU → GPU → CPU**), labels speakers, structures th Markdown notes, and optionally augments them with a locally hosted LLM (Ollama). Audio and transcripts **never leave the machine** unless you explicitly configure a destination. -> **Status: working application (v0.2.0).** Capture (system audio **+ your microphone**), -> transcription (CPU / Intel NPU / Vulkan GPU), speaker diarization, storage + crash recovery, -> local-LLM summaries, AI tags, opt-in recording with **in-app playback**, at-rest encryption, -> and self-hosted sync are implemented and ship as a single signed **MSI + NSIS** universal -> installer. Outlook `.pst`/calendar context (recurring-event import + filtering) is landing; -> the coding-agent (MCP) handoff is in progress. Build order and remaining tasks are in -> [`docs/05-roadmap.md`](docs/05-roadmap.md). +> **Status: working application (v0.4.0).** Capture (system audio **+ your microphone**, with a +> live **dual level meter**), transcription (CPU / Intel NPU / Vulkan GPU) with a **fluid live +> transcript**, speaker diarization, storage + crash recovery, local-LLM summaries, AI tags, +> opt-in recording with **in-app playback**, at-rest encryption, self-hosted sync, **Outlook +> `.pst`/calendar import with optional auto-record**, **importing an existing recording from a +> file or URL**, and a loopback **MCP server** for coding-agent handoff are all implemented and +> ship as a single signed **MSI + NSIS** universal installer. Build order and remaining tasks +> are in [`docs/05-roadmap.md`](docs/05-roadmap.md). ## Why WhispAssist — Granola vs Meetily vs WhispAssist @@ -27,17 +28,21 @@ transcripts **never leave the machine** unless you explicitly configure a destin | **Speaker diarization** | ✅ cloud | ⚠️ limited | ✅ offline (sherpa-onnx) | | **Summaries / notes AI** | ☁️ cloud LLM | ✅ local (Ollama) / BYO | ✅ local (Ollama, localhost **or LAN**) + optional hosted | | **Default data egress** | ☁️ audio + notes to cloud | 🔒 local (cloud optional) | 🔒 **none** — everything off by default, allowlist-enforced | -| **Calendar / Outlook `.pst` context** | ✅ cloud calendar | ❌ | ⚙️ local `.pst` — in progress | +| **Calendar / Outlook `.pst` context** | ✅ cloud calendar | ❌ | ✅ local `.pst` import + **auto-record on events** | +| **Import an existing recording (file/URL)** | ❌ live capture only | ❌ | ✅ any file or URL (ffmpeg + yt-dlp) | | **Opt-in recording + consent notice** | ⚠️ partial | ❌ | ✅ off by default, one-time consent | | **At-rest encryption** | ☁️ server-side | ❌ | ✅ vault: Argon2id + XChaCha20-Poly1305 | | **Self-hosted sync w/ client-side encryption** | ❌ | ⚠️ | ✅ WebDAV + OAuth, **encrypt-before-upload** | -| **Coding-agent (MCP) handoff** | ❌ | ❌ | ⚙️ local MCP server — in progress | +| **Coding-agent (MCP) server** | ✅ **cloud** MCP (notes via their servers) | ❌ | ✅ **local, loopback-only**, token-gated, adds no egress | | **License** | Proprietary | Open source (MIT) | Open source (MIT / Apache-2.0) | | **Cost** | Subscription | Free | Free | -Comparison reflects each project's public positioning as of mid-2026. Granola and Meetily -are independent products and their capabilities evolve — verify current details before relying -on any row. +Comparison reflects each project's public positioning as of mid-2026. Granola now ships an +MCP server too, but it is **cloud-hosted** — an agent reaching it pulls your notes through +Granola's servers; WhispAssist's MCP server is **loopback-only and adds no egress of its own** +(data leaves only via the connected agent's own provider, which WA discloses). Granola and +Meetily are independent products and their capabilities evolve — verify current details before +relying on any row. **The short version:** Granola is the polished cloud option (your audio and notes are processed on their servers). Meetily is the closest peer — open-source and self-hosted — but is @@ -46,19 +51,29 @@ accelerated, zero-egress-by-default** option: it exploits the NPU/GPU in modern everything on the device unless you opt in, and adds Windows-specific context (Outlook) and a coding-agent handoff. -## What's built (v0.2.0) +## What's built (v0.4.0) -- **Bot-free capture — now both sides** — WASAPI loopback records the system mix (all - participants), and an optional **microphone** path captures your own voice, mixed into both the - live transcript and the saved recording. Pick a specific output/input or turn the mic off in - **Settings ▸ Hardware**. No meeting bot, no per-app plumbing. -- **Local transcription with a hardware ladder** — whisper.cpp via `whisper-rs` on CPU; the - **Intel NPU** via ONNX Runtime + OpenVINO; **GPU via Vulkan** (a single binary that runs on - NVIDIA, AMD, and Intel). WA detects the hardware, picks the best backend - (**NPU → NVIDIA → AMD → Intel → CPU**), streams partial transcripts live, and shows the active - backend in the UI. -- **Speaker diarization** — `sherpa-onnx` (pyannote segmentation + speaker-embedding - clustering), fully offline. +- **Bot-free capture — now both sides, with a live dual meter** — WASAPI loopback records the + system mix (all participants), and an optional **microphone** path captures your own voice, + mixed into both the live transcript and the saved recording. While recording, a **level meter + overlays the system and microphone signals in two colours** so you can see both sides are being + picked up. Pick a specific output/input or turn the mic off in **Settings ▸ Hardware**. No + meeting bot, no per-app plumbing. +- **Local transcription with a hardware ladder — and a fluid live transcript** — whisper.cpp via + `whisper-rs` on CPU; the **Intel NPU** via ONNX Runtime + OpenVINO; **GPU via Vulkan** (a single + binary that runs on NVIDIA, AMD, and Intel). WA detects the hardware, picks the best backend + (**NPU → NVIDIA → AMD → Intel → CPU**), and shows it in the UI. The **live transcript streams a + growing line that refreshes ~once a second and commits at natural pauses** — words appear as + they're spoken instead of in fixed multi-second blocks, so sentences aren't chopped across + lines. +- **Speaker diarization** — `sherpa-onnx` (pyannote segmentation + speaker-embedding clustering), + fully offline. Install the two diarization models in **Settings ▸ Hardware** and finished + recordings are split by speaker (Speaker 1, Speaker 2, …); the microphone speaker is + auto-labelled from a short voiceprint. +- **Import an existing recording** — add a meeting from a **local audio/video file or a URL** + (YouTube, a streaming page, or a direct media link). WhispAssist transcribes and diarizes it + just like a live recording. Uses **`ffmpeg`** (and **`yt-dlp`** for URLs), which you install + yourself — neither is bundled. - **Notes & summaries** — Markdown notes; local-LLM summaries via **Ollama** on `localhost` **or a private LAN endpoint** (RFC-1918), with a full advanced-parameter panel (system prompt, `think`, `keep_alive`, `num_ctx`, sampling/repetition/mirostat, etc.). @@ -80,12 +95,20 @@ coding-agent handoff. destination holds only ciphertext. Credentials live only in the OS credential store. - **Optional hosted AI** — Anthropic and OpenAI-compatible providers behind the same `LlmProvider` interface, off by default (third-party egress, keys in the OS credential store). +- **Outlook `.pst` / calendar context, with auto-record** — import events and attendees from a + local Outlook `.pst` backup (read-only, range-limited, de-duplicated, with cleanup), attach + meetings to events, and — opt-in — **auto-start recording when a calendar event begins** while + the app is open (a one-shot timer, no background polling). +- **Local MCP server for coding-agent handoff** — hand meeting context to your own coding agents + (Claude, Codex, Copilot, OpenCode) over a **loopback-only, token-gated** MCP server that is off + by default, scope-limited, audited, and **adds no egress** — data leaves only via the agent's + own provider, which WA discloses. - **One universal installer** — a single signed MSI and NSIS `-setup.exe` that covers every - machine: Vulkan for all GPUs, the Intel NPU path, and CPU fallback. The Vulkan loader is bundled - so it launches even on machines without a GPU driver. + machine: Vulkan for all GPUs, the Intel NPU path (+ DirectML fallback), and CPU. The Vulkan + loader is bundled so it launches even on machines without a GPU driver. -**In progress:** Outlook `.pst` + calendar context, the local **MCP server** that hands meeting -context to your coding agents (Claude, Codex, Copilot, OpenCode), and MS Graph calendar. +**In progress:** Microsoft Graph calendar (cloud calendar via OAuth) and an optional CUDA +(NVIDIA-only) build variant. ## Quick start (install) @@ -121,7 +144,7 @@ and its alternatives are recorded in [`docs/adr/`](docs/adr/) (ADR-0001–0011). - **Storage:** SQLite + on-disk audio/transcript files - **Local LLM:** Ollama HTTP API (localhost or a private LAN endpoint) - **Encryption:** Argon2id + XChaCha20-Poly1305 envelope vault; secrets in the OS credential store -- **Calendar / Outlook:** `outlook-pst` for `.pst`, OS notifications for reminders *(in progress)* +- **Calendar / Outlook:** `readpst` for `.pst` import, OS scheduled toasts for action-item reminders - **Sync:** WebDAV primary set + OneDrive/Dropbox/Box via OAuth — off by default (ADR-0010) - **External AI / agents:** hosted providers behind `LlmProvider`; a loopback-only **MCP server** for coding-agent handoff — off by default (ADR-0011)