iamdoubz
2cc48c9d4d
Recordings are now native-quality (~50-100 MB) and put() used to buffer the whole file into memory with a single PUT/upload call; OneDrive/Graph also caps a single PUT at 250 MB. This replaces that with disk-streamed, chunked uploads on both sync backends (FR-SYNC-2/5): - Both WebDavTarget::put and OneDriveTarget::put now stream the file off disk in fixed-size buffers (tokio::fs::File + BufReader) instead of tokio::fs::read()'ing it whole — memory use is O(chunk), not O(file size), for every upload, large or small. - Files at or below LARGE_FILE_THRESHOLD (8 MiB) still take a single streamed PUT; only large artifacts (in practice, .wav recordings) take the chunked path, so small transcript/notes/summary uploads are unaffected. - WebDAV (Nextcloud/ownCloud): implements the chunking-v2 protocol — MKCOL an upload collection keyed deterministically off the remote path's sha256 (stable across retries), PUT lexically-sortable parts, MOVE the virtual `.file` to assemble server-side. A retry PROPFINDs the collection first and skips any part already landed, so a partially-uploaded large file resumes instead of restarting at byte 0 — genuine mid-file resume, not just chunk-level retry. Generic WebDAV targets (Seafile, Synology, unbranded servers) have no equivalent server-side assembly endpoint, so they fall back to one streamed PUT above the threshold too — memory-safe, just not chunk-resumable (documented ponytail simplification). - OneDrive/Graph: implements the createUploadSession + Content-Range byte-range PUT flow (chunks capped at 10 MiB, a multiple of Graph's required 320 KiB granularity). A failed chunk is retried a few times within the same put() call. ponytail-documented ceiling: the session URL isn't persisted on the sync_jobs row, so a retry from a *later* pump cycle (e.g. after an app restart) starts a fresh session and re-uploads from byte 0 rather than resuming — true cross-attempt resume would need a sync_jobs column, left as a follow-up. - Progress is reported via the existing ProgressSink after each chunk, not just once at the end, so the UI sees steady movement on a large upload (FR-SYNC-11). - OneDriveTarget's Graph URLs are now built from a `graph_base()` helper (overridable via WA_GRAPH_BASE_URL, unset in production) so the chunked session flow can be exercised against a local mock in tests. Tests (all against small, in-process, dependency-free mock servers — std::net only, no new crates, same pattern as oauth::LoopbackRedirect): - nextcloud_chunked_upload_assembles_and_resumes: multi-chunk upload assembles byte-for-byte, then a resumed upload with one chunk pre-seeded skips exactly that chunk. - small_file_skips_chunking_even_on_a_chunking_capable_target: a small file never takes the chunked path. - onedrive_chunked_upload_via_graph_session_mock: multi-chunk Graph upload session assembles byte-for-byte with per-chunk progress. - parse_chunk_sizes_extracts_href_and_length_ignoring_namespace_prefix and chunk_name_is_zero_padded_and_lexically_sortable: pure unit tests for the PROPFIND-response scraper and chunk-naming scheme. Skip-if-unchanged (SHA-256, in storage::upsert_sync_job) and the exponential-backoff retry queue (storage::claim_due_sync_jobs) are untouched — this only changes how SyncTarget::put moves bytes. Verified: cargo fmt clean; cargo clippy --features sync and --features mcp both clean (-D warnings); cargo test --lib (default features: audio, cpu-transcription, diarization, pst, sync, npu) 165 passed, 0 failed, 7 ignored (unrelated hardware-gated tests); cargo test --features sync --lib: 18 passed, 0 failed, 1 ignored (webdav_round_trip, opt-in live-server test per existing convention).
WhispAssist (WA)
A privacy-first, Windows-native meeting assistant that runs entirely on-device.
WhispAssist captures system audio, transcribes it locally with Whisper-class models using on-device acceleration (NPU → GPU → CPU), labels speakers, structures the result into Markdown notes, and optionally augments them with a locally hosted LLM (Ollama). Audio and transcripts never leave the machine unless you explicitly configure a destination.
Status: working application (v0.2.0). Capture (system audio + your microphone), transcription (CPU / Intel NPU / Vulkan GPU), speaker diarization, storage + crash recovery, local-LLM summaries, AI tags, opt-in recording with in-app playback, at-rest encryption, and self-hosted sync are implemented and ship as a single signed MSI + NSIS universal installer. Outlook
.pst/calendar context (recurring-event import + filtering) is landing; the coding-agent (MCP) handoff is in progress. Build order and remaining tasks are indocs/05-roadmap.md.
Why WhispAssist — Granola vs Meetily vs WhispAssist
| Granola | Meetily | WhispAssist | |
|---|---|---|---|
| Positioning | Cloud AI notepad | Open-source self-hosted assistant | Local, Windows-native assistant |
| Platform | macOS, Windows | macOS, Windows, Linux | Windows 10/11 |
| Bot-free system-audio capture | ✅ | ✅ | ✅ WASAPI loopback |
| Transcription | ☁️ cloud | ✅ local (Whisper) | ✅ local (whisper.cpp) |
| NPU / GPU auto-acceleration | n/a (cloud) | ⚠️ CPU/GPU, manual | ✅ NPU→NVIDIA→AMD→Intel→CPU ladder, one binary |
| Speaker diarization | ✅ cloud | ⚠️ limited | ✅ offline (sherpa-onnx) |
| Summaries / notes AI | ☁️ cloud LLM | ✅ local (Ollama) / BYO | ✅ local (Ollama, localhost or LAN) + optional hosted |
| Default data egress | ☁️ audio + notes to cloud | 🔒 local (cloud optional) | 🔒 none — everything off by default, allowlist-enforced |
Calendar / Outlook .pst context |
✅ cloud calendar | ❌ | ⚙️ local .pst — in progress |
| Opt-in recording + consent notice | ⚠️ partial | ❌ | ✅ off by default, one-time consent |
| At-rest encryption | ☁️ server-side | ❌ | ✅ vault: Argon2id + XChaCha20-Poly1305 |
| Self-hosted sync w/ client-side encryption | ❌ | ⚠️ | ✅ WebDAV + OAuth, encrypt-before-upload |
| Coding-agent (MCP) handoff | ❌ | ❌ | ⚙️ local MCP server — in progress |
| License | Proprietary | Open source (MIT) | Open source (MIT / Apache-2.0) |
| Cost | Subscription | Free | Free |
Comparison reflects each project's public positioning as of mid-2026. Granola and Meetily are independent products and their capabilities evolve — verify current details before relying on any row.
The short version: Granola is the polished cloud option (your audio and notes are processed on their servers). Meetily is the closest peer — open-source and self-hosted — but is cross-platform-generic and leans on manual setup. WhispAssist is the Windows-native, hardware- accelerated, zero-egress-by-default option: it exploits the NPU/GPU in modern laptops, keeps everything on the device unless you opt in, and adds Windows-specific context (Outlook) and a coding-agent handoff.
What's built (v0.2.0)
- Bot-free capture — now both sides — WASAPI loopback records the system mix (all participants), and an optional microphone path captures your own voice, mixed into both the live transcript and the saved recording. Pick a specific output/input or turn the mic off in Settings ▸ Hardware. No meeting bot, no per-app plumbing.
- Local transcription with a hardware ladder — whisper.cpp via
whisper-rson CPU; the Intel NPU via ONNX Runtime + OpenVINO; GPU via Vulkan (a single binary that runs on NVIDIA, AMD, and Intel). WA detects the hardware, picks the best backend (NPU → NVIDIA → AMD → Intel → CPU), streams partial transcripts live, and shows the active backend in the UI. - Speaker diarization —
sherpa-onnx(pyannote segmentation + speaker-embedding clustering), fully offline. - Notes & summaries — Markdown notes; local-LLM summaries via Ollama on
localhostor a private LAN endpoint (RFC-1918), with a full advanced-parameter panel (system prompt,think,keep_alive,num_ctx, sampling/repetition/mirostat, etc.). - Storage & crash recovery — SQLite + on-disk audio/transcripts under
%LOCALAPPDATA%\WhispAssist. Audio is the source of truth; notes and transcripts regenerate after a crash. - Opt-in recording + in-app playback — off by default;
.wavretained only when you turn it on, after a one-time consent notice. Play a saved recording back in the app — encrypted recordings are decrypted in memory on the fly (nothing plaintext is written to disk). Recordings are 16-bit for roughly half the size, and an accidental recording can be cancelled (audio + transcript deleted). - Notes, summaries & AI tags — Markdown notes with an Editor/Preview toggle; local-LLM summaries and one-click tag generation with a chip-based tag editor and tag filtering.
- At-rest encryption vault — Argon2id key derivation + XChaCha20-Poly1305; transcripts, notes, summaries, and recordings sealed on disk; startup unlock gate; keys zeroized on lock.
- Self-hosted sync (optional, off by default) — WebDAV (Nextcloud, ownCloud, Cloudreve, Seafile, Synology) plus OneDrive/Dropbox/Box (OAuth 2.0 PKCE); durable retry queue with backoff and live per-item upload progress; client-side encryption before upload so the destination holds only ciphertext. Credentials live only in the OS credential store.
- Optional hosted AI — Anthropic and OpenAI-compatible providers behind the same
LlmProviderinterface, off by default (third-party egress, keys in the OS credential store). - One universal installer — a single signed MSI and NSIS
-setup.exethat covers every machine: Vulkan for all GPUs, the Intel NPU path, and CPU fallback. The Vulkan loader is bundled so it launches even on machines without a GPU driver.
In progress: Outlook .pst + calendar context, the local MCP server that hands meeting
context to your coding agents (Claude, Codex, Copilot, OpenCode), and MS Graph calendar.
Quick start (install)
Requirements: Windows 10 or 11 (x64). WhispAssist needs the WebView2 runtime — preinstalled on Windows 11; on Windows 10 the installer fetches it automatically (so keep an internet connection during setup).
- Download the latest
WhispAssist_<version>_x64_en-US.msifrom the releases page. - Double-click the
.msiand follow the prompts, accepting the UAC prompt to install. The installer is signed; if Windows SmartScreen appears, choose More info → Run anyway. - Launch WhispAssist from the Start menu.
On first run WA detects your hardware and picks the best transcription backend
(NPU → NVIDIA → AMD → Intel → CPU). It works on your CPU or GPU (GPU via Vulkan) out of the
box; to use an Intel NPU, open Settings ▸ Hardware and download the one-time NPU
acceleration package. The app runs without admin rights, never adds itself to startup, and keeps all
data under %LOCALAPPDATA%\WhispAssist.
Prefer the NSIS installer? Grab WhispAssist_<version>_x64-setup.exe from the same page.
Technology
WhispAssist is a Tauri 2 application: a small Rust core with a compiled Svelte + TypeScript
frontend rendered through the OS WebView2 (no bundled browser → low idle memory). Every decision
and its alternatives are recorded in docs/adr/ (ADR-0001–0011).
- Shell / IPC: Tauri 2 (Rust ⇄ WebView2)
- Audio capture: WASAPI loopback
- Transcription:
whisper-rs(whisper.cpp: CPU / Vulkan / CUDA) + ONNX Runtime (ort) with the OpenVINO execution provider for the Intel NPU path - Diarization:
sherpa-onnx, fully offline - Storage: SQLite + on-disk audio/transcript files
- Local LLM: Ollama HTTP API (localhost or a private LAN endpoint)
- Encryption: Argon2id + XChaCha20-Poly1305 envelope vault; secrets in the OS credential store
- Calendar / Outlook:
outlook-pstfor.pst, OS notifications for reminders (in progress) - Sync: WebDAV primary set + OneDrive/Dropbox/Box via OAuth — off by default (ADR-0010)
- External AI / agents: hosted providers behind
LlmProvider; a loopback-only MCP server for coding-agent handoff — off by default (ADR-0011)
Repository layout
WhispAssist/
├── docs/ # The engineering plan (read this first)
│ ├── 00-overview.md … 07-research-findings.md
│ └── adr/ Architecture Decision Records (0001–0011)
├── src-tauri/ # Rust core — implemented service modules:
│ └── src/{audio,transcription,diarization,storage,llm,calendar,
│ hardware,notes,sync,mcp,vault}
├── src/ # Svelte + TypeScript frontend (views, stores, API bindings)
├── packaging/ # NPU/OpenVINO runtime bundle + release assets
├── scripts/ # Dev / model-download helpers
└── tests/ # Fixtures + cross-service integration tests
Getting started (for builders)
Prerequisites: Rust (stable), Node.js 20+, the Tauri CLI, and the WebView2 runtime
(preinstalled on Windows 11). Native builds also need the VS 2022 Build Tools (load
vcvars64.bat first).
npm install
npm run tauri dev # CPU/NPU build
Release build (single universal installer). whisper.cpp's GPU backends are compiled in (not downloaded at runtime), so the release build needs a one-time toolchain setup — the Vulkan SDK, a Ninja generator, and a short target dir (to dodge Windows' 260-char path limit in the shader build):
# after: Vulkan SDK installed, ninja.exe on PATH, vcvars64 loaded
set VULKAN_SDK=C:\VulkanSDK\1.4.350.0
set CMAKE_GENERATOR=Ninja
set CARGO_TARGET_DIR=C:\wt
npm run tauri build -- --features vulkan --config src-tauri/tauri.vulkan.conf.json
This one build covers every machine: Vulkan accelerates all GPUs (NVIDIA/AMD/Intel), the Intel
NPU path works via the runtime OpenVINO download, and CPU is the fallback. The --features vulkan
binary links vulkan-1.dll, so build.rs stages the redistributable Vulkan loader (from
VULKAN_SDK\Bin, or System32) next to the exe and tauri.vulkan.conf.json bundles it into the
installer — the app then launches even on a machine with no GPU driver (it reports zero Vulkan
devices and decodes on the CPU). DirectML is intentionally not offered here because Vulkan already
covers those GPUs; it's the GPU path only in the plain npm run tauri build (no Vulkan) variant,
kept as an internal fallback.
CUDA (NVIDIA-only, faster) is planned as an optional variant. The full, gotcha-annotated build recipe lives in the project notes.
Privacy guarantee
WhispAssist originates no outbound connection for audio or transcript content except to
destinations you explicitly configure — an LLM endpoint (local Ollama by default, or a hosted
AI provider if you choose one) and any sync targets you enable — plus explicit model downloads.
Everything optional is off by default; with nothing configured, WA makes no content egress at
all. The local MCP server (for handing meetings to your coding agents) is inbound on
loopback and adds no egress — data only leaves via the agent's own provider, which WA discloses.
The set of reachable hosts is an allowlist derived from your settings and enforced in the core
(and verified by a CI network test). Recording is opt-in; sync/AI credentials live in the OS
credential store, never in config files. See the privacy requirements (FR-SEC-*, NFR-SEC-*,
FR-SYNC-*, FR-MCP-*) in docs/01-requirements.md and ADRs 0009–0011.
License
Dual-licensed under MIT or Apache-2.0, at your option. See LICENSE.