From b19e01c171be96001324a4ad44e16865c64b9c1d Mon Sep 17 00:00:00 2001 From: iamdoubz <> Date: Tue, 30 Jun 2026 16:11:55 -0500 Subject: [PATCH] Add all scaffolding, design, and decision documentation AKA first commit --- LICENSE | 41 +++++++++++++++++-------- README.md | 92 +++++++++++++++++++++++++++++++++++++++++++++++++++++-- 2 files changed, 118 insertions(+), 15 deletions(-) diff --git a/LICENSE b/LICENSE index 7762406..8014fcf 100644 --- a/LICENSE +++ b/LICENSE @@ -1,18 +1,33 @@ +WhispAssist is dual-licensed under either of: + + * Apache License, Version 2.0 (https://www.apache.org/licenses/LICENSE-2.0) + * MIT license (https://opensource.org/licenses/MIT) + +at your option. + +Unless you explicitly state otherwise, any contribution intentionally submitted for +inclusion in the work by you, as defined in the Apache-2.0 license, shall be dual +licensed as above, without any additional terms or conditions. + +-------------------------------------------------------------------------------- MIT License -Copyright (c) 2026 iamdoubz +Copyright (c) 2026 WhispAssist contributors -Permission is hereby granted, free of charge, to any person obtaining a copy of this software and -associated documentation files (the "Software"), to deal in the Software without restriction, including -without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell -copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the -following conditions: +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: -The above copyright notice and this permission notice shall be included in all copies or substantial -portions of the Software. +The above copyright notice and this permission notice shall be included in all +copies or substantial portions of the Software. -THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT -LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO -EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER -IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE -USE OR OTHER DEALINGS IN THE SOFTWARE. +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +SOFTWARE. diff --git a/README.md b/README.md index b2791c5..b287f3e 100644 --- a/README.md +++ b/README.md @@ -1,3 +1,91 @@ -# WhispAssist +# WhispAssist (WA) -A privacy-first, open-sourced, Windows-native meeting assistant that runs entirely on-device \ No newline at end of file +**A privacy-first, Windows-native meeting assistant that runs entirely on-device.** + +WhispAssist captures system audio, transcribes it locally with Whisper-class models using +on-device acceleration (NPU → GPU → CPU), structures the result into Markdown notes, and +optionally augments them with a locally hosted LLM (Ollama). Audio and transcripts **never +leave the machine** unless the user explicitly exports them. + +> Status: **planning + scaffold**. This repository currently contains the full engineering +> plan (`docs/`) and a compiling-intent skeleton (`src-tauri/`, `src/`). No feature code is +> implemented yet. See [`docs/05-roadmap.md`](docs/05-roadmap.md) for the build order. + +## Why WhispAssist + +| | Granola | Meetily | **WhispAssist** | +|---|---|---|---| +| Local transcription | ❌ cloud | ✅ | ✅ | +| Notes/summaries stay local | ❌ cloud AI | ⚠️ optional | ✅ local-only by design | +| NPU/GPU auto-acceleration | n/a | partial | ✅ NPU→GPU→CPU ladder | +| Calendar + Outlook `.pst` context | ✅ (cloud) | ❌ | ✅ local | +| Bot-free system-audio capture | ✅ | ✅ | ✅ | +| License | proprietary | MIT | open source | + +## Technology + +WhispAssist is built as a **Tauri 2** application: a small Rust core with a compiled +**Svelte** web frontend rendered through the OS WebView2 (no bundled browser → low idle +memory). The choice and its alternatives are recorded in [`docs/adr/`](docs/adr/). + +- **Shell / IPC:** Tauri 2 (Rust ⇄ WebView2) +- **Audio capture:** WASAPI loopback (`wasapi` crate) +- **Transcription:** `whisper-rs` (whisper.cpp: CPU/Vulkan/CUDA) + ONNX Runtime (`ort`) DirectML for the NPU path +- **Diarization:** `sherpa-onnx` (pyannote segmentation + speaker-embedding clustering), fully offline +- **Storage:** SQLite (`sqlx`/`rusqlite`) + on-disk audio/transcript files +- **Local LLM:** Ollama HTTP API on `localhost:11434` +- **Calendar / Outlook:** `outlook-pst` crate for `.pst`, OS notifications for reminders +- **Optional recording:** opt-in (default off), saved as `.wav`, with a consent reminder (ADR-0009) +- **Optional sync:** upload artifacts to your own server — WebDAV covers **Nextcloud, ownCloud, Cloudreve, Seafile, Synology**; **OneDrive/Dropbox/Box** via OAuth. Off by default (ADR-0010) +- **Optional AI/agent integration:** hosted summary providers (Anthropic, OpenAI-compatible) behind the same provider model, **and** a local **MCP server** so your own coding agents (**Claude, Codex, Copilot, OpenCode, …**) can pull meeting context and "feature briefs" to start coding. Off by default; the MCP server is inbound/loopback only (ADR-0011) + +## Repository layout + +``` +WhispAssist/ +├── docs/ # The engineering plan (read this first) +│ ├── 00-overview.md Vision, goals, glossary +│ ├── 01-requirements.md Functional + non-functional requirements (traceable IDs) +│ ├── 02-architecture.md Components, data flow, threading model +│ ├── 03-data-model.md SQLite schema, file layout, transcript JSON +│ ├── 04-api-contracts.md Tauri commands/events + internal Rust service traits +│ ├── 05-roadmap.md 8 phases, task breakdown, acceptance criteria +│ ├── 06-test-strategy.md Test plan per phase + quality gates +│ ├── 07-research-findings.md Validated stack with sources +│ └── adr/ Architecture Decision Records (0001–0010) +├── src-tauri/ # Rust core (service module skeletons) +├── src/ # Svelte frontend skeleton +├── scripts/ # Dev/model-download helper scripts +└── tests/ # Test fixtures + integration test scaffolding +``` + +## Getting started (for builders) + +Prerequisites once implementation begins: Rust (stable), Node.js 20+, the Tauri CLI, and +WebView2 runtime (preinstalled on Windows 11). Then: + +```bash +npm install +npm run tauri dev # once src-tauri/Cargo.toml dependencies are filled in +``` + +The current skeleton intentionally does **not** compile end-to-end — modules contain typed +stubs and `todo!()` markers that map 1:1 to roadmap tasks. Start at Phase 1 in +[`docs/05-roadmap.md`](docs/05-roadmap.md). + +## Privacy guarantee + +WhispAssist originates **no outbound connection for audio or transcript content** except to +destinations **you explicitly configure** — an LLM endpoint (local Ollama by default, or a hosted AI +provider if you choose one) and any sync targets you enable — plus explicit model downloads. +Everything optional is **off by default**; with nothing configured, WA makes no content egress at +all. The local **MCP server** (for handing meetings to your coding agents) is **inbound on loopback +and adds no egress** — data only leaves via the agent's own provider, which WA discloses. The set of +reachable hosts is an allowlist derived from your settings and enforced in the core (and verified by +a CI network test). Recording is opt-in; sync/AI credentials live in the OS credential store, never +in config files. See the privacy requirements (`FR-SEC-*`, `NFR-SEC-*`, `FR-SYNC-*`, `FR-MCP-*`) in +[`docs/01-requirements.md`](docs/01-requirements.md) and ADRs 0009–0011. + +## License + +Dual-licensed under MIT or Apache-2.0, at your option. See [`LICENSE`](LICENSE).