Files

T

iamdoubz aae22a0e23 Add all scaffolding, design, and decision documentation AKA first commit

2026-06-30 16:12:55 -05:00

16 KiB

Raw Blame History

05 — Roadmap

Eight phases, each a shippable increment. Every task carries an ID and the requirement(s) it satisfies; every phase has acceptance criteria that gate "done" (verified by the tests in 06-test-strategy.md). Phases are ordered so each builds on a working previous one. Suggested effort labels: S ≤1 day, M a few days, L ~1–2 weeks (team-dependent, indicative only).

Legend: [T1.2] = task; → FR/NFR satisfied.

Phase 1 — Foundation: capture + CPU transcription

Goal: a window where you click Record, system audio is captured to disk, and plain transcript text streams in on CPU.

[T1.1] Scaffold Tauri 2 + Svelte project; window, build, dev loop. M → ADR-0001/0002
[T1.2] audio: WASAPI loopback capture thread (polling), PCM ring buffer, WAV writer. L → FR-CAP-1/2
[T1.3] Recording lifecycle commands + recording://state events; start/stop/pause/resume. M → FR-CAP-3
[T1.4] Recording indicator: in-app banner + tray icon. S → FR-CAP-4
[T1.5] transcription: integrate whisper-rs CPU; load small model; window+overlap feeder. L → FR-TRX-1, NFR-MNT-4
[T1.6] Stream interim/final segments → transcript://segment; minimal transcript view. M → FR-TRX-2
[T1.7] Idle discipline: no threads/timers when not recording; model unloaded when idle. S → NFR-RES-1
[T1.8] "Record this meeting" toggle (default OFF) + mid-meeting set_recording_retention; "saving" indicator. S → FR-REC-1/3
[T1.9] Consent notice + one-time acknowledgment before first retained recording. S → FR-REC-2

Acceptance: Record a 5-min meeting; transcript appears live within a few seconds of speech on a CPU-only laptop; stopping leaves the app idle with near-zero CPU. With "Record" ON, a playable audio.wav remains; with it OFF (default), no audio file remains after finalize. The consent notice appears before the first retained recording and must be acknowledged once. (Tests: P1 suite.)

Phase 2 — Storage, notes, persistence, recovery

Goal: meetings are saved, listable, editable as Markdown, exportable, and survive a crash.

[T2.1] storage: SQLite schema + migrations; meeting folder layout; transactional writes. L → FR-STORE-1
[T2.2] Persist transcript.json + metadata on stop; status lifecycle. M → FR-STORE-1
[T2.3] Meetings list (create/open/delete) UI + commands. M → FR-STORE-5
[T2.4] notes: assemble Markdown (speaker-tagged) from segments; render + raw side-by-side. M → FR-NOTE-1/6
[T2.5] Markdown editor with toolbar (headings/bold/italic/lists/checkboxes); update_notes. M → FR-NOTE-2
[T2.6] Export single meeting to .md and bundle (audio+transcript+notes). M → FR-NOTE-3, FR-STORE-3
[T2.10] Finalize path honors retention: delete working WAV only after transcript finalized; set recorded/audio_path (FR-REC-4). S → FR-REC-1/4
[T2.7] Auto-save notes/transcript at intervals. S → FR-REL-2
[T2.8] Startup recover-scan: audio-without-transcript → recovering; re-run flow. M → FR-REL-1, NFR-REL-3
[T2.9] Storage settings: base directory + retention policy (size/age) + background enforcement. M → FR-STORE-2

Acceptance: Kill the app mid-meeting; on relaunch the meeting is recoverable from audio and transcription can be regenerated. Notes edit + .md export round-trip. Retention deletes the oldest meeting when the cap is exceeded and never touches a recording in progress. (Tests: P2 suite.)

Phase 3 — Hardware acceleration + model management

Goal: detect and use NPU/GPU; switch models; show the active backend; batch re-transcribe.

[T3.1] hardware: enumerate CPU/GPU (DXGI) + NPU (ONNX/Windows ML); rank backends. L → FR-HW-1
[T3.2] Backend selection (auto order + manual override); construct matching Transcriber. M → FR-HW-2
[T3.3] whisper.cpp accelerated builds behind Cargo features (CUDA/Vulkan); CPU always builds. L → FR-TRX-5, NFR-MNT-4
[T3.4] NPU path: ort + DirectML Whisper-ONNX Transcriber (same trait/output). L → FR-TRX-5, ADR-0004
[T3.5] Graceful fallback NPU→GPU→CPU on failure; hardware://changed notice. M → FR-HW-4, NFR-REL-2
[T3.6] Show active backend, model size, est. RTF in Settings + recording bar. S → FR-HW-3
[T3.7] Model management UI: choose Whisper size; download/remove with progress. M → FR-MODEL-1
[T3.8] Batch re-transcription with a larger model post-meeting. M → FR-TRX-3
[T3.9] "Low overhead" preset (CPU + small model + no realtime summary + reduced indexing). S → NFR-RES-3, NFR-PERF-3

Acceptance: On a machine with a supported GPU/NPU, the active backend is detected and shown, and transcription is measurably faster than the CPU baseline; forcing a backend failure falls back cleanly to CPU without aborting. Output segments are equivalent across backends. (Tests: P3 suite.)

Phase 4 — Speaker diarization & naming

Goal: segments get speaker labels; users name/merge speakers during and after meetings.

[T4.1] diarization: integrate sherpa-onnx (segmentation+embedding+clustering) via FFI. L → FR-SPK-1, ADR-0005
[T4.2] Align speaker spans to transcript segments by timestamp overlap. M → FR-SPK-1
[T4.3] Live provisional speaker turns from cheap segmentation during recording. M → FR-SPK-2
[T4.4] In-session naming: name a speaker → applies to past & future segments. M → FR-SPK-2
[T4.5] Post-meeting review screen: rename + merge over-split speakers. M → FR-SPK-3
[T4.6] Persist speaker→name mapping; apply at render/export (non-destructive). S → FR-SPK-5
[T4.7] Diarization models in model-management; download/select. S → FR-MODEL-1

Acceptance: A two-person recording yields ≥2 speakers; naming a speaker relabels all their segments live; merging two IDs into one updates notes and export; renaming never rewrites segment speaker IDs in storage. (Tests: P4 suite.)

Phase 5 — Local LLM integration

Goal: configure a local LLM and get summaries, decisions, and editable action items.

[T5.1] llm: Ollama client (/api/tags, /api/chat streaming) behind LlmProvider. M → FR-LLM-1, ADR-0007
[T5.2] Provider settings: Ollama | custom endpoint | off; llm_status + reachability. M → FR-LLM-1
[T5.3] Local-endpoint guard + "data leaves WA" banner for remote/proxy endpoints. S → FR-LLM-6, FR-SEC-1
[T5.4] Prompt assembly: transcript + metadata + optional template → summary/decisions/actions. M → FR-LLM-2
[T5.5] Stream tokens to summary panel (llm://token/done); persist summary.json. M → FR-LLM-4
[T5.6] Parse action items into editable list; confirm → action_items rows. M → FR-LLM-3
[T5.7] Guided provider install + hardware-aware model suggestions when none detected. M → FR-LLM-5

Acceptance: With Ollama running, a finished meeting produces a streamed summary plus a list of action items the user can edit and confirm; with the LLM off or unreachable, capture/transcription/ notes still work and the UI degrades gracefully. (Tests: P5 suite.)

Phase 6 — Calendar & Outlook .pst integration

Goal: import .pst events/attendees, attach recordings to events, suggest speaker names.

[T6.1] calendar: .pst reader (outlook-pst; libpff fallback) → events + attendees. L → FR-CAL-1, ADR-0008
[T6.2] import_pst command with optional password; progress events; persist events/participants. M → FR-CAL-1
[T6.3] Calendar/meetings view: upcoming/recent; attach a recording to an event. M → FR-CAL-2
[T6.4] Pre-meeting context panel: title, organizer, participants. S → FR-CAL-3
[T6.5] Participant-aware naming: attendee dropdown in speaker UI + "add new name". M → FR-SPK-4
[T6.6] Link recording ↔ historical .pst event; continuity of named speakers. M → FR-CAL-4

Acceptance: Importing a sample .pst yields events with attendees; selecting an event prefills the speaker-naming dropdown; PST parse failure is non-fatal and surfaced clearly. (Tests: P6 suite.)

Phase 7 — UX polish, themes, accessibility, privacy panel

Goal: the modern, fast, themeable, accessible experience.

[T7.1] Final three-pane layout (list · transcript/notes · summary/actions/participants). M → FR-UX-1
[T7.2] Light/dark themes following system + override; tokens/contrast pass. M → FR-UX-2
[T7.3] Live waveform/level meter; device-change resilience. M → FR-CAP-5/6
[T7.4] Keyboard shortcuts (record start/stop, view toggles, template apply). S → FR-UX-3
[T7.5] Screen-reader labels, focus order, ARIA; contrast audit. M → FR-UX-4
[T7.6] Privacy panel: active backend + LLM endpoint + local-only confirmation; privacy_self_check. M → FR-SEC-2
[T7.7] Startup/perf pass: cold start < ~2 s, lazy-load models, idle audit. M → NFR-PERF-4, NFR-RES-1

Acceptance: Themes switch with the OS and override; keyboard-only operation works for core flows; a contrast/screen-reader audit passes; the privacy panel accurately reflects egress; cold start meets target. (Tests: P7 suite.)

Phase 8 — Advanced features

Goal: templates, search/tagging, reminders, richer export, encryption.

[T8.1] Note templates by meeting type (sections per type); apply on creation. M → FR-NOTE-5
[T8.2] Full-text search (FTS5) across transcripts + notes. M → FR-SEARCH-1
[T8.3] Tagging + list filters (date/tag/participant). M → FR-SEARCH-2
[T8.4] PDF/Word export via local conversion. M → FR-NOTE-4
[T8.5] Bulk export by date range/tag. S → FR-STORE-4
[T8.6] Local reminders / OS notifications for action items. M → FR-CAL-5
[T8.7] Multi-language transcription + UI localization scaffold. M → FR-TRX-4
[T8.8] Optional at-rest encryption + password-protected vault. L → FR-SEC-3
[T8.9] (Later) Microsoft Graph calendar source behind CalendarSource (opt-in, consented). L → FR-CAL-6

Acceptance: Search returns relevant meetings; templates structure new notes; PDF/Word exports render correctly offline; reminders fire locally; encryption (when enabled) protects the storage root and unlocks with the vault password. (Tests: P8 suite.)

Phase 9 — Remote sync / upload

Goal: optionally upload meeting artifacts to user-configured destinations — off by default, self-hostable first. (ADR-0010.) Can begin in parallel after Phase 2 since it only needs the storage/file layer; OAuth sub-phase is independent of the WebDAV one.

9a — Core + WebDAV primary set (Nextcloud, ownCloud, Cloudreve, Seafile, Synology)

[T9.1] sync module: SyncTarget/SyncManager traits; sync_targets/sync_jobs schema (migration 0002). M → FR-SYNC-1
[T9.2] WebDavTarget: PUT/MKCOL/PROPFIND; per-provider base paths; chunked upload for large files. L → FR-SYNC-2
[T9.3] Target config UI + add/update/remove/list_sync_targets; per-target artifact + trigger selection. M → FR-SYNC-2/3
[T9.4] test_sync_target (reachability + auth) with provider-specific setup hints (e.g. enable SeafDAV). M → FR-SYNC-4
[T9.5] Durable queue: enqueue on finalize / "Upload now"; backoff retry; SHA-256 skip-if-unchanged; sync://job events. L → FR-SYNC-5
[T9.6] Credentials in OS credential store (keyring); never in settings/DB. M → FR-SYNC-6, NFR-SEC-4
[T9.7] TLS enforcement; refuse plaintext http except explicit per-target LAN opt-in. S → FR-SYNC-7
[T9.8] Sync state + target labeling in UI (self-hosted vs third-party); wire into privacy panel + privacy_self_check. M → FR-SYNC-8, FR-SEC-2

9b — Secondary OAuth targets (OneDrive, Dropbox, Box)

[T9.9] OAuth 2.0 PKCE with loopback redirect; token storage in credential store; begin_oauth_link. L → FR-SYNC-9
[T9.10] OneDriveTarget (MS Graph), DropboxTarget, BoxTarget: upload + resumable sessions behind SyncTarget. L → FR-SYNC-9
[T9.11] Third-party "data leaves your device" banner + explicit enable acknowledgment. S → FR-SYNC-8

9c — Optional

[T9.12] Client-side encryption before upload (uses the Phase 8 vault): destination holds only ciphertext. M → FR-SYNC-10

Acceptance: With sync disabled (default), the egress allowlist contains no sync hosts and nothing uploads. After adding and enabling a WebDAV target (tested against a real Nextcloud/Seafile or an rclone serve webdav instance), finishing a meeting uploads the selected artifacts to the correct remote path; interrupting the network leaves a failed job that retries and completes; an unchanged re-upload is skipped. Credentials never appear in settings.json/wa.db. Third-party targets show the leaves-your-device banner. (Tests: P9 suite.)

Phase 10 — External AI & coding-agent integration

Goal: optionally use hosted AI for summaries, and hand a meeting off to the user's coding agent to "get started right away." (ADR-0011.) All off by default. Needs Phase 5 (LLM) for distillation; otherwise independent.

10a — Cloud summary providers (Layer 1)

[T10.1] Extend LlmProvider with OpenAiCompatProvider (/v1/chat/completions) and AnthropicProvider (/v1/messages). M → FR-AI-1
[T10.2] set_llm_provider; API keys in OS credential store; host added to egress allowlist; third-party banner. M → FR-AI-2, FR-SEC-1
[T10.3] Per-use provider selection + active-provider display. S → FR-AI-3

10b — WhispAssist as an MCP server + feature briefs (Layer 2, the primary handoff)

[T10.4] mcp module on rmcp: loopback MCP server, off by default, token-gated; Streamable HTTP (/mcp) + stdio adapter. L → FR-MCP-1/6, NFR-SEC-5
[T10.5] Tools-first surface: list_recent_meetings, get_transcript, get_action_items, get_feature_brief. M → FR-MCP-2
[T10.6] FeatureBriefBuilder: distill transcript → structured brief; create/list/get_feature_brief; persist to briefs/. L → FR-MCP-4
[T10.7] Scope control (none|selected|all, recordings excluded by default) + set_brief_exposed. M → FR-MCP-3
[T10.8] Disclosure UI ("connected agents may forward data") + mcp_access_log audit. M → FR-MCP-5
[T10.9] Privacy panel + privacy_self_check show MCP state and confirm it adds no egress. S → FR-MCP-7, FR-SEC-2

10c — Push & task-tracker handoff (Layer 3, later)

[T10.10] AgentRunner: spawn claude -p / codex exec / opencode run / copilot headless against a repo; stream output. L → FR-AGENT-1
[T10.11] IssueTracker: create a GitHub issue from a brief; optional assign-to-Copilot-cloud. M → FR-AGENT-2

Acceptance: With everything off (default), no AI host is on the allowlist and no MCP port is open. Enabling a hosted provider routes a summary to it and shows the third-party banner. Enabling the MCP server lets a local agent (e.g. Claude Code) call get_feature_brief and receive a usable spec; the access is logged; WA opens no outbound socket for it (egress test unchanged). Recordings are never served unless explicitly allowed. (Tests: P10 suite.)

Cross-cutting (every phase)

Privacy invariant FR-SEC-1 verified continuously: the CI network test asserts egress stays within the settings-derived allowlist (LLM + enabled sync hosts + model downloads). See 06-test-strategy.md.
cargo fmt/clippy -D warnings + Prettier/ESLint clean.
Update docs/ whenever a contract or behavior changes (CLAUDE.md rule).

Dependency graph (high level)

P1 ─► P2 ─► P3 ─► P4 ─► P5 ─► P10 (AI/MCP; 10a/10b need P5's LLM)
        │        └► P6 ─► P7 ─► P8
        └► P9 (sync; needs only P2's storage layer — can run in parallel)
P5 and P6 can proceed in parallel after P4; P7 needs P3–P6 features to polish; P8 last.
P9 needs only P2; 9b (OAuth) is independent of 9a (WebDAV); 9c needs P8's vault.
P10 needs P5; 10b (MCP) is the priority; 10c (push/issue) is later and optional.

Suggested first milestone (thin vertical slice)

T1.1 → T1.2 → T1.5 → T1.6 → T2.1 → T2.2 → T2.4 gives a usable "record → live transcript → saved Markdown notes" loop — the smallest thing worth dogfooding.

16 KiB Raw Blame History Unescape Escape