0 Fork 0

Files

T

iamdoubz 4cf5eb89b4 docs(roadmap): post-v0.2.0 execution plan (Phase 10 + reliability milestones)

2026-07-06 23:19:51 -05:00

23 KiB

Raw Blame History

05 — Roadmap

Eight phases, each a shippable increment. Every task carries an ID and the requirement(s) it satisfies; every phase has acceptance criteria that gate "done" (verified by the tests in 06-test-strategy.md). Phases are ordered so each builds on a working previous one. Suggested effort labels: S ≤1 day, M a few days, L ~1–2 weeks (team-dependent, indicative only).

Legend: [T1.2] = task; → FR/NFR satisfied.

Phase 1 — Foundation: capture + CPU transcription

Goal: a window where you click Record, system audio is captured to disk, and plain transcript text streams in on CPU.

[T1.1] Scaffold Tauri 2 + Svelte project; window, build, dev loop. M → ADR-0001/0002
[T1.2] audio: WASAPI loopback capture thread (polling), PCM ring buffer, WAV writer. L → FR-CAP-1/2
[T1.3] Recording lifecycle commands + recording://state events; start/stop/pause/resume. M → FR-CAP-3
[T1.4] Recording indicator: in-app banner + tray icon. S → FR-CAP-4
[T1.5] transcription: integrate whisper-rs CPU; load small model; window+overlap feeder. L → FR-TRX-1, NFR-MNT-4
[T1.6] Stream interim/final segments → transcript://segment; minimal transcript view. M → FR-TRX-2
[T1.7] Idle discipline: no threads/timers when not recording; model unloaded when idle. S → NFR-RES-1
[T1.8] "Record this meeting" toggle (default OFF) + mid-meeting set_recording_retention; "saving" indicator. S → FR-REC-1/3
[T1.9] Consent notice + one-time acknowledgment before first retained recording. S → FR-REC-2

Acceptance: Record a 5-min meeting; transcript appears live within a few seconds of speech on a CPU-only laptop; stopping leaves the app idle with near-zero CPU. With "Record" ON, a playable audio.wav remains; with it OFF (default), no audio file remains after finalize. The consent notice appears before the first retained recording and must be acknowledged once. (Tests: P1 suite.)

Phase 2 — Storage, notes, persistence, recovery

Goal: meetings are saved, listable, editable as Markdown, exportable, and survive a crash.

[T2.1] storage: SQLite schema + migrations; meeting folder layout; transactional writes. L → FR-STORE-1
[T2.2] Persist transcript.json + metadata on stop; status lifecycle. M → FR-STORE-1
[T2.3] Meetings list (create/open/delete) UI + commands. M → FR-STORE-5
[T2.4] notes: assemble Markdown (speaker-tagged) from segments; render + raw side-by-side. M → FR-NOTE-1/6
[T2.5] Markdown editor with toolbar (headings/bold/italic/lists/checkboxes); update_notes. M → FR-NOTE-2
[T2.6] Export single meeting to .md and bundle (audio+transcript+notes). M → FR-NOTE-3, FR-STORE-3
[T2.10] Finalize path honors retention: delete working WAV only after transcript finalized; set recorded/audio_path (FR-REC-4). S → FR-REC-1/4
[T2.7] Auto-save notes/transcript at intervals. S → FR-REL-2
[T2.8] Startup recover-scan: audio-without-transcript → recovering; re-run flow. M → FR-REL-1, NFR-REL-3
[T2.9] Storage settings: base directory + retention policy (size/age) + background enforcement. M → FR-STORE-2

Acceptance: Kill the app mid-meeting; on relaunch the meeting is recoverable from audio and transcription can be regenerated. Notes edit + .md export round-trip. Retention deletes the oldest meeting when the cap is exceeded and never touches a recording in progress. (Tests: P2 suite.)

Phase 3 — Hardware acceleration + model management

Goal: detect and use NPU/GPU; switch models; show the active backend; batch re-transcribe.

[T3.1] hardware: enumerate CPU/GPU (DXGI) + NPU (ONNX/Windows ML); rank backends. L → FR-HW-1
[T3.2] Backend selection (auto order + manual override); construct matching Transcriber. M → FR-HW-2
[T3.3] whisper.cpp accelerated builds behind Cargo features (CUDA/Vulkan); CPU always builds. L → FR-TRX-5, NFR-MNT-4
[T3.4] NPU path: ort + DirectML Whisper-ONNX Transcriber (same trait/output). L → FR-TRX-5, ADR-0004
[T3.5] Graceful fallback NPU→GPU→CPU on failure; hardware://changed notice. M → FR-HW-4, NFR-REL-2
[T3.6] Show active backend, model size, est. RTF in Settings + recording bar. S → FR-HW-3
[T3.7] Model management UI: choose Whisper size; download/remove with progress. M → FR-MODEL-1
[T3.8] Batch re-transcription with a larger model post-meeting. M → FR-TRX-3
[T3.9] "Low overhead" preset (CPU + small model + no realtime summary + reduced indexing). S → NFR-RES-3, NFR-PERF-3

Acceptance: On a machine with a supported GPU/NPU, the active backend is detected and shown, and transcription is measurably faster than the CPU baseline; forcing a backend failure falls back cleanly to CPU without aborting. Output segments are equivalent across backends. (Tests: P3 suite.)

Phase 4 — Speaker diarization & naming

Goal: segments get speaker labels; users name/merge speakers during and after meetings.

[T4.1] diarization: integrate sherpa-onnx (segmentation+embedding+clustering) via FFI. L → FR-SPK-1, ADR-0005
[T4.2] Align speaker spans to transcript segments by timestamp overlap. M → FR-SPK-1
[T4.3] Live provisional speaker turns from cheap segmentation during recording. M → FR-SPK-2
[T4.4] In-session naming: name a speaker → applies to past & future segments. M → FR-SPK-2
[T4.5] Post-meeting review screen: rename + merge over-split speakers. M → FR-SPK-3
[T4.6] Persist speaker→name mapping; apply at render/export (non-destructive). S → FR-SPK-5
[T4.7] Diarization models in model-management; download/select. S → FR-MODEL-1

Acceptance: A two-person recording yields ≥2 speakers; naming a speaker relabels all their segments live; merging two IDs into one updates notes and export; renaming never rewrites segment speaker IDs in storage. (Tests: P4 suite.)

Phase 5 — Local LLM integration

Goal: configure a local LLM and get summaries, decisions, and editable action items.

[T5.1] llm: Ollama client (/api/tags, /api/chat streaming) behind LlmProvider. M → FR-LLM-1, ADR-0007
[T5.2] Provider settings: Ollama | custom endpoint | off; llm_status + reachability. M → FR-LLM-1
[T5.3] Local-endpoint guard + "data leaves WA" banner for remote/proxy endpoints. S → FR-LLM-6, FR-SEC-1
[T5.4] Prompt assembly: transcript + metadata + optional template → summary/decisions/actions. M → FR-LLM-2
[T5.5] Stream tokens to summary panel (llm://token/done); persist summary.json. M → FR-LLM-4
[T5.6] Parse action items into editable list; confirm → action_items rows. M → FR-LLM-3
[T5.7] Guided provider install + hardware-aware model suggestions when none detected. M → FR-LLM-5

Acceptance: With Ollama running, a finished meeting produces a streamed summary plus a list of action items the user can edit and confirm; with the LLM off or unreachable, capture/transcription/ notes still work and the UI degrades gracefully. (Tests: P5 suite.)

Phase 6 — Calendar & Outlook .pst integration

Goal: import .pst events/attendees, attach recordings to events, suggest speaker names.

[T6.1] calendar: .pst reader (outlook-pst; libpff fallback) → events + attendees. L → FR-CAL-1, ADR-0008
[T6.2] import_pst command with optional password; progress events; persist events/participants. M → FR-CAL-1
[T6.3] Calendar/meetings view: upcoming/recent; attach a recording to an event. M → FR-CAL-2
[T6.4] Pre-meeting context panel: title, organizer, participants. S → FR-CAL-3
[T6.5] Participant-aware naming: attendee dropdown in speaker UI + "add new name". M → FR-SPK-4
[T6.6] Link recording ↔ historical .pst event; continuity of named speakers. M → FR-CAL-4

Acceptance: Importing a sample .pst yields events with attendees; selecting an event prefills the speaker-naming dropdown; PST parse failure is non-fatal and surfaced clearly. (Tests: P6 suite.)

Phase 7 — UX polish, themes, accessibility, privacy panel

Goal: the modern, fast, themeable, accessible experience.

[T7.1] Final three-pane layout (list · transcript/notes · summary/actions/participants). M → FR-UX-1
[T7.2] Light/dark themes following system + override; tokens/contrast pass. M → FR-UX-2
[T7.3] Live waveform/level meter; device-change resilience. M → FR-CAP-5/6
[T7.4] Keyboard shortcuts (record start/stop, view toggles, template apply). S → FR-UX-3
[T7.5] Screen-reader labels, focus order, ARIA; contrast audit. M → FR-UX-4
[T7.6] Privacy panel: active backend + LLM endpoint + local-only confirmation; privacy_self_check. M → FR-SEC-2
[T7.7] Startup/perf pass: cold start < ~2 s, lazy-load models, idle audit. M → NFR-PERF-4, NFR-RES-1

Acceptance: Themes switch with the OS and override; keyboard-only operation works for core flows; a contrast/screen-reader audit passes; the privacy panel accurately reflects egress; cold start meets target. (Tests: P7 suite.)

Phase 8 — Advanced features

Goal: templates, search/tagging, reminders, richer export, encryption.

[T8.1] Note templates by meeting type (sections per type); apply on creation. M → FR-NOTE-5
[T8.2] Full-text search (FTS5) across transcripts + notes. M → FR-SEARCH-1
[T8.3] Tagging + list filters (date/tag/participant). M → FR-SEARCH-2
[T8.4] PDF/Word export via local conversion. M → FR-NOTE-4
[T8.5] Bulk export by date range/tag. S → FR-STORE-4
[T8.6] Local reminders / OS notifications for action items. M → FR-CAL-5
[T8.7] Multi-language transcription + UI localization scaffold. M → FR-TRX-4
[T8.8] Optional at-rest encryption + password-protected vault. L → FR-SEC-3
[T8.9] (Later) Microsoft Graph calendar source behind CalendarSource (opt-in, consented). L → FR-CAL-6

Acceptance: Search returns relevant meetings; templates structure new notes; PDF/Word exports render correctly offline; reminders fire locally; encryption (when enabled) protects the storage root and unlocks with the vault password. (Tests: P8 suite.)

Phase 9 — Remote sync / upload

Goal: optionally upload meeting artifacts to user-configured destinations — off by default, self-hostable first. (ADR-0010.) Can begin in parallel after Phase 2 since it only needs the storage/file layer; OAuth sub-phase is independent of the WebDAV one.

9a — Core + WebDAV primary set (Nextcloud, ownCloud, Cloudreve, Seafile, Synology)

[T9.1] sync module: SyncTarget/SyncManager traits; sync_targets/sync_jobs schema (migration 0002). M → FR-SYNC-1
[T9.2] WebDavTarget: PUT/MKCOL/PROPFIND; per-provider base paths; chunked upload for large files. L → FR-SYNC-2
[T9.3] Target config UI + add/update/remove/list_sync_targets; per-target artifact + trigger selection. M → FR-SYNC-2/3
[T9.4] test_sync_target (reachability + auth) with provider-specific setup hints (e.g. enable SeafDAV). M → FR-SYNC-4
[T9.5] Durable queue: enqueue on finalize / "Upload now"; backoff retry; SHA-256 skip-if-unchanged; sync://job events. L → FR-SYNC-5
[T9.6] Credentials in OS credential store (keyring); never in settings/DB. M → FR-SYNC-6, NFR-SEC-4
[T9.7] TLS enforcement; refuse plaintext http except explicit per-target LAN opt-in. S → FR-SYNC-7
[T9.8] Sync state + target labeling in UI (self-hosted vs third-party); wire into privacy panel + privacy_self_check. M → FR-SYNC-8, FR-SEC-2

9b — Secondary OAuth targets (OneDrive, Dropbox, Box)

[T9.9] OAuth 2.0 PKCE with loopback redirect; token storage in credential store; begin_oauth_link. L → FR-SYNC-9
[T9.10] OneDriveTarget (MS Graph), DropboxTarget, BoxTarget: upload + resumable sessions behind SyncTarget. L → FR-SYNC-9
[T9.11] Third-party "data leaves your device" banner + explicit enable acknowledgment. S → FR-SYNC-8

9c — Optional

[T9.12] Client-side encryption before upload (uses the Phase 8 vault): destination holds only ciphertext. M → FR-SYNC-10

Acceptance: With sync disabled (default), the egress allowlist contains no sync hosts and nothing uploads. After adding and enabling a WebDAV target (tested against a real Nextcloud/Seafile or an rclone serve webdav instance), finishing a meeting uploads the selected artifacts to the correct remote path; interrupting the network leaves a failed job that retries and completes; an unchanged re-upload is skipped. Credentials never appear in settings.json/wa.db. Third-party targets show the leaves-your-device banner. (Tests: P9 suite.)

Phase 10 — External AI & coding-agent integration

Goal: optionally use hosted AI for summaries, and hand a meeting off to the user's coding agent to "get started right away." (ADR-0011.) All off by default. Needs Phase 5 (LLM) for distillation; otherwise independent.

10a — Cloud summary providers (Layer 1)

[T10.1] Extend LlmProvider with OpenAiCompatProvider (/v1/chat/completions) and AnthropicProvider (/v1/messages). M → FR-AI-1
[T10.2] set_llm_provider; API keys in OS credential store; host added to egress allowlist; third-party banner. M → FR-AI-2, FR-SEC-1
[T10.3] Per-use provider selection + active-provider display. S → FR-AI-3

10b — WhispAssist as an MCP server + feature briefs (Layer 2, the primary handoff)

[T10.4] mcp module on rmcp: loopback MCP server, off by default, token-gated; Streamable HTTP (/mcp) + stdio adapter. L → FR-MCP-1/6, NFR-SEC-5
[T10.5] Tools-first surface: list_recent_meetings, get_transcript, get_action_items, get_feature_brief. M → FR-MCP-2
[T10.6] FeatureBriefBuilder: distill transcript → structured brief; create/list/get_feature_brief; persist to briefs/. L → FR-MCP-4
[T10.7] Scope control (none|selected|all, recordings excluded by default) + set_brief_exposed. M → FR-MCP-3
[T10.8] Disclosure UI ("connected agents may forward data") + mcp_access_log audit. M → FR-MCP-5
[T10.9] Privacy panel + privacy_self_check show MCP state and confirm it adds no egress. S → FR-MCP-7, FR-SEC-2

10c — Push & task-tracker handoff (Layer 3, later)

[T10.10] AgentRunner: spawn claude -p / codex exec / opencode run / copilot headless against a repo; stream output. L → FR-AGENT-1
[T10.11] IssueTracker: create a GitHub issue from a brief; optional assign-to-Copilot-cloud. M → FR-AGENT-2

Acceptance: With everything off (default), no AI host is on the allowlist and no MCP port is open. Enabling a hosted provider routes a summary to it and shows the third-party banner. Enabling the MCP server lets a local agent (e.g. Claude Code) call get_feature_brief and receive a usable spec; the access is logged; WA opens no outbound socket for it (egress test unchanged). Recordings are never served unless explicitly allowed. (Tests: P10 suite.)

Cross-cutting (every phase)

Privacy invariant FR-SEC-1 verified continuously: the CI network test asserts egress stays within the settings-derived allowlist (LLM + enabled sync hosts + model downloads). See 06-test-strategy.md.
cargo fmt/clippy -D warnings + Prettier/ESLint clean.
Update docs/ whenever a contract or behavior changes (CLAUDE.md rule).

Dependency graph (high level)

P1 ─► P2 ─► P3 ─► P4 ─► P5 ─► P10 (AI/MCP; 10a/10b need P5's LLM)
        │        └► P6 ─► P7 ─► P8
        └► P9 (sync; needs only P2's storage layer — can run in parallel)
P5 and P6 can proceed in parallel after P4; P7 needs P3–P6 features to polish; P8 last.
P9 needs only P2; 9b (OAuth) is independent of 9a (WebDAV); 9c needs P8's vault.
P10 needs P5; 10b (MCP) is the priority; 10c (push/issue) is later and optional.

Suggested first milestone (thin vertical slice)

T1.1 → T1.2 → T1.5 → T1.6 → T2.1 → T2.2 → T2.4 gives a usable "record → live transcript → saved Markdown notes" loop — the smallest thing worth dogfooding.

Remaining work — post-v0.2.0 execution plan

As of v0.2.0, Phases 1–9 ship (capture incl. microphone, CPU/NPU/Vulkan transcription, diarization, storage/recovery, notes, local-LLM summaries, PST calendar, UX/a11y, templates/search/tags/export/reminders/encryption, WebDAV + OneDrive sync) in a single universal installer. What remains is Phase 10 (external AI & agent handoff, ADR-0011) plus a few breadth/reliability items. This section sequences that work by leverage, dependency, and the privacy invariant. Task IDs reference the Phase 10 list above; finer sub-tasks add a letter suffix. Traits/contracts for all of this already exist in 04-api-contracts.md (FeatureBriefBuilder, McpServer, hosted LlmProvider impls, AgentRunner, IssueTracker); tests live in 06-test-strategy.md (P10 + the cross-cutting egress gate).

Status snapshot (what's actually in the tree):

Built: Phases 1–9; OpenAiCompatProvider (used as the Phase 5 "custom" endpoint).
Stubbed (commands return Err(not_implemented(...))): create/list/get_feature_brief, set_brief_exposed, mcp_status, set_mcp_enabled, set_mcp_scope, mcp_access_log, run_agent, create_issue_from_brief.
Skeleton/absent: mcp/mod.rs (trait + todo!() only); AnthropicProvider (returns "isn't built yet"); MS Graph CalendarSource; multi-language selection; DropboxTarget/ BoxTarget (OAuth wired in sync/oauth.rs, upload impl missing); chunked/resumable upload.

Sequence: M1 → M2 (briefs are the MCP payload); M3 can run in parallel with M1/M2; M4 is opportunistic; M5 is last and optional.

M1 — Feature briefs (do first; standalone value) → FR-MCP-4 (T10.6)

Needs only the already-built LLM, delivers value before any MCP transport (view/copy a brief), and is the exact payload M2 serves. No new egress (uses the configured LlmProvider).

[M1.1] FeatureBriefBuilder::build(meeting_id, target_repo): prompt-assemble transcript + metadata → {problem, desired_outcome, acceptance_criteria[], target_repo, context_excerpts[]} via the configured provider; tolerant parse with graceful fallback on a malformed model reply. M
[M1.2] Persist to briefs/<id>.json (schema-versioned per 03-data-model.md); implement create_feature_brief/list_feature_briefs/get_feature_brief (replace the stubs). M
[M1.3] UI: "Create feature brief" on a finished meeting; view/edit/copy; target-repo field. M
[M1.4] set_brief_exposed + persisted exposed flag (the scope foundation M2 reads). S
Acceptance: a golden transcript yields a schema-valid brief; it persists and re-opens; with the LLM off/unreachable the command errors cleanly (no panic, no partial write).

M2 — MCP server (the differentiator) → FR-MCP-1/2/3/5/6/7 (T10.4/10.5/10.7/10.8/10.9)

The unique capability (meeting → coding-agent handoff) and a privacy reinforcement: inbound on loopback, zero added egress.

[M2.1] mcp module on rmcp: loopback bind + token gate; Streamable HTTP (/mcp) + stdio adapter; implement McpServer::start/stop (replace todo!()). L → FR-MCP-1/6, NFR-SEC-5
[M2.2] Tools-first surface: list_recent_meetings, get_transcript, get_action_items, get_feature_brief. M → FR-MCP-2
[M2.3] Scope control set_mcp_scope(none|selected|all); recordings never served unless explicitly allowed; enforce in every tool handler. M → FR-MCP-3
[M2.4] Lifecycle: mcp_status/set_mcp_enabled (replace stubs); persist enabled/scope in settings, token in the OS credential store (never settings.json). M → FR-MCP-1
[M2.5] Disclosure UI ("connected agents may forward data") + mcp_access_log rows / mcp://access events on every tool read. M → FR-MCP-5
[M2.6] Privacy panel shows MCP state and confirms it adds no egress; extend privacy_self_check. S → FR-MCP-7, FR-SEC-2
Acceptance: an in-process MCP client gets a schema-valid brief; server binds loopback only (assert a non-loopback bind is refused) and requires a token; the egress test is unchanged with MCP on (merge blocker, FR-MCP-7); recordings not served unless allowed; every read logged.
Depends on: M1 (a served tool). Guardrail: the cross-cutting egress gate must stay green.

M3 — Hosted AI providers (finish 10a) → FR-AI-1/2/3 (T10.1/10.2/10.3)

Commodity but low-effort (OpenAI-compat already exists); gives a cloud-summary choice and pairs with M1 (a brief can be distilled by a hosted model). Adds allowlisted egress by design.

[M3.1] Implement AnthropicProvider (/v1/messages, x-api-key, streaming) behind LlmProvider; is_local() == false. M → FR-AI-1
[M3.2] set_llm_provider: store the API key in the OS credential store via credential_ref (never settings/DB); add the host to the settings-derived egress allowlist. M → FR-AI-2, NFR-SEC-4
[M3.3] Third-party "data leaves your device" banner + one-time acknowledgment; per-use provider selection + active-provider display wherever a summary is generated. S → FR-AI-2/3
Acceptance: summaries route to mocked OpenAI-compat and Anthropic endpoints with correct shapes; key read only from the credential store; host on the allowlist only when configured; banner shown before first hosted use.

M4 — Reliability & breadth (opportunistic, by value)

[M4.1] Chunked/resumable upload (T9.2 refinement) — top reliability item: recordings are now native-quality (~50–100 MB) and put() buffers the whole file in memory; OneDrive/Graph caps a single PUT at 250 MB. Stream from disk in chunks (Nextcloud chunked upload + Graph upload session). M → FR-SYNC-2/5
[M4.2] Multi-language transcription (T8.7): multilingual model option; whisper language param (select/auto); persist per-meeting language; UI localization scaffold. M → FR-TRX-4
[M4.3] Dropbox/Box upload targets (T9.10): implement DropboxTarget/BoxTarget SyncTarget impls (OAuth is already wired). M → FR-SYNC-9
[M4.4] (Defer) MS Graph calendar source (T8.9) behind CalendarSource, consented, metadata-only. L → FR-CAL-6 (Could)

M5 — Push handoff (Layer 3, last & optional) → FR-AGENT-1/2 (Could) (T10.10/10.11)

Only on demand — the MCP handoff (M2) already covers the agent use case (agents pull context).

[M5.1] AgentRunner: spawn claude -p / codex exec / opencode run / copilot headless against a chosen repo from a brief; stream output (run_agent). L → FR-AGENT-1
[M5.2] IssueTracker: create a GitHub issue from a confirmed action item / brief; optional Copilot-cloud assign (create_issue_from_brief). Third-party egress: off/labeled/allowlisted. M → FR-AGENT-2

Privacy checkpoints (gate every milestone): the egress self-test stays green (FR-SEC-1); enabling MCP adds no allowlist host (FR-MCP-7); hosted AI/agent handoff adds only the explicitly configured host; all secrets (AI keys, MCP token, OAuth tokens) live only in the OS credential store (NFR-SEC-4). Plus the standing gates: cargo fmt/clippy -D warnings, Prettier/ESLint/tsc, and a docs update whenever a command/event contract changes (CLAUDE.md).

23 KiB Raw Blame History Unescape Escape