WhispAssist/docs/adr/0011-external-ai-and-agent-integration.md

ADR-0011 — External AI providers & coding-agent integration

• Status: Accepted
• Date: 2026-06-30
• Context source: User request (2026-06-30); research in 07-research-findings.md (MCP section)

Context

WhispAssist already summarizes meetings with a local LLM (Ollama, ADR-0007). The user wants to also tie into external AI tools — Claude, OpenAI Codex, GitHub Copilot, OpenCode, and eventually others — for two purposes:

1. Summaries / follow-ups (what the local LLM does today), optionally using a hosted model.
2. Turn a meeting into actionable dev work — e.g. a customer asks for a feature on a call and a developer can "get started right away" (scaffold, branch, PR) from that request.

These are two different jobs needing two different mechanisms. Codex, Copilot, and OpenCode are coding agents, not summarization endpoints — calling them for a summary is the wrong shape.

Research finding that drives the decision: all four targets are MCP clients today (Claude Code/Desktop, Codex CLI, Copilot across IDE/CLI/cloud, OpenCode), and an official Rust MCP SDK (rmcp) exists. So a single MCP server in WA's Rust core is consumable by all of them — and by future tools — without per-tool adapters.

Decision — a layered model, MCP-server-first for handoff

Layer 1 — Cloud summary providers (extends ADR-0007)

Add hosted providers behind the existing llm::LlmProvider trait: Anthropic Messages API and an OpenAI-compatible client (covers OpenAI, OpenRouter, LM Studio, gateways). This is "use Claude/ GPT to write summaries." Off by default; third-party "data leaves WA" banner; same provider model as Ollama. (Note: Anthropic's native endpoint is /v1/messages, not OpenAI's /v1/chat/completions, so it needs its own adapter.)

Layer 2 — WhispAssist is an MCP server (the primary handoff; PULL model)

WA hosts a local MCP server, off by default, exposing meeting context and a small set of tools (tools-first, because Copilot's cloud agent supports MCP tools but not resources/prompts):

• list_recent_meetings, get_transcript, get_action_items,
• get_feature_brief(meeting_id) / create_feature_brief(...) — the bridge primitive below.

The developer drives from their own agent ("Claude, grab the feature request from this morning's call and scaffold it"); the agent connects to WA as an MCP client and pulls structured context. WA stays agent-agnostic — implement once, works with Claude/Codex/Copilot/OpenCode and "eventually others."

• Transports: local stdio (agent spawns a thin WA MCP adapter) and Streamable HTTP on loopback (WA, already long-running and holding the DB, hosts http://127.0.0.1:<port>/mcp). HTTP+SSE is deprecated; do not use it.
• Auth/scope: loopback-bound; token required; explicit per-server enable; scope control over which meetings/artifacts are exposed; recordings excluded unless explicitly allowed.
• Audit: log what an agent read.

The bridge primitive — "feature brief"

WA distills a feature request from a transcript (via the configured LLM) into a structured, agent-ready spec: problem, desired outcome, acceptance criteria, target repo/context, source meeting. It is the object that turns "we talked about X" into "here's a branch." Exposed as the get_feature_brief MCP tool and reused by the push/issue paths (Layer 3).

Layer 3 — Push & task-tracker handoff (documented now, built later)

• AgentRunner trait: optionally spawn a local coding-agent CLI headless (claude -p, codex exec, opencode run, copilot) against a chosen repo to produce a branch/PR — a one-click "scaffold this" button in WA.
• Task-tracker handoff: create a GitHub issue from a confirmed action item; optionally assign Copilot's cloud agent, which opens a PR. This is the right path for Copilot cloud (it runs in GitHub's cloud and cannot reach a localhost MCP server).

Privacy reconciliation (critical)

• The MCP server adds no WA egress. It listens on loopback; WA opens no new outbound socket, so the egress allowlist and the CI network test are unaffected. The data only leaves the device when the connected agent sends it to its provider's cloud — outside WA's control. WA must disclose this ("WhispAssist serves this locally; the agent you connected may send it to its provider").
• Cloud summary providers (Layer 1) and task-tracker handoff (Layer 3) are real third-party egress — off by default, explicit opt-in, clearly labeled, and added to the settings-derived allowlist exactly like remote sync (ADR-0010) and remote LLM (ADR-0007).
• Everything here is off by default. With nothing configured, WA remains fully local.

Consequences

• Positive: one MCP server covers all four agents + future ones (the "eventually others" is free); Rust rmcp keeps it lightweight; the local-only identity holds because the server is loopback and egress stays opt-in; the feature-brief primitive is a genuinely WA-specific value-add.
• Negative / care: tools-first design required for Copilot compatibility; MCP server is an inbound surface, so loopback-binding + token + scope + audit are mandatory; Anthropic needs a bespoke adapter; the Layer-3 push/issue paths each need per-agent/per-tracker work, hence deferred.

Revisit if

A meeting-to-code standard emerges beyond MCP, or Copilot cloud gains localhost/OAuth-remote MCP (would let the pull model serve it directly without the issue handoff).