2.6 KiB
ADR-0007 — Local LLM integration: Ollama HTTP (localhost-only)
- Status: Accepted
- Date: 2026-06-30
- Context source: Design doc §"Local LLM Integration (Ollama / Unsloth)"
Context
WA augments notes with summaries, decisions, and action items via a local LLM, configurable by the user, and must never originate cloud calls for meeting content itself.
Decision
Integrate via the Ollama REST API on http://localhost:11434 behind an llm::LlmProvider
trait. Use /api/tags to list installed models, /api/chat (OpenAI-style messages) for
summarization with streaming token output, and /api/pull to assist guided model download.
Provide three provider options in Settings:
- Ollama (default, autodetected),
- Custom OpenAI-compatible endpoint (any local server the user runs — covers Unsloth-served models, llama.cpp server, LM Studio, etc.),
- Disabled (notes-only, no LLM).
WA validates that the configured host resolves to loopback/local; if a user deliberately points at a remote/proxying endpoint, WA shows a clear banner that upstream behavior is outside WA's control (the design doc's Unsloth-proxy caveat).
Consequences
- Positive: Ollama is the de-facto local LLM runtime with a simple, stable API and streaming; the trait + "custom endpoint" keeps us provider-agnostic and honors the design's pluggability; guided install/pull improves first-run UX with hardware-aware model suggestions.
- Negative: depends on the user having Ollama (or another local server) installed → need detection + guided setup and a graceful "LLM unavailable, notes still work" path; summary quality varies by model.
- Prompts are assembled from transcript + meeting metadata + optional template (FR-LLM-2); output is parsed into editable action items (FR-LLM-3) that the user confirms before they become tasks.
Update (2026-06-30) — cloud providers behind the same trait
LlmProvider is extended (ADR-0011, Layer 1) with optional hosted providers for summaries:
Anthropic Messages API (/v1/messages — native, not OpenAI-shaped) and an OpenAI-compatible
client (OpenAI, OpenRouter, LM Studio, gateways). These are off by default, explicitly opt-in,
and labeled as third-party egress; their hosts join the settings-derived egress allowlist
(FR-SEC-1). The is_local() guard still distinguishes local from remote so the UI can warn. Ollama
remains the default and the only zero-egress option.
Revisit if
A clearly better local-LLM runtime/protocol emerges, or we embed an inference engine directly (would remove the external dependency but enlarge the app).