50 lines
2.6 KiB
Markdown
50 lines
2.6 KiB
Markdown
# ADR-0007 — Local LLM integration: Ollama HTTP (localhost-only)
|
|
|
|
- **Status:** Accepted
|
|
- **Date:** 2026-06-30
|
|
- **Context source:** Design doc §"Local LLM Integration (Ollama / Unsloth)"
|
|
|
|
## Context
|
|
|
|
WA augments notes with summaries, decisions, and action items via a **local** LLM, configurable
|
|
by the user, and must never originate cloud calls for meeting content itself.
|
|
|
|
## Decision
|
|
|
|
Integrate via the **Ollama REST API** on `http://localhost:11434` behind an `llm::LlmProvider`
|
|
trait. Use `/api/tags` to list installed models, `/api/chat` (OpenAI-style messages) for
|
|
summarization with **streaming** token output, and `/api/pull` to assist guided model download.
|
|
Provide three provider options in Settings:
|
|
1. **Ollama** (default, autodetected),
|
|
2. **Custom OpenAI-compatible endpoint** (any local server the user runs — covers Unsloth-served
|
|
models, llama.cpp server, LM Studio, etc.),
|
|
3. **Disabled** (notes-only, no LLM).
|
|
|
|
WA validates that the configured host resolves to **loopback/local**; if a user deliberately
|
|
points at a remote/proxying endpoint, WA shows a clear banner that upstream behavior is outside
|
|
WA's control (the design doc's Unsloth-proxy caveat).
|
|
|
|
## Consequences
|
|
|
|
- **Positive:** Ollama is the de-facto local LLM runtime with a simple, stable API and streaming;
|
|
the trait + "custom endpoint" keeps us provider-agnostic and honors the design's pluggability;
|
|
guided install/pull improves first-run UX with hardware-aware model suggestions.
|
|
- **Negative:** depends on the user having Ollama (or another local server) installed → need
|
|
detection + guided setup and a graceful "LLM unavailable, notes still work" path; summary quality
|
|
varies by model.
|
|
- Prompts are assembled from transcript + meeting metadata + optional template (FR-LLM-2); output
|
|
is parsed into editable action items (FR-LLM-3) that the user confirms before they become tasks.
|
|
|
|
## Update (2026-06-30) — cloud providers behind the same trait
|
|
|
|
`LlmProvider` is extended (ADR-0011, Layer 1) with optional **hosted** providers for summaries:
|
|
**Anthropic Messages API** (`/v1/messages` — native, not OpenAI-shaped) and an **OpenAI-compatible**
|
|
client (OpenAI, OpenRouter, LM Studio, gateways). These are **off by default**, explicitly opt-in,
|
|
and labeled as third-party egress; their hosts join the settings-derived egress allowlist
|
|
(FR-SEC-1). The `is_local()` guard still distinguishes local from remote so the UI can warn. Ollama
|
|
remains the default and the only zero-egress option.
|
|
|
|
## Revisit if
|
|
A clearly better local-LLM runtime/protocol emerges, or we embed an inference engine directly
|
|
(would remove the external dependency but enlarge the app).
|