205 lines
12 KiB
Markdown
205 lines
12 KiB
Markdown
# CLAUDE.md — Working agreement for AI builders
|
|
|
|
This file orients an AI coding agent (or human) working in this repository. Read it before
|
|
making changes.
|
|
|
|
<!-- MEMANTO-MANAGED-SECTION -->
|
|
## MEMANTO - Your Active Memory Companion
|
|
|
|
**MEMANTO is not a passive store. It is an active companion agent that works alongside you.**
|
|
Don't treat MEMANTO like a static blob you query once and forget. It's a teammate you keep
|
|
talking to, every preference, decision, and correction flows through it. MEMANTO remembers,
|
|
recalls, and answers so you hold context across sessions, honor prior decisions, and avoid
|
|
repeating mistakes the user already corrected.
|
|
|
|
Every memory operation in this session goes through MEMANTO. There is no exception.
|
|
|
|
> **CRITICAL**: All `memanto` commands are **shell commands**. Always run them using the Bash tool.
|
|
> Never simulate, describe, or "pretend to call" them. If you cannot run the shell, say so explicitly instead of inventing memory state.
|
|
|
|
### NON-NEGOTIABLE RULES
|
|
|
|
These are not suggestions. Follow each one on every turn.
|
|
|
|
0. **Activate the `whispassist` agent at the start of every session, before any memory op.** Run
|
|
`memanto agent activate whispassist` first thing. This machine hosts multiple projects and the
|
|
session-start sync may activate a *different* project's agent (e.g. `whispassist`), so the
|
|
auto-synced `MEMORY.md` can belong to the wrong project — do not trust it as LastERP context
|
|
until you've activated `whispassist` and re-synced. Confirm with `memanto agent list` (the
|
|
active one is marked). All `recall`/`remember`/`answer` calls read and write the *active*
|
|
agent's store, so getting this wrong silently pollutes or mis-reads another project's memory.
|
|
1. **Read `MEMORY.md` before doing anything.** It is auto-synced at session start and holds
|
|
the user's preferences, facts, goals, instructions, decisions, and commitments from every
|
|
prior session. You MUST honor what is written there. If you act against it, you are
|
|
breaking continuity the user is paying for.
|
|
2. **Search memory before saying you don't know.** If the user asks about past context, an
|
|
earlier decision, a preference, or anything you are unsure about, you MUST run `recall`
|
|
or `answer` first. Saying "I don't have context" without searching is a failure.
|
|
3. **Store proactively. Do not wait to be asked.** The moment a memory-worthy event happens
|
|
— a preference stated, a decision made, a fact learned, an instruction given, a goal set,
|
|
a mistake corrected — run `memanto remember` immediately, in the same turn.
|
|
4. **Always pass full metadata to `remember`.** Every `memanto remember` call MUST include
|
|
`--type`, `--confidence`, `--provenance`, and `--source <your_agent_name>`. Never let
|
|
these default. Untyped, unsourced memories pollute the agent's recall quality.
|
|
5. **One memory operation goes through MEMANTO. All of them do.** Do not keep mental notes,
|
|
in-context scratch pads, or "I'll remember this for next time" promises. If it matters
|
|
beyond this turn, it goes into MEMANTO. If it doesn't, drop it.
|
|
|
|
### Memory Operations — Use the Right One
|
|
|
|
MEMANTO gives you three primitives. They are equal-priority. Pick by intent, not by habit.
|
|
|
|
| You want to... | Use | Why |
|
|
|---|---|---|
|
|
| Read raw memory chunks and apply them as context | `memanto recall "query"` | Best for context-building, multi-step work, comparing options |
|
|
| Get one synthesized, grounded answer to a direct question | `memanto answer "question"` | Best for "what did we decide / prefer / commit to?" — saves you reading and merging |
|
|
| Persist something memory-worthy | `memanto remember "content" --type ... --confidence ... --provenance ... --source ...` | Every preference, decision, fact, instruction, goal, lesson |
|
|
| See what changed since last time | `memanto recall --changed-since "last 7 days"` | Catching up after a break |
|
|
| See the most recent memories | `memanto recall --recent` | Fast context refresh |
|
|
|
|
Do NOT always default to `recall`. If the user asked a direct question, `answer` is usually
|
|
the right tool — it returns a grounded synthesis so you don't burn tokens re-reading raw
|
|
chunks.
|
|
|
|
### When to Call `remember` (Examples — Run Immediately)
|
|
|
|
- User says *"I prefer tabs over spaces"*:
|
|
`memanto remember "User prefers tabs over spaces for indentation" --type preference --confidence 1.0 --provenance explicit_statement --source <your_agent_name>`
|
|
- You decide to use Library X for reason Y:
|
|
`memanto remember "Chose Library X for reason Y; commit abc123" --type decision --confidence 0.95 --provenance inferred --source <your_agent_name>`
|
|
- User corrects an approach:
|
|
`memanto remember "User corrected: use pytest, not unittest" --type learning --confidence 1.0 --provenance corrected --source <your_agent_name>`
|
|
- A failed approach taught you something:
|
|
`memanto remember "Batch size > 100 fails with TimeoutError" --type error --confidence 0.95 --provenance observed --source <your_agent_name>`
|
|
|
|
### Command Reference
|
|
|
|
```bash
|
|
# Store — ALWAYS pass full metadata
|
|
memanto remember "content" --type <type> --confidence <0.0-1.0> --provenance <provenance> --source <agent_name>
|
|
|
|
# Recall raw context
|
|
memanto recall "query" # semantic search
|
|
memanto recall "query" --type <type> --limit 10 # filtered search
|
|
memanto recall --recent --limit 10 # newest first, no query
|
|
memanto recall --as-of "2026-01-15" # state at a point in time
|
|
memanto recall --changed-since "last 7 days" # what changed since
|
|
|
|
# Synthesized answer (grounded RAG over memories)
|
|
memanto answer "question"
|
|
|
|
# Re-sync MEMORY.md (project-local cache)
|
|
memanto memory sync --project-dir .
|
|
```
|
|
|
|
**Memory types** (use the closest fit, do not invent new ones):
|
|
`fact`, `preference`, `instruction`, `decision`, `event`, `goal`, `commitment`,
|
|
`observation`, `learning`, `relationship`, `context`, `artifact`, `error`.
|
|
|
|
**Provenance values**: `explicit_statement`, `inferred`, `observed`, `corrected`,
|
|
`validated`, `imported`.
|
|
|
|
**Confidence**: `1.0` for explicit user statements; `0.9-0.95` for strong consensus;
|
|
`0.8-0.85` for observed patterns (3+ times); `0.6-0.75` for emerging patterns.
|
|
|
|
> **Note**: The `memanto-memory` skill contains reference guidelines only (best practices, confidence levels, tagging). It is NOT executable — always use Bash for memanto commands.
|
|
<!-- /MEMANTO-MANAGED-SECTION -->
|
|
|
|
## What this project is
|
|
|
|
WhispAssist (WA) is a **fully local, open-source, Windows-native** meeting assistant. The
|
|
non-negotiable product invariant: **no audio or transcript leaves the device except to
|
|
destinations the user has explicitly configured** — namely a *local* LLM endpoint (ADR-0007) and
|
|
enabled **sync targets** (ADR-0010). Both are **off by default**. With neither configured, WA
|
|
makes no content egress at all. Treat any outbound connection for meeting content that is *not*
|
|
to a user-configured, allowlisted destination as a design violation requiring sign-off.
|
|
|
|
Two related product rules:
|
|
- **Recording is opt-in (ADR-0009).** Audio is retained as `.wav` only when the user turns on
|
|
"Record this meeting" (default off); otherwise working audio is deleted on finalize. A one-time
|
|
consent notice (recording may be unlawful without consent) must be acknowledged first.
|
|
- **Sync is explicit (ADR-0010).** Credentials live only in the OS credential store (never in
|
|
`settings.json`/`wa.db`); TLS is required; third-party targets are clearly labeled. The egress
|
|
allowlist is derived from settings and enforced in the Rust `sync`/`llm` services.
|
|
|
|
External-AI rule (ADR-0011): hosted summary providers (Anthropic/OpenAI-compatible) are optional,
|
|
off by default, third-party egress (keys in the OS credential store). WA can also expose a **local
|
|
MCP server** so the user's own coding agents (Claude/Codex/Copilot/OpenCode) pull meeting context —
|
|
this is **inbound on loopback, off by default, token-gated, scope-limited, audited, and adds no
|
|
egress**. The data only leaves the device via the connected agent's own provider, which WA discloses.
|
|
|
|
## Source of truth
|
|
|
|
The plan in `docs/` is authoritative. When in doubt:
|
|
|
|
1. Requirements & acceptance criteria → `docs/01-requirements.md`
|
|
2. Component boundaries & data flow → `docs/02-architecture.md`
|
|
3. DB schema, file paths, JSON shapes → `docs/03-data-model.md`
|
|
4. Function signatures & events → `docs/04-api-contracts.md`
|
|
5. What to build next & in what order → `docs/05-roadmap.md`
|
|
6. How to prove it works → `docs/06-test-strategy.md`
|
|
7. Why the stack is what it is → `docs/adr/`
|
|
|
|
If code and docs disagree, the docs win unless you update them in the same change and note why.
|
|
|
|
## Stack (decided — see ADRs before changing)
|
|
|
|
- **App shell:** Tauri 2 (Rust core + WebView2). ADR-0001.
|
|
- **Frontend:** Svelte + TypeScript, Vite. ADR-0002.
|
|
- **Transcription:** `whisper-rs` primary; `ort` (ONNX Runtime + DirectML) for NPU. ADR-0003/0004.
|
|
- **Diarization:** `sherpa-onnx`. ADR-0005.
|
|
- **Storage:** SQLite + files under `%LOCALAPPDATA%\WhispAssist`. ADR-0006.
|
|
- **LLM:** Ollama HTTP, localhost only. ADR-0007.
|
|
- **Calendar/PST:** `outlook-pst` crate. ADR-0008.
|
|
- **Recording:** opt-in retention + consent. ADR-0009.
|
|
- **Sync:** WebDAV (Nextcloud/ownCloud/Cloudreve/Seafile/Synology) primary; OneDrive/Dropbox/Box via OAuth. ADR-0010.
|
|
- **External AI / agents:** hosted LLM providers behind `LlmProvider`; WA-as-MCP-server (`rmcp`) for agent handoff; feature briefs. ADR-0011.
|
|
|
|
## Architectural rules
|
|
|
|
- The Rust core is split into independent service modules (`audio`, `transcription`,
|
|
`diarization`, `storage`, `llm`, `calendar`, `hardware`, `notes`, `sync`, `mcp`). Each exposes a
|
|
trait in its `mod.rs`. Depend on the **trait**, not the concrete type, so engines/providers can be
|
|
swapped (one `WebDavTarget` covers all primary sync targets; one `McpServer` serves all agents).
|
|
- The frontend never touches the filesystem, the database, or the network directly. It calls
|
|
Tauri **commands** and subscribes to Tauri **events** (see `docs/04-api-contracts.md`).
|
|
- **Audio is the source of truth.** Persist raw audio first; transcripts/notes are derived and
|
|
must be regenerable after a crash. Never delete audio as a side effect of a transcript op.
|
|
- Long-running work (capture, inference) runs off the UI/IPC thread. Stream partial results via
|
|
events; never block a command for the duration of a meeting.
|
|
- Hardware acceleration is a runtime decision, not a compile-time one where avoidable. Always
|
|
degrade gracefully NPU → NVIDIA → AMD → Intel → CPU and surface the chosen backend in the UI.
|
|
|
|
## Conventions
|
|
|
|
- Rust: `cargo fmt` + `cargo clippy -- -D warnings` must pass. Public items get doc comments.
|
|
- TS/Svelte: Prettier + ESLint. Strict TypeScript.
|
|
- Errors: Rust uses `thiserror` per-module error enums surfaced as typed command errors; no
|
|
`unwrap()`/`expect()` on paths reachable from user input or hardware state.
|
|
- Commits: conventional commits (`feat:`, `fix:`, `docs:`, `test:`, `chore:`). Reference the
|
|
requirement or task ID where relevant (e.g. `feat(audio): WASAPI loopback capture (FR-CAP-1)`).
|
|
- Tests live next to the code (`#[cfg(test)]`) for units; cross-service tests in `/tests`.
|
|
|
|
## Definition of done (per task)
|
|
|
|
A task is done when: code compiles with no clippy warnings; the relevant requirement's
|
|
acceptance criteria are demonstrably met; tests named in `docs/06-test-strategy.md` for that
|
|
phase pass; and docs are updated if behavior or contracts changed.
|
|
|
|
## Guardrails
|
|
|
|
- Do not add telemetry, analytics, crash reporting, or auto-update phone-home without an ADR
|
|
and an explicit opt-in. Default to off and local.
|
|
- Never persist sync credentials or OAuth tokens to `settings.json`, `wa.db`, logs, or events —
|
|
only the OS credential store. Never return secrets from commands like `list_sync_targets`.
|
|
- Recording defaults OFF; sync defaults OFF; hosted AI providers default OFF; MCP server defaults
|
|
OFF. Any new egress path must be added to the settings-derived allowlist and covered by the
|
|
privacy egress test before merge.
|
|
- The MCP server is inbound/loopback-only and token-gated; it must never bind a non-loopback
|
|
address, never serve recordings unless explicitly allowed, and never add a host to the egress
|
|
allowlist. Every agent read is logged.
|
|
- Do not require admin privileges for normal operation.
|
|
- Do not add WA to OS startup without explicit user consent (NFR-RES-4).
|
|
- Keep idle resource use near zero (NFR-RES-1): no busy loops, no polling timers running when
|
|
not recording.
|