Add all scaffolding, design, and decision documentation AKA first commit
This commit is contained in:
iamdoubz
@@ -0,0 +1,111 @@
|
||||
# 07 — Research Findings (Stack Validation)
|
||||
|
||||
This document records the technical feasibility checks performed before committing to a stack.
|
||||
Each finding includes the conclusion that fed into the ADRs. Dates reflect research done
|
||||
**June 2026**; re-verify versions at implementation time.
|
||||
|
||||
## Summary
|
||||
|
||||
Every load-bearing assumption in the design document is supported by a mature, offline-capable,
|
||||
permissively licensed component. No blocker was found. The one area needing care is the **NPU
|
||||
path** (newer, vendor-dependent) and **`.pst` parsing** (use a battle-tested library, treat as
|
||||
read-only). Both have viable primary + fallback options.
|
||||
|
||||
## Findings
|
||||
|
||||
### App shell — Tauri 2 ✅
|
||||
Tauri 2.0 reached stable in **October 2024** and is the current major line. On Windows it
|
||||
renders through **WebView2** (the OS Chromium-based webview) rather than bundling a browser,
|
||||
which is the key reason its binaries and memory footprint are far smaller than Electron's —
|
||||
directly serving WA's "fast, low memory" goal. Cross-platform (Windows/macOS/Linux) leaves the
|
||||
door open for the design doc's "future" non-Windows support without a rewrite.
|
||||
→ **ADR-0001.**
|
||||
|
||||
### Frontend — Svelte ✅
|
||||
Svelte compiles components to small imperative JS with no runtime framework shipped, giving the
|
||||
smallest bundle and lowest per-view memory among mainstream options — consistent with the
|
||||
low-overhead NFRs. Pairs cleanly with Tauri via Vite. → **ADR-0002.**
|
||||
|
||||
### Audio capture — WASAPI loopback in Rust ✅
|
||||
The `wasapi` crate provides safe Rust access to WASAPI including a **loopback** example
|
||||
(capture system output without a meeting bot). `cpal` is the cross-platform fallback. Known
|
||||
constraint: `AUDCLNT_STREAMFLAGS_LOOPBACK` and `AUDCLNT_STREAMFLAGS_EVENTCALLBACK` cannot be
|
||||
combined, so the capture loop must **poll** rather than rely on event callbacks. This shapes
|
||||
the audio thread design in `docs/02-architecture.md`. → **ADR-0001 / architecture.**
|
||||
|
||||
### Transcription — whisper-rs (+ ONNX for NPU) ✅
|
||||
`whisper-rs` wraps **whisper.cpp** with feature-gated acceleration: **CUDA** (NVIDIA),
|
||||
**Vulkan** (cross-vendor GPU incl. AMD/Intel), **ROCm/hipBLAS**, **Metal**, **OpenBLAS**, and
|
||||
plain CPU. whisper.cpp has **no native NPU backend**, so the NPU tier is served separately via
|
||||
**ONNX Runtime** (see below) running a Whisper-ONNX model. This split — whisper.cpp for
|
||||
CPU/GPU, ONNX for NPU — is the basis of the acceleration ladder. → **ADR-0003.**
|
||||
|
||||
### Hardware acceleration — ONNX Runtime `ort` + DirectML/Windows ML ✅
|
||||
The Rust `ort` crate exposes ONNX Runtime execution providers including **DirectML**, which can
|
||||
target an **NPU** (`device_filter = "npu"`) on Windows. Execution providers fall back in order
|
||||
(NPU → GPU → CPU) automatically when an operator is unsupported. **Windows ML** is now GA as the
|
||||
production on-device runtime on Windows 11. DirectX 12 is required for DirectML (ubiquitous on
|
||||
modern hardware). Note: Microsoft is steering new Windows ONNX work toward Windows ML over raw
|
||||
DirectML, so the hardware layer is abstracted (`hardware::Backend`) to allow swapping the
|
||||
provider without touching callers. → **ADR-0004.**
|
||||
|
||||
### Diarization — sherpa-onnx ✅
|
||||
`sherpa-onnx` (k2-fsa) provides **offline speaker diarization**: pyannote segmentation model +
|
||||
a speaker-embedding extractor (e.g. 3D-Speaker ERes2Net) + clustering, all as ONNX, fully
|
||||
offline. It ships C/C++ APIs callable from Rust via FFI. This satisfies "Speaker 1/2 …" labeling
|
||||
and post-meeting merge/rename without any cloud dependency. → **ADR-0005.**
|
||||
|
||||
### Storage — SQLite ✅
|
||||
SQLite (via `sqlx` for async or `rusqlite` for sync) is the standard embedded store, matches
|
||||
Meetily's proven model, requires no server, and supports full-text search (FTS5) for the
|
||||
transcript search requirement. → **ADR-0006.**
|
||||
|
||||
### Local LLM — Ollama HTTP ✅
|
||||
Ollama serves a REST API on **`http://localhost:11434`** with `/api/chat` (OpenAI-style
|
||||
messages), `/api/generate` (one-shot), `/api/tags` (list models), `/api/pull` (download), and
|
||||
token **streaming** via newline-delimited JSON. WA calls only this local endpoint; a "custom
|
||||
endpoint" option lets advanced users point at any OpenAI-compatible local server. → **ADR-0007.**
|
||||
|
||||
### Outlook `.pst` — outlook-pst / libpff ✅
|
||||
The Rust `outlook-pst` crate offers **read-only** access to PST files modeled on the published
|
||||
MS-PST specification (emails, folders, and — relevant here — calendar appointments/attendees).
|
||||
The mature C library **libpff** (with `pffexport`) is the fallback for tricky/encrypted files
|
||||
via FFI or a sidecar. Treat PST strictly as read-only input. → **ADR-0008.**
|
||||
|
||||
### External AI / MCP — all targets are MCP clients; Rust SDK exists ✅
|
||||
- **Claude** (Code & Desktop): MCP client over stdio + HTTP; Claude Code also runs **headless**
|
||||
(`claude -p`, `--bare`) and ships the **Claude Agent SDK** (Python/TS) — usable for the optional
|
||||
push path. Claude Code can itself act as an MCP server.
|
||||
- **OpenAI Codex** (CLI): MCP via **stdio** (local child process) **and remote Streamable HTTP**
|
||||
(OAuth/bearer); managed with `codex mcp`; `codex exec` for headless. Has an Agents SDK.
|
||||
- **GitHub Copilot:** MCP across **IDE, CLI, and the cloud agent**. Caveat: the cloud agent supports
|
||||
MCP **tools only** (no resources/prompts) and **no OAuth-remote** MCP, and runs in GitHub's cloud
|
||||
(can't reach a localhost server) — so WA exposes capabilities as **tools**, and Copilot *cloud* is
|
||||
better served by the GitHub-issue handoff.
|
||||
- **OpenCode:** full MCP client (local + remote), 75+ LLM providers, has an SDK.
|
||||
- **Transports:** MCP defines exactly **stdio** and **Streamable HTTP** (HTTP+SSE deprecated). For a
|
||||
long-running desktop app holding shared state, a **loopback Streamable HTTP** `/mcp` endpoint fits;
|
||||
a thin **stdio** adapter covers agents that spawn their server.
|
||||
- **Rust SDK:** the official **`rmcp`** crate (`modelcontextprotocol/rust-sdk`, v0.16+, `server`
|
||||
feature) implements tools/resources/prompts with pluggable transports — so WA's Rust core can host
|
||||
the server natively and lightweightly.
|
||||
- **Summary APIs:** **Anthropic** is native `/v1/messages` (needs its own adapter); **OpenAI** is
|
||||
`/v1/chat/completions`; most others are OpenAI-compatible. → **ADR-0011** (and ADR-0007 update).
|
||||
|
||||
## Open items to confirm at build time
|
||||
- Exact `whisper-rs` version and which acceleration features build cleanly on the Windows CI image.
|
||||
- `ort` ↔ ONNX Runtime binary version pinning for the DirectML/NPU path on target Windows builds.
|
||||
- `outlook-pst` coverage of password-protected and very large PST files; fall back to libpff if gaps.
|
||||
- Whether to consume `sherpa-onnx` as a prebuilt C library or build from source in CI.
|
||||
|
||||
## Sources
|
||||
- [Tauri 2.0 Stable Release](https://v2.tauri.app/blog/tauri-20/) · [Tauri vs Electron 2026](https://tech-insider.org/tauri-vs-electron-2026/)
|
||||
- [whisper-rs (crates.io)](https://crates.io/crates/whisper-rs) · [whisper-rs repo](https://github.com/tazz4843/whisper-rs)
|
||||
- [ort execution providers](https://ort.pyke.io/perf/execution-providers) · [ONNX Runtime DirectML EP](https://onnxruntime.ai/docs/execution-providers/DirectML-ExecutionProvider.html) · [Windows ML execution providers](https://learn.microsoft.com/en-us/windows/ai/new-windows-ml/supported-execution-providers)
|
||||
- [sherpa-onnx speaker diarization](https://k2-fsa.github.io/sherpa/onnx/speaker-diarization/index.html)
|
||||
- [wasapi (crates.io)](https://crates.io/crates/wasapi) · [cpal](https://github.com/RustAudio/cpal)
|
||||
- [Ollama API docs](https://github.com/ollama/ollama/blob/main/docs/api.md)
|
||||
- [outlook-pst (docs.rs)](https://docs.rs/outlook-pst/latest/outlook_pst/) · [libpff](https://github.com/libyal/libpff)
|
||||
- MCP clients: [Copilot MCP + coding agent](https://docs.github.com/en/copilot/concepts/agents/coding-agent/mcp-and-coding-agent) · [Codex MCP](https://developers.openai.com/codex/mcp) · [OpenCode MCP/config](https://opencode.ai/docs/config/) · [Claude Code headless](https://code.claude.com/docs/en/headless)
|
||||
- MCP protocol: [Transports spec](https://modelcontextprotocol.io/specification/2025-03-26/basic/transports) · [Rust SDK `rmcp`](https://github.com/modelcontextprotocol/rust-sdk) ([docs.rs](https://docs.rs/rmcp))
|
||||
- Summary APIs: [Anthropic Messages vs OpenAI Chat Completions](https://portkey.ai/blog/open-ai-responses-api-vs-chat-completions-vs-anthropic-anthropic-messages-api/)
|
||||
Reference in New Issue
Block a user