2.0 KiB
ADR-0003 — Transcription engine: whisper-rs (whisper.cpp) with pluggable backends
- Status: Accepted
- Date: 2026-06-30
- Context source: Design doc §"Transcription Engine", §"Transcription Modes"
Context
WA needs accurate, offline, multilingual speech-to-text with both a low-latency streaming mode and a higher-accuracy batch mode, runnable across CPU and several GPU vendors, and (separately) on NPUs.
Decision
Use whisper-rs (Rust bindings to whisper.cpp) as the primary engine, behind a
transcription::Transcriber trait so the concrete engine is swappable. whisper.cpp gives:
- CPU (AVX2/AVX-512) baseline that runs anywhere,
- Vulkan for cross-vendor GPU (AMD/Intel/NVIDIA),
- CUDA for NVIDIA fast path,
- selectable model sizes (tiny→large) for the speed/accuracy and "low-overhead" presets,
- multilingual models for FR-LANG-1.
The NPU path is not served by whisper.cpp (it has no NPU backend); it is a second
Transcriber implementation using ONNX Runtime — see ADR-0004. Both implementations satisfy the
same trait and emit the same TranscriptSegment stream.
Consequences
- Positive: one mature dependency covers CPU + all desktop GPUs; model-size switching maps directly to the performance presets; trait boundary lets the NPU/ONNX engine slot in without touching callers (audio, storage, UI).
- Negative: two engines (whisper.cpp + ONNX) to maintain for full hardware coverage; whisper.cpp diarization is weak, so diarization is a separate component (ADR-0005); building acceleration features on Windows CI needs the right toolchain (CUDA/Vulkan SDKs) — gated by Cargo features so CPU-only always builds.
- Streaming mode: feed fixed audio windows with overlap; emit interim segments, then finalize.
- Batch mode: optional re-run with a larger model after stop for maximum accuracy.
Revisit if
A single engine gains solid NPU + GPU + CPU coverage (would let us drop the ONNX engine), or a materially better local ASR model (e.g. Parakeet) outperforms Whisper for our languages.