42 lines
2.0 KiB
Markdown
42 lines
2.0 KiB
Markdown
# ADR-0003 — Transcription engine: whisper-rs (whisper.cpp) with pluggable backends
|
|
|
|
- **Status:** Accepted
|
|
- **Date:** 2026-06-30
|
|
- **Context source:** Design doc §"Transcription Engine", §"Transcription Modes"
|
|
|
|
## Context
|
|
|
|
WA needs accurate, offline, multilingual speech-to-text with both a low-latency streaming mode
|
|
and a higher-accuracy batch mode, runnable across CPU and several GPU vendors, and (separately)
|
|
on NPUs.
|
|
|
|
## Decision
|
|
|
|
Use **`whisper-rs`** (Rust bindings to **whisper.cpp**) as the primary engine, behind a
|
|
`transcription::Transcriber` trait so the concrete engine is swappable. whisper.cpp gives:
|
|
- CPU (AVX2/AVX-512) baseline that runs anywhere,
|
|
- **Vulkan** for cross-vendor GPU (AMD/Intel/NVIDIA),
|
|
- **CUDA** for NVIDIA fast path,
|
|
- selectable model sizes (tiny→large) for the speed/accuracy and "low-overhead" presets,
|
|
- multilingual models for FR-LANG-1.
|
|
|
|
The **NPU** path is *not* served by whisper.cpp (it has no NPU backend); it is a second
|
|
`Transcriber` implementation using ONNX Runtime — see ADR-0004. Both implementations satisfy the
|
|
same trait and emit the same `TranscriptSegment` stream.
|
|
|
|
## Consequences
|
|
|
|
- **Positive:** one mature dependency covers CPU + all desktop GPUs; model-size switching maps
|
|
directly to the performance presets; trait boundary lets the NPU/ONNX engine slot in without
|
|
touching callers (audio, storage, UI).
|
|
- **Negative:** two engines (whisper.cpp + ONNX) to maintain for full hardware coverage;
|
|
whisper.cpp diarization is weak, so diarization is a separate component (ADR-0005); building
|
|
acceleration features on Windows CI needs the right toolchain (CUDA/Vulkan SDKs) — gated by
|
|
Cargo features so CPU-only always builds.
|
|
- **Streaming mode:** feed fixed audio windows with overlap; emit interim segments, then finalize.
|
|
- **Batch mode:** optional re-run with a larger model after stop for maximum accuracy.
|
|
|
|
## Revisit if
|
|
A single engine gains solid NPU + GPU + CPU coverage (would let us drop the ONNX engine), or a
|
|
materially better local ASR model (e.g. Parakeet) outperforms Whisper for our languages.
|