WhispAssist/docs/adr/0004-hardware-acceleration.md

ADR-0004 — Hardware acceleration strategy & detection ladder

• Status: Accepted
• Date: 2026-06-30
• Context source: Design doc §"Hardware Detection and Transcription Strategy"

Context

The design mandates a strict preference order NPU → NVIDIA → AMD → Intel → CPU, detected at startup and on demand, with graceful fallback and the chosen backend surfaced to the user.

Decision

Introduce a hardware module that performs capability detection and returns a ranked list of available Backends; the transcription layer selects the highest-ranked one and constructs the matching Transcriber.

Detection mapping:

Tier	Detection	Execution path
NPU	ONNX Runtime / Windows ML enumeration; DirectML device_filter="npu"	ONNX Transcriber (Whisper-ONNX) via ort + DirectML
NVIDIA GPU	adapter enumeration (DXGI) + CUDA availability	whisper.cpp CUDA
AMD GPU	DXGI adapter; Vulkan device present	whisper.cpp Vulkan (DirectML alt.)
Intel GPU	DXGI adapter; Vulkan device present	whisper.cpp Vulkan
CPU	always	whisper.cpp CPU (AVX2/AVX-512 if present)

• ONNX Runtime execution providers also fall back internally (NPU→GPU→CPU) per-operator, giving a second safety net on the NPU path.
• The selected backend, model size, and an estimated real-time factor are reported via the hardware_status command and shown in Settings + the recording bar (FR-HW-3).
• A user override lets advanced users pin a specific backend (FR-HW-2); "Low overhead" preset forces CPU + small model + no real-time summarization (NFR-RES-3).

Consequences

• Positive: one abstraction satisfies the full ladder; per-operator EP fallback makes the NPU path robust; clean place to add future vendors/runtimes (Windows ML is abstracted behind Backend, so a DirectML→Windows ML migration is local to this module).
• Negative: NPU support is hardware/driver dependent and the newest, least-tested tier — must degrade silently to GPU/CPU and never hard-fail; requires pinning ort↔ONNX Runtime binary versions on Windows builds.

Revisit if

Windows ML supersedes DirectML for our models, or a vendor SDK gives materially better NPU throughput than the ONNX/DirectML path.