# ADR-0004 — Hardware acceleration strategy & detection ladder

- **Status:** Accepted
- **Date:** 2026-06-30
- **Context source:** Design doc §"Hardware Detection and Transcription Strategy"

## Context

The design mandates a strict preference order **NPU → NVIDIA → AMD → Intel → CPU**, detected at
startup and on demand, with graceful fallback and the chosen backend surfaced to the user.

## Decision

Introduce a `hardware` module that performs **capability detection** and returns a ranked list of
available `Backend`s; the transcription layer selects the highest-ranked one and constructs the
matching `Transcriber`.

Detection mapping:

| Tier | Detection | Execution path |
|---|---|---|
| NPU | ONNX Runtime / Windows ML enumeration; DirectML `device_filter="npu"` | ONNX `Transcriber` (Whisper-ONNX) via `ort` + DirectML |
| NVIDIA GPU | adapter enumeration (DXGI) + CUDA availability | whisper.cpp **CUDA** |
| AMD GPU | DXGI adapter; Vulkan device present | whisper.cpp **Vulkan** (DirectML alt.) |
| Intel GPU | DXGI adapter; Vulkan device present | whisper.cpp **Vulkan** |
| CPU | always | whisper.cpp **CPU** (AVX2/AVX-512 if present) |

- ONNX Runtime execution providers also fall back internally (NPU→GPU→CPU) per-operator, giving a
  second safety net on the NPU path.
- The selected backend, model size, and an estimated real-time factor are reported via the
  `hardware_status` command and shown in Settings + the recording bar (FR-HW-3).
- A user override lets advanced users pin a specific backend (FR-HW-2); "Low overhead" preset
  forces CPU + small model + no real-time summarization (NFR-RES-3).

## Consequences

- **Positive:** one abstraction satisfies the full ladder; per-operator EP fallback makes the
  NPU path robust; clean place to add future vendors/runtimes (Windows ML is abstracted behind
  `Backend`, so a DirectML→Windows ML migration is local to this module).
- **Negative:** NPU support is hardware/driver dependent and the newest, least-tested tier —
  must degrade silently to GPU/CPU and never hard-fail; requires pinning `ort`↔ONNX Runtime
  binary versions on Windows builds.

## Revisit if
Windows ML supersedes DirectML for our models, or a vendor SDK gives materially better NPU
throughput than the ONNX/DirectML path.

## Update (2026-07-16): NPU path uses OpenVINO; decoder split off the CPU

As built (T3.4 + follow-up), the NPU tier runs on **ONNX Runtime + the OpenVINO EP**
(`device_type=NPU`), not DirectML — Intel's prebuilt OpenVINO ORT runtime is downloaded on
demand (DirectML remains the *GPU* fallback EP for AMD/Intel in non-Vulkan builds). The engine
splits Whisper across devices:

- **encoder** (fixed shape): OpenVINO **NPU**;
- **decoder** (autoregressive, KV-cache greedy via Optimum's merged export): OpenVINO **GPU**
  (Intel iGPU, same runtime bundle) when available, else the CPU EP —
  `WA_ONNX_DECODER_DEVICE=cpu|gpu` overrides.

Rationale: the dynamic-shaped decode loop is a poor NPU fit, and pre-KV-cache it re-ran the full
token prefix every step on the CPU — the dominant CPU cost while transcribing. With the merged
decoder + iGPU placement, sustained CPU during transcription is near-idle on NPU+iGPU systems.
The engine logs both EPs at load (`transcription engine: ONNX …`).