Feature npu #10

2026-07-02T19:59:13-05:00

iamdoubz commented

2026-07-02 19:59:13 -05:00

No description provided.

iamdoubz added 16 commits 2026-07-02 19:59:13 -05:00

fix(transcription): reduce whisper.cpp encoder cost per streaming window 8530a22899

whisper.cpp's encoder always runs over a full, padded 30s mel window
(1500 encoder positions) unless audio_ctx is explicitly reduced. The
~4s live-transcription streaming windows were never setting it, so
every window paid the full 30s-equivalent encode cost on one serial
worker thread -- the actual cause of the reported 30-65s lag between
speaking and a segment appearing, not "CPU is just slow".

Scales audio_ctx proportionally to the window's real sample count
instead (audio_ctx_for_window), floored and capped to whisper.cpp's
own valid range. Verified with a controlled A/B against the real
base.en-q5_1 model: ~30% faster even under heavy concurrent CPU
contention on this machine, which is masking most of the improvement
--the fix targets the encoder's O(n^2)-ish attention cost specifically
(1500 -> ~201 positions for a 4s window), so the gain on an otherwise-
idle machine should be substantially larger.

feat(hardware): on-demand NPU runtime paths + readiness probe (T3.4) d5f7b1ac95

build(hardware): enable SetupAPI feature for NPU detection (T3.4) 0ffcd704b3

feat(hardware): detect Intel NPU via ComputeAccelerator class (T3.4, FR-HW-1) b97e5ebe85

build(transcription): add npu feature (ort + OpenVINO EP, rustfft) (T3.4) d7b98b06cd

feat(transcription): Whisper log-mel front-end for NPU path (T3.4) e875c5de9a

feat(transcription): Whisper ONNX model catalog + downloader (T3.4) 8fe6b5ceb1

feat(transcription): OnnxNpuTranscriber — NPU encoder + CPU greedy decode (T3.4) f8731adae3

feat(transcription): wire OnnxNpuTranscriber behind npu feature (T3.4) f58c0f2d6c

feat(transcription): expose onnx model_dir for the dispatcher (T3.4) a9335f487d

refactor(transcription): let streaming worker take &dyn Transcriber (T3.4) 92752698c5

feat(transcription): dispatch NPU backend + NPU package download command (T3.4) 69ed6256db

feat(hardware): register NPU package command + background model fetch (T3.4) 2b435f3629