Feature chore bug 003 #17
@@ -41,3 +41,8 @@ Thumbs.db
|
||||
# NPU runtime bundle: a large binary artifact hosted as a Gitea package, not
|
||||
# committed. The folder's README is tracked; the zip is produced locally.
|
||||
packaging/npu-runtime/*.zip
|
||||
|
||||
# Vulkan loader staged next to the exe / into src-tauri by build.rs for the
|
||||
# --features vulkan build (bundled into the installer); a redistributable blob,
|
||||
# not committed.
|
||||
src-tauri/vulkan-1.dll
|
||||
|
||||
@@ -7,11 +7,13 @@ on-device acceleration (**NPU → GPU → CPU**), labels speakers, structures th
|
||||
Markdown notes, and optionally augments them with a locally hosted LLM (Ollama). Audio and
|
||||
transcripts **never leave the machine** unless you explicitly configure a destination.
|
||||
|
||||
> **Status: working application (v0.1.5).** Capture, transcription (CPU / Intel NPU / Vulkan
|
||||
> GPU), speaker diarization, storage + crash recovery, local-LLM summaries, opt-in recording,
|
||||
> at-rest encryption, and self-hosted sync are implemented and ship as signed **MSI + NSIS**
|
||||
> installers. Outlook `.pst`/calendar context and the coding-agent (MCP) handoff are in
|
||||
> progress. Build order and remaining tasks are in [`docs/05-roadmap.md`](docs/05-roadmap.md).
|
||||
> **Status: working application (v0.2.0).** Capture (system audio **+ your microphone**),
|
||||
> transcription (CPU / Intel NPU / Vulkan GPU), speaker diarization, storage + crash recovery,
|
||||
> local-LLM summaries, AI tags, opt-in recording with **in-app playback**, at-rest encryption,
|
||||
> and self-hosted sync are implemented and ship as a single signed **MSI + NSIS** universal
|
||||
> installer. Outlook `.pst`/calendar context (recurring-event import + filtering) is landing;
|
||||
> the coding-agent (MCP) handoff is in progress. Build order and remaining tasks are in
|
||||
> [`docs/05-roadmap.md`](docs/05-roadmap.md).
|
||||
|
||||
## Why WhispAssist — Granola vs Meetily vs WhispAssist
|
||||
|
||||
@@ -44,10 +46,12 @@ accelerated, zero-egress-by-default** option: it exploits the NPU/GPU in modern
|
||||
everything on the device unless you opt in, and adds Windows-specific context (Outlook) and a
|
||||
coding-agent handoff.
|
||||
|
||||
## What's built (v0.1.5)
|
||||
## What's built (v0.2.0)
|
||||
|
||||
- **Bot-free capture** — WASAPI loopback records the system mix (all participants) with no
|
||||
meeting bot and no per-app plumbing.
|
||||
- **Bot-free capture — now both sides** — WASAPI loopback records the system mix (all
|
||||
participants), and an optional **microphone** path captures your own voice, mixed into both the
|
||||
live transcript and the saved recording. Pick a specific output/input or turn the mic off in
|
||||
**Settings ▸ Hardware**. No meeting bot, no per-app plumbing.
|
||||
- **Local transcription with a hardware ladder** — whisper.cpp via `whisper-rs` on CPU; the
|
||||
**Intel NPU** via ONNX Runtime + OpenVINO; **GPU via Vulkan** (a single binary that runs on
|
||||
NVIDIA, AMD, and Intel). WA detects the hardware, picks the best backend
|
||||
@@ -61,17 +65,24 @@ coding-agent handoff.
|
||||
- **Storage & crash recovery** — SQLite + on-disk audio/transcripts under
|
||||
`%LOCALAPPDATA%\WhispAssist`. Audio is the source of truth; notes and transcripts regenerate
|
||||
after a crash.
|
||||
- **Opt-in recording** — off by default; `.wav` retained only when you turn it on, after a
|
||||
one-time consent notice.
|
||||
- **Opt-in recording + in-app playback** — off by default; `.wav` retained only when you turn it
|
||||
on, after a one-time consent notice. Play a saved recording back in the app — encrypted
|
||||
recordings are decrypted **in memory on the fly** (nothing plaintext is written to disk).
|
||||
Recordings are 16-bit for roughly half the size, and an accidental recording can be **cancelled**
|
||||
(audio + transcript deleted).
|
||||
- **Notes, summaries & AI tags** — Markdown notes with an **Editor/Preview** toggle; local-LLM
|
||||
summaries and one-click **tag generation** with a chip-based tag editor and tag filtering.
|
||||
- **At-rest encryption vault** — Argon2id key derivation + XChaCha20-Poly1305; transcripts,
|
||||
notes, summaries, and recordings sealed on disk; startup unlock gate; keys zeroized on lock.
|
||||
- **Self-hosted sync (optional, off by default)** — WebDAV (Nextcloud, ownCloud, Cloudreve,
|
||||
Seafile, Synology) plus OneDrive/Dropbox/Box (OAuth 2.0 PKCE); durable retry queue with
|
||||
backoff; **client-side encryption before upload** so the destination holds only ciphertext.
|
||||
Credentials live only in the OS credential store.
|
||||
backoff and **live per-item upload progress**; **client-side encryption before upload** so the
|
||||
destination holds only ciphertext. Credentials live only in the OS credential store.
|
||||
- **Optional hosted AI** — Anthropic and OpenAI-compatible providers behind the same
|
||||
`LlmProvider` interface, off by default (third-party egress, keys in the OS credential store).
|
||||
- **Installers** — signed MSI and NSIS `-setup.exe`.
|
||||
- **One universal installer** — a single signed MSI and NSIS `-setup.exe` that covers every
|
||||
machine: Vulkan for all GPUs, the Intel NPU path, and CPU fallback. The Vulkan loader is bundled
|
||||
so it launches even on machines without a GPU driver.
|
||||
|
||||
**In progress:** Outlook `.pst` + calendar context, the local **MCP server** that hands meeting
|
||||
context to your coding agents (Claude, Codex, Copilot, OpenCode), and MS Graph calendar.
|
||||
@@ -142,18 +153,28 @@ npm install
|
||||
npm run tauri dev # CPU/NPU build
|
||||
```
|
||||
|
||||
**GPU (Vulkan) build.** whisper.cpp's GPU backends are compiled in (not downloaded at runtime),
|
||||
so a GPU build needs a one-time toolchain setup — the **Vulkan SDK**, a **Ninja** generator, and
|
||||
a short target dir (to dodge Windows' 260-char path limit in the shader build):
|
||||
**Release build (single universal installer).** whisper.cpp's GPU backends are compiled in (not
|
||||
downloaded at runtime), so the release build needs a one-time toolchain setup — the **Vulkan SDK**,
|
||||
a **Ninja** generator, and a short target dir (to dodge Windows' 260-char path limit in the shader
|
||||
build):
|
||||
|
||||
```bash
|
||||
# after: Vulkan SDK installed, ninja.exe on PATH, vcvars64 loaded
|
||||
set VULKAN_SDK=C:\VulkanSDK\1.4.350.0
|
||||
set CMAKE_GENERATOR=Ninja
|
||||
set CARGO_TARGET_DIR=C:\wt
|
||||
npm run tauri build -- --features vulkan
|
||||
npm run tauri build -- --features vulkan --config src-tauri/tauri.vulkan.conf.json
|
||||
```
|
||||
|
||||
This one build covers **every** machine: Vulkan accelerates all GPUs (NVIDIA/AMD/Intel), the Intel
|
||||
NPU path works via the runtime OpenVINO download, and CPU is the fallback. The `--features vulkan`
|
||||
binary links `vulkan-1.dll`, so `build.rs` stages the redistributable Vulkan **loader** (from
|
||||
`VULKAN_SDK\Bin`, or System32) next to the exe and `tauri.vulkan.conf.json` bundles it into the
|
||||
installer — the app then launches even on a machine with no GPU driver (it reports zero Vulkan
|
||||
devices and decodes on the CPU). DirectML is intentionally not offered here because Vulkan already
|
||||
covers those GPUs; it's the GPU path only in the plain `npm run tauri build` (no Vulkan) variant,
|
||||
kept as an internal fallback.
|
||||
|
||||
CUDA (NVIDIA-only, faster) is planned as an optional variant. The full, gotcha-annotated build
|
||||
recipe lives in the project notes.
|
||||
|
||||
|
||||
@@ -16,7 +16,9 @@ Each requirement has a stable ID used across the roadmap, tests, and commits. Pr
|
||||
| FR-CAP-4 | M | 1 | Show an unambiguous "recording active" indicator (in-app banner + tray icon). |
|
||||
| FR-CAP-5 | S | 7 | Render a live input waveform / level meter while recording. |
|
||||
| FR-CAP-6 | S | 7 | Handle audio device changes mid-recording without losing the session. |
|
||||
| FR-CAP-7 | S | 1 | Optionally capture the user's **microphone** alongside loopback and mix it into the live transcript (default ON, local-only/no egress, selectable device + "off"). |
|
||||
| FR-CAP-7 | S | 1 | Optionally capture the user's **microphone** alongside loopback, mixing it into **both** the live transcript and the saved recording (default ON, local-only/no egress, selectable device + "off"). |
|
||||
| FR-CAP-8 | S | 1 | Write the retained recording as **16-bit PCM** at the device's native rate/channels — roughly half the size of the 32-bit-float mix, with no material quality loss for speech. |
|
||||
| FR-CAP-9 | S | 1 | **Cancel** an in-progress recording: stop capture, delete working files, and remove the meeting from the DB entirely (for one started by mistake — no finalize/transcript/sync). |
|
||||
|
||||
### Recording retention & consent (REC) — see ADR-0009
|
||||
|
||||
@@ -26,6 +28,7 @@ Each requirement has a stable ID used across the roadmap, tests, and commits. Pr
|
||||
| FR-REC-2 | M | 1 | Before retaining a recording for the first time (and shown near the toggle thereafter), display a consent notice: recording without participants' consent may be illegal in some regions; advise checking local laws. Require a one-time acknowledgment; store it. This is a caution, not legal advice. |
|
||||
| FR-REC-3 | M | 1 | When retention is on, the UI indicates the meeting is being **saved** (in addition to the "recording active" indicator, FR-CAP-4). |
|
||||
| FR-REC-4 | M | 2 | Deleting working audio on finalize happens only **after** the transcript is successfully finalized; never race with crash recovery (audio stays source of truth until then). |
|
||||
| FR-REC-5 | S | 2 | Play a meeting's retained `.wav` back in the app (decrypting a vault-sealed recording on demand for playback). |
|
||||
|
||||
### Hardware acceleration (HW)
|
||||
|
||||
@@ -153,6 +156,7 @@ the local LLM endpoint, and it is **off by default**. Primary targets are self-h
|
||||
| FR-SYNC-8 | M | 9 | Surface sync state in the UI and label targets: self-hosted/primary as "your server"; third-party clouds carry a clear "data leaves your device to a third party" banner. |
|
||||
| FR-SYNC-9 | S | 9 | **Secondary** targets via provider APIs + OAuth 2.0 (PKCE, loopback redirect): **OneDrive** (MS Graph), **Dropbox**, **Box**. |
|
||||
| FR-SYNC-10 | C | 9 | Optional client-side encryption of artifacts before upload (ties to FR-SEC-3): destination holds only ciphertext. |
|
||||
| FR-SYNC-11 | S | 9 | Show **live per-item upload progress** while syncing (stream the upload body and report bytes sent per artifact). |
|
||||
|
||||
### UX, accessibility, recovery (UX)
|
||||
|
||||
|
||||
@@ -125,7 +125,7 @@ privacy_self_check(): {
|
||||
## 2. Tauri events (Rust → frontend)
|
||||
|
||||
```ts
|
||||
"recording://state" { meetingId, state: "recording"|"paused"|"stopped", elapsedMs }
|
||||
"recording://state" { meetingId, state: "recording"|"paused"|"stopped"|"cancelled", elapsedMs }
|
||||
"recording://level" { meetingId, rms: number, peak: number } // waveform (FR-CAP-5)
|
||||
"recording://device" { meetingId, recovered: boolean, message: string } // capture device change (FR-CAP-6)
|
||||
"transcript://segment" { meetingId, segment: TranscriptSegment } // live segments (FR-TRX-2)
|
||||
|
||||
+1
-1
@@ -1,7 +1,7 @@
|
||||
{
|
||||
"name": "whispassist",
|
||||
"private": true,
|
||||
"version": "0.1.6",
|
||||
"version": "0.2.0",
|
||||
"type": "module",
|
||||
"description": "Privacy-first, fully local Windows meeting assistant.",
|
||||
"license": "MIT OR Apache-2.0",
|
||||
|
||||
Generated
+1
-1
@@ -5975,7 +5975,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "whispassist"
|
||||
version = "0.1.6"
|
||||
version = "0.2.0"
|
||||
dependencies = [
|
||||
"argon2",
|
||||
"async-trait",
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
[package]
|
||||
name = "whispassist"
|
||||
version = "0.1.6"
|
||||
version = "0.2.0"
|
||||
description = "Privacy-first, fully local Windows meeting assistant"
|
||||
authors = ["WhispAssist contributors"]
|
||||
license = "MIT OR Apache-2.0"
|
||||
|
||||
@@ -14,5 +14,51 @@ fn main() {
|
||||
println!("cargo:rustc-env=WA_GIT_HASH={hash}");
|
||||
println!("cargo:rerun-if-changed=../.git/logs/HEAD");
|
||||
|
||||
// The `vulkan` build links `vulkan-1.dll` at load time, so the exe won't
|
||||
// launch on a machine that lacks the Vulkan loader (no GPU driver / bare VM).
|
||||
// Bundling the redistributable loader (Apache-2.0) next to the exe makes the
|
||||
// single universal installer start everywhere — with no GPU it simply reports
|
||||
// zero devices and we fall back to CPU. Copied both next to the built exe (so
|
||||
// `tauri dev`/`cargo run` work) and into the crate dir where the bundler picks
|
||||
// it up as a resource (see tauri.vulkan.conf.json).
|
||||
if std::env::var_os("CARGO_FEATURE_VULKAN").is_some() {
|
||||
stage_vulkan_loader();
|
||||
}
|
||||
|
||||
tauri_build::build();
|
||||
}
|
||||
|
||||
/// Locate `vulkan-1.dll` (Vulkan SDK first, then System32) and copy it beside
|
||||
/// the compiled exe and into the crate dir for bundling. Warns rather than fails
|
||||
/// so a dev build on a machine with the loader already on PATH still succeeds.
|
||||
fn stage_vulkan_loader() {
|
||||
use std::path::{Path, PathBuf};
|
||||
|
||||
let source = std::env::var_os("VULKAN_SDK")
|
||||
.map(|sdk| Path::new(&sdk).join("Bin").join("vulkan-1.dll"))
|
||||
.filter(|p| p.exists())
|
||||
.or_else(|| {
|
||||
let sys = PathBuf::from(r"C:\Windows\System32\vulkan-1.dll");
|
||||
sys.exists().then_some(sys)
|
||||
});
|
||||
let Some(source) = source else {
|
||||
println!(
|
||||
"cargo:warning=vulkan feature is on but vulkan-1.dll wasn't found \
|
||||
(set VULKAN_SDK); the installer won't bundle the Vulkan loader"
|
||||
);
|
||||
return;
|
||||
};
|
||||
|
||||
// Beside the exe: OUT_DIR is target/<profile>/build/<pkg>-<hash>/out, so three
|
||||
// parents up is target/<profile> (correct even under CARGO_TARGET_DIR=C:\wt).
|
||||
if let Some(out_dir) = std::env::var_os("OUT_DIR") {
|
||||
if let Some(exe_dir) = Path::new(&out_dir).ancestors().nth(3) {
|
||||
let _ = std::fs::copy(&source, exe_dir.join("vulkan-1.dll"));
|
||||
}
|
||||
}
|
||||
// Into the crate dir for the bundler resource.
|
||||
let manifest_dir = std::env::var("CARGO_MANIFEST_DIR").expect("CARGO_MANIFEST_DIR");
|
||||
if let Err(e) = std::fs::copy(&source, Path::new(&manifest_dir).join("vulkan-1.dll")) {
|
||||
println!("cargo:warning=failed to stage vulkan-1.dll for bundling: {e}");
|
||||
}
|
||||
}
|
||||
|
||||
+396
-50
@@ -9,9 +9,11 @@
|
||||
//! crate's actual source and doesn't hold for this dependency; the capture loop
|
||||
//! below waits on the WASAPI event handle (with a short timeout so it can also
|
||||
//! notice a stop/pause request) rather than busy-polling.
|
||||
//! - Writes PCM to disk continuously, in the device's native format, so `audio.wav`
|
||||
//! is a byte-accurate capture (audio = source of truth). A downmixed/resampled
|
||||
//! 16kHz-mono copy is pushed into the bounded `FrameSink` for live transcription;
|
||||
//! - Writes PCM to disk continuously as 16-bit at the device's native rate/channels
|
||||
//! (FR-CAP-8), so `audio.wav` is a compact but faithful recording (audio = source
|
||||
//! of truth) — about half the size of the 32-bit-float mix. A downmixed/resampled
|
||||
//! 16kHz-mono copy (from the full-resolution samples) is pushed into the bounded
|
||||
//! `FrameSink` for live transcription;
|
||||
//! a full consumer drops preview frames (`try_send`) without blocking the disk
|
||||
//! write or the capture thread.
|
||||
//! - Does no inference itself.
|
||||
@@ -20,9 +22,9 @@ use hound::{SampleFormat, WavSpec, WavWriter};
|
||||
use std::fs::File;
|
||||
use std::io::BufWriter;
|
||||
use std::path::Path;
|
||||
use std::sync::atomic::{AtomicBool, Ordering};
|
||||
use std::sync::atomic::{AtomicBool, AtomicU32, Ordering};
|
||||
use std::sync::mpsc::SyncSender;
|
||||
use std::sync::Arc;
|
||||
use std::sync::{Arc, Mutex};
|
||||
use std::thread::{self, JoinHandle};
|
||||
use std::time::{Duration, Instant};
|
||||
use wasapi::{
|
||||
@@ -152,6 +154,63 @@ pub fn list_capture_devices() -> Result<Vec<AudioDeviceInfo>, AudioError> {
|
||||
list_devices(&Direction::Capture)
|
||||
}
|
||||
|
||||
/// Shared channel the microphone capture fills and the loopback capture drains,
|
||||
/// so the user's voice is mixed into the recording at the loopback's native rate
|
||||
/// (FR-CAP-7 in the saved `.wav`, not just the transcript). The loopback thread
|
||||
/// publishes its `rate` once it opens so the mic knows what to resample to; `buf`
|
||||
/// is capped to ~0.5s to bound the two independent clocks' drift.
|
||||
#[cfg(feature = "audio")]
|
||||
pub struct MicBridge {
|
||||
rate: AtomicU32,
|
||||
buf: Mutex<std::collections::VecDeque<f32>>,
|
||||
}
|
||||
|
||||
#[cfg(feature = "audio")]
|
||||
impl MicBridge {
|
||||
pub fn shared() -> Arc<Self> {
|
||||
Arc::new(Self {
|
||||
rate: AtomicU32::new(0),
|
||||
buf: Mutex::new(std::collections::VecDeque::new()),
|
||||
})
|
||||
}
|
||||
|
||||
/// Loopback: take `n` mic samples aligned to this callback's frames, padding
|
||||
/// with silence if the mic hasn't produced enough yet (underrun).
|
||||
fn pull(&self, n: usize) -> Vec<f32> {
|
||||
let mut out = Vec::with_capacity(n);
|
||||
if let Ok(mut buf) = self.buf.lock() {
|
||||
for _ in 0..n {
|
||||
out.push(buf.pop_front().unwrap_or(0.0));
|
||||
}
|
||||
} else {
|
||||
out.resize(n, 0.0);
|
||||
}
|
||||
out
|
||||
}
|
||||
|
||||
/// Mic: append resampled samples, dropping the oldest beyond `cap` so a
|
||||
/// faster mic clock (overrun) can't grow the buffer or the latency unbounded.
|
||||
fn push(&self, samples: &[f32], cap: usize) {
|
||||
if let Ok(mut buf) = self.buf.lock() {
|
||||
buf.extend(samples);
|
||||
while buf.len() > cap {
|
||||
buf.pop_front();
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Number of audio frames in a raw WASAPI byte buffer of the given format.
|
||||
#[cfg(feature = "audio")]
|
||||
fn frame_count(bytes: &[u8], format: &WaveFormat) -> usize {
|
||||
let bytes_per_frame = (format.get_bitspersample() as usize / 8) * format.get_nchannels() as usize;
|
||||
if bytes_per_frame == 0 {
|
||||
0
|
||||
} else {
|
||||
bytes.len() / bytes_per_frame
|
||||
}
|
||||
}
|
||||
|
||||
/// Default Windows WASAPI implementation (Phase 1, feature `audio`).
|
||||
#[cfg(feature = "audio")]
|
||||
pub struct WasapiCapture;
|
||||
@@ -162,6 +221,7 @@ impl WasapiCapture {
|
||||
/// `Some` only for the loopback stream (the byte-accurate recording); the mic
|
||||
/// stream produces frames for transcription but no WAV. `direction` selects
|
||||
/// loopback (Render) vs. plain microphone capture (Capture).
|
||||
#[allow(clippy::too_many_arguments)]
|
||||
fn start_capture(
|
||||
&self,
|
||||
thread_name: &str,
|
||||
@@ -170,6 +230,7 @@ impl WasapiCapture {
|
||||
device_id: Option<&str>,
|
||||
frame_sink: FrameSink,
|
||||
event_sink: EventSink,
|
||||
bridge: Option<Arc<MicBridge>>,
|
||||
) -> Result<CaptureHandle, AudioError> {
|
||||
let running = Arc::new(AtomicBool::new(true));
|
||||
let paused = Arc::new(AtomicBool::new(false));
|
||||
@@ -189,6 +250,7 @@ impl WasapiCapture {
|
||||
&event_sink,
|
||||
&running_th,
|
||||
&paused_th,
|
||||
bridge.as_ref(),
|
||||
)
|
||||
})
|
||||
.map_err(|e| AudioError::Capture(format!("spawn failed: {e}")))?;
|
||||
@@ -199,6 +261,48 @@ impl WasapiCapture {
|
||||
thread,
|
||||
})
|
||||
}
|
||||
|
||||
/// Loopback capture that mixes the microphone (delivered via `bridge`) into
|
||||
/// the recording, so the saved `.wav` contains both sides of the meeting at
|
||||
/// the loopback's native rate/channels (FR-CAP-7).
|
||||
pub fn start_loopback_recording(
|
||||
&self,
|
||||
wav_path: &Path,
|
||||
device_id: Option<&str>,
|
||||
frame_sink: FrameSink,
|
||||
event_sink: EventSink,
|
||||
bridge: Arc<MicBridge>,
|
||||
) -> Result<CaptureHandle, AudioError> {
|
||||
self.start_capture(
|
||||
"wa-audio-capture",
|
||||
Some(wav_path),
|
||||
Direction::Render,
|
||||
device_id,
|
||||
frame_sink,
|
||||
event_sink,
|
||||
Some(bridge),
|
||||
)
|
||||
}
|
||||
|
||||
/// Microphone capture that both feeds the live-transcript mixer (`frame_sink`)
|
||||
/// and pushes its audio into `bridge` for the loopback thread to record.
|
||||
pub fn start_microphone_recording(
|
||||
&self,
|
||||
device_id: Option<&str>,
|
||||
frame_sink: FrameSink,
|
||||
event_sink: EventSink,
|
||||
bridge: Arc<MicBridge>,
|
||||
) -> Result<CaptureHandle, AudioError> {
|
||||
self.start_capture(
|
||||
"wa-mic-capture",
|
||||
None,
|
||||
Direction::Capture,
|
||||
device_id,
|
||||
frame_sink,
|
||||
event_sink,
|
||||
Some(bridge),
|
||||
)
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(feature = "audio")]
|
||||
@@ -217,6 +321,7 @@ impl AudioCapture for WasapiCapture {
|
||||
device_id,
|
||||
frame_sink,
|
||||
event_sink,
|
||||
None,
|
||||
)
|
||||
}
|
||||
|
||||
@@ -233,6 +338,7 @@ impl AudioCapture for WasapiCapture {
|
||||
device_id,
|
||||
frame_sink,
|
||||
event_sink,
|
||||
None,
|
||||
)
|
||||
}
|
||||
|
||||
@@ -376,6 +482,7 @@ const LEVEL_EMIT_INTERVAL: Duration = Duration::from_millis(50);
|
||||
/// WASAPI client and the WAV writer; exits (and finalizes the WAV) once `running`
|
||||
/// is cleared.
|
||||
#[cfg(feature = "audio")]
|
||||
#[allow(clippy::too_many_arguments)]
|
||||
fn capture_loop(
|
||||
wav_path: Option<&Path>,
|
||||
direction: Direction,
|
||||
@@ -384,15 +491,25 @@ fn capture_loop(
|
||||
event_sink: &EventSink,
|
||||
running: &AtomicBool,
|
||||
paused: &AtomicBool,
|
||||
bridge: Option<&Arc<MicBridge>>,
|
||||
) -> Result<CaptureSummary, AudioError> {
|
||||
wasapi::initialize_mta()
|
||||
.ok()
|
||||
.map_err(|e| AudioError::Device(format!("COM init failed: {e}")))?;
|
||||
|
||||
let is_loopback = matches!(direction, Direction::Render);
|
||||
// Only the loopback stream emits level updates (FR-CAP-5) and writes a WAV;
|
||||
// the mic stream just contributes frames to the transcript mixer (FR-CAP-7).
|
||||
let emit_level = wav_path.is_some();
|
||||
let mut session = open_capture_session(&direction, device_id)?;
|
||||
// Loopback publishes its rate so the mic knows what to resample to before
|
||||
// pushing into the shared bridge (mic-into-recording, FR-CAP-7).
|
||||
if is_loopback {
|
||||
if let Some(b) = bridge {
|
||||
b.rate
|
||||
.store(session.format.get_samplespersec(), Ordering::Relaxed);
|
||||
}
|
||||
}
|
||||
let spec = wav_spec_for(&session.format)?;
|
||||
let mut writer = match wav_path {
|
||||
Some(path) => Some(WavWriter::create(path, spec).map_err(|e| {
|
||||
@@ -407,6 +524,9 @@ fn capture_loop(
|
||||
.map_err(|e| AudioError::Capture(format!("start_stream failed: {e}")))?;
|
||||
|
||||
let mut resampler = Resampler::new(session.format.get_samplespersec());
|
||||
// Mic-only: resamples the mic to the loopback's rate for the recording bridge
|
||||
// (created lazily once the loopback has published its rate).
|
||||
let mut bridge_resampler: Option<Resampler> = None;
|
||||
let mut queue: std::collections::VecDeque<u8> = std::collections::VecDeque::new();
|
||||
let mut frames_written: u64 = 0;
|
||||
let mut last_level_emit = Instant::now() - LEVEL_EMIT_INTERVAL;
|
||||
@@ -438,6 +558,13 @@ fn capture_loop(
|
||||
.start_stream()
|
||||
.map_err(|e| AudioError::Capture(format!("restart_stream failed: {e}")))?;
|
||||
resampler = Resampler::new(new_session.format.get_samplespersec());
|
||||
bridge_resampler = None;
|
||||
if is_loopback {
|
||||
if let Some(b) = bridge {
|
||||
b.rate
|
||||
.store(new_session.format.get_samplespersec(), Ordering::Relaxed);
|
||||
}
|
||||
}
|
||||
session = new_session;
|
||||
let _ = event_sink.try_send(CaptureEvent::DeviceChanged {
|
||||
recovered: true,
|
||||
@@ -456,11 +583,34 @@ fn capture_loop(
|
||||
continue;
|
||||
}
|
||||
|
||||
// Loopback: mix the mic in per-frame as we write (mic silence when it
|
||||
// hasn't produced enough), so the recording holds both sides (FR-CAP-7).
|
||||
if let Some(w) = writer.as_mut() {
|
||||
frames_written += write_wav_bytes(w, &bytes, &session.format)?;
|
||||
let mic = match bridge {
|
||||
Some(b) if is_loopback => b.pull(frame_count(&bytes, &session.format)),
|
||||
_ => Vec::new(),
|
||||
};
|
||||
frames_written += write_wav_bytes(w, &bytes, &session.format, &mic)?;
|
||||
}
|
||||
let mono = decode_mono_f32(&bytes, &session.format)?;
|
||||
|
||||
// Mic: feed the shared bridge (resampled to the loopback's rate) so the
|
||||
// loopback thread can fold it into the recording.
|
||||
if !is_loopback {
|
||||
if let Some(b) = bridge {
|
||||
let rate = b.rate.load(Ordering::Relaxed);
|
||||
if rate != 0 {
|
||||
let rs = bridge_resampler.get_or_insert_with(|| {
|
||||
Resampler::new_to(session.format.get_samplespersec(), rate)
|
||||
});
|
||||
let resampled = rs.process(&mono);
|
||||
if !resampled.is_empty() {
|
||||
b.push(&resampled, (rate / 2) as usize);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
if emit_level && last_level_emit.elapsed() >= LEVEL_EMIT_INTERVAL {
|
||||
let _ = event_sink.try_send(CaptureEvent::Level(audio_level(&mono)));
|
||||
last_level_emit = Instant::now();
|
||||
@@ -486,58 +636,79 @@ fn capture_loop(
|
||||
})
|
||||
}
|
||||
|
||||
/// The on-disk recording is always 16-bit PCM at the device's native rate and
|
||||
/// channel count — half the size of the 32-bit-float mix format WASAPI hands us,
|
||||
/// with no perceptible quality loss for speech (FR-CAP-8). The full-resolution
|
||||
/// native samples still feed transcription untouched (see `decode_mono_f32`).
|
||||
fn wav_spec_for(format: &WaveFormat) -> Result<WavSpec, AudioError> {
|
||||
let sample_format = match format
|
||||
.get_subformat()
|
||||
.map_err(|e| AudioError::Device(format!("unrecognized mix format: {e}")))?
|
||||
{
|
||||
SampleType::Float => SampleFormat::Float,
|
||||
SampleType::Int => SampleFormat::Int,
|
||||
};
|
||||
Ok(WavSpec {
|
||||
channels: format.get_nchannels(),
|
||||
sample_rate: format.get_samplespersec(),
|
||||
bits_per_sample: format.get_bitspersample(),
|
||||
sample_format,
|
||||
bits_per_sample: 16,
|
||||
sample_format: SampleFormat::Int,
|
||||
})
|
||||
}
|
||||
|
||||
/// Write raw WASAPI capture bytes to the WAV writer in their native format.
|
||||
/// Returns the number of frames written. Only the two mix formats WASAPI shared
|
||||
/// mode actually produces in practice (32-bit float, 16-bit PCM) are supported;
|
||||
/// anything else is a loud error rather than a silently corrupt recording.
|
||||
/// Write raw WASAPI capture bytes to the WAV writer as 16-bit PCM (FR-CAP-8),
|
||||
/// quantizing the 32-bit-float mix down so `audio.wav` is ~half the size, and
|
||||
/// adding one `mic` sample per frame to every channel so the recording carries
|
||||
/// the user's voice too (FR-CAP-7); `mic` may be shorter than the frame count
|
||||
/// (missing entries mix as silence) or empty (loopback only). Returns the number
|
||||
/// of frames written. Only the two mix formats WASAPI shared mode actually
|
||||
/// produces (32-bit float, 16-bit PCM) are supported; anything else is a loud
|
||||
/// error rather than a silently corrupt recording.
|
||||
fn write_wav_bytes(
|
||||
writer: &mut WavWriter<BufWriter<File>>,
|
||||
bytes: &[u8],
|
||||
format: &WaveFormat,
|
||||
mic: &[f32],
|
||||
) -> Result<u64, AudioError> {
|
||||
let sample_type = format
|
||||
.get_subformat()
|
||||
.map_err(|e| AudioError::Device(format!("unrecognized mix format: {e}")))?;
|
||||
let channels = format.get_nchannels() as u64;
|
||||
let channels = format.get_nchannels() as usize;
|
||||
if channels == 0 {
|
||||
return Ok(0);
|
||||
}
|
||||
let mut frames = 0u64;
|
||||
match (sample_type, format.get_bitspersample()) {
|
||||
(SampleType::Float, 32) => {
|
||||
for chunk in bytes.chunks_exact(4) {
|
||||
let v = f32::from_le_bytes(chunk.try_into().unwrap());
|
||||
writer
|
||||
.write_sample(v)
|
||||
.map_err(|e| AudioError::Capture(format!("wav write: {e}")))?;
|
||||
for frame in bytes.chunks_exact(4 * channels) {
|
||||
let add = mic.get(frames as usize).copied().unwrap_or(0.0);
|
||||
for c in frame.chunks_exact(4) {
|
||||
let v = f32::from_le_bytes(c.try_into().unwrap()) + add;
|
||||
writer
|
||||
.write_sample(f32_to_i16(v))
|
||||
.map_err(|e| AudioError::Capture(format!("wav write: {e}")))?;
|
||||
}
|
||||
frames += 1;
|
||||
}
|
||||
Ok(bytes.len() as u64 / 4 / channels.max(1))
|
||||
}
|
||||
(SampleType::Int, 16) => {
|
||||
for chunk in bytes.chunks_exact(2) {
|
||||
let v = i16::from_le_bytes(chunk.try_into().unwrap());
|
||||
writer
|
||||
.write_sample(v)
|
||||
.map_err(|e| AudioError::Capture(format!("wav write: {e}")))?;
|
||||
for frame in bytes.chunks_exact(2 * channels) {
|
||||
let add = mic.get(frames as usize).copied().unwrap_or(0.0);
|
||||
for c in frame.chunks_exact(2) {
|
||||
let v = i16::from_le_bytes(c.try_into().unwrap()) as f32 / i16::MAX as f32 + add;
|
||||
writer
|
||||
.write_sample(f32_to_i16(v))
|
||||
.map_err(|e| AudioError::Capture(format!("wav write: {e}")))?;
|
||||
}
|
||||
frames += 1;
|
||||
}
|
||||
Ok(bytes.len() as u64 / 2 / channels.max(1))
|
||||
}
|
||||
(st, bits) => Err(AudioError::Device(format!(
|
||||
"unsupported capture format: {st} {bits}-bit"
|
||||
))),
|
||||
(st, bits) => {
|
||||
return Err(AudioError::Device(format!(
|
||||
"unsupported capture format: {st} {bits}-bit"
|
||||
)))
|
||||
}
|
||||
}
|
||||
Ok(frames)
|
||||
}
|
||||
|
||||
/// Quantize a `[-1.0, 1.0]` float sample to 16-bit PCM, clamping out-of-range
|
||||
/// values so an over-unity sample wraps to a click instead of full-scale noise.
|
||||
fn f32_to_i16(v: f32) -> i16 {
|
||||
(v.clamp(-1.0, 1.0) * i16::MAX as f32) as i16
|
||||
}
|
||||
|
||||
/// Downmix raw WASAPI capture bytes to mono `f32` in `[-1.0, 1.0]`, at the
|
||||
@@ -588,14 +759,22 @@ const TARGET_SAMPLE_RATE: u32 = 16_000;
|
||||
/// resampler if transcription accuracy complaints trace back to aliasing.`
|
||||
struct Resampler {
|
||||
src_rate: u32,
|
||||
dst_rate: u32,
|
||||
pos: f64,
|
||||
carry: Vec<f32>,
|
||||
}
|
||||
|
||||
impl Resampler {
|
||||
fn new(src_rate: u32) -> Self {
|
||||
Self::new_to(src_rate, TARGET_SAMPLE_RATE)
|
||||
}
|
||||
|
||||
/// Resample `src_rate` -> `dst_rate` (used for the mic->loopback-rate bridge
|
||||
/// as well as the 16kHz transcription path).
|
||||
fn new_to(src_rate: u32, dst_rate: u32) -> Self {
|
||||
Self {
|
||||
src_rate,
|
||||
dst_rate,
|
||||
pos: 0.0,
|
||||
carry: Vec::new(),
|
||||
}
|
||||
@@ -605,7 +784,7 @@ impl Resampler {
|
||||
if mono_in.is_empty() {
|
||||
return Vec::new();
|
||||
}
|
||||
if self.src_rate == TARGET_SAMPLE_RATE {
|
||||
if self.src_rate == self.dst_rate {
|
||||
return mono_in.to_vec();
|
||||
}
|
||||
|
||||
@@ -616,7 +795,7 @@ impl Resampler {
|
||||
return Vec::new();
|
||||
}
|
||||
|
||||
let ratio = self.src_rate as f64 / TARGET_SAMPLE_RATE as f64;
|
||||
let ratio = self.src_rate as f64 / self.dst_rate as f64;
|
||||
let mut out = Vec::new();
|
||||
while (self.pos as usize) + 1 < buf.len() {
|
||||
let i = self.pos as usize;
|
||||
@@ -835,38 +1014,95 @@ mod tests {
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn wav_writer_roundtrip_produces_valid_header_and_duration() {
|
||||
fn wav_writer_writes_16bit_from_a_float_mix_and_keeps_duration() {
|
||||
// A 32-bit-float mix is captured but written as 16-bit PCM at the same
|
||||
// rate/channels (FR-CAP-8) — half the bytes, same frame count.
|
||||
let dir = std::env::temp_dir().join(format!("wa-test-{}", uuid::Uuid::new_v4()));
|
||||
std::fs::create_dir_all(&dir).unwrap();
|
||||
let path = dir.join("audio.wav");
|
||||
|
||||
let spec = WavSpec {
|
||||
channels: 2,
|
||||
sample_rate: 48_000,
|
||||
bits_per_sample: 32,
|
||||
sample_format: SampleFormat::Float,
|
||||
};
|
||||
let mut writer = WavWriter::create(&path, spec).unwrap();
|
||||
let samples: Vec<f32> = (0..2000).map(|i| (i as f32 / 2000.0) - 0.5).collect();
|
||||
let bytes: Vec<u8> = samples.iter().flat_map(|s| s.to_le_bytes()).collect();
|
||||
|
||||
// format.get_subformat()/get_bitspersample() need a real WaveFormat; build one
|
||||
// via WaveFormat::new instead of a live device for this pure unit test.
|
||||
let format = WaveFormat::new(32, 32, &SampleType::Float, 48_000, 2, None);
|
||||
write_wav_bytes(&mut writer, &bytes, &format).unwrap();
|
||||
let spec = wav_spec_for(&format).unwrap();
|
||||
assert_eq!(spec.bits_per_sample, 16);
|
||||
assert_eq!(spec.sample_format, SampleFormat::Int);
|
||||
|
||||
let mut writer = WavWriter::create(&path, spec).unwrap();
|
||||
let samples: Vec<f32> = (0..2000).map(|i| (i as f32 / 2000.0) - 0.5).collect();
|
||||
let bytes: Vec<u8> = samples.iter().flat_map(|s| s.to_le_bytes()).collect();
|
||||
write_wav_bytes(&mut writer, &bytes, &format, &[]).unwrap();
|
||||
writer.finalize().unwrap();
|
||||
|
||||
let reader = hound::WavReader::open(&path).unwrap();
|
||||
let read_spec = reader.spec();
|
||||
assert_eq!(read_spec.channels, 2);
|
||||
assert_eq!(read_spec.sample_rate, 48_000);
|
||||
assert_eq!(read_spec.bits_per_sample, 32);
|
||||
assert_eq!(read_spec.bits_per_sample, 16);
|
||||
let frame_count = reader.duration();
|
||||
assert_eq!(frame_count as usize, samples.len() / 2);
|
||||
|
||||
let _ = std::fs::remove_dir_all(&dir);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn f32_to_i16_clamps_and_scales() {
|
||||
assert_eq!(f32_to_i16(0.0), 0);
|
||||
assert_eq!(f32_to_i16(1.0), i16::MAX);
|
||||
assert_eq!(f32_to_i16(2.0), i16::MAX); // clamped, no wrap
|
||||
assert_eq!(f32_to_i16(-2.0), -i16::MAX); // clamped to -32767
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn write_wav_bytes_mixes_the_mic_into_every_channel() {
|
||||
// Silent stereo loopback + one mic sample of 0.5 for the single frame
|
||||
// should land 0.5 (16383) on BOTH channels — the user's voice recorded.
|
||||
let dir = std::env::temp_dir().join(format!("wa-test-{}", uuid::Uuid::new_v4()));
|
||||
std::fs::create_dir_all(&dir).unwrap();
|
||||
let path = dir.join("audio.wav");
|
||||
let format = WaveFormat::new(32, 32, &SampleType::Float, 48_000, 2, None);
|
||||
let mut writer = WavWriter::create(&path, wav_spec_for(&format).unwrap()).unwrap();
|
||||
// One stereo frame of silence (L=0.0, R=0.0).
|
||||
let bytes: Vec<u8> = [0.0f32, 0.0]
|
||||
.iter()
|
||||
.flat_map(|s| s.to_le_bytes())
|
||||
.collect();
|
||||
let frames = write_wav_bytes(&mut writer, &bytes, &format, &[0.5]).unwrap();
|
||||
writer.finalize().unwrap();
|
||||
assert_eq!(frames, 1);
|
||||
|
||||
let samples: Vec<i16> = hound::WavReader::open(&path)
|
||||
.unwrap()
|
||||
.into_samples::<i16>()
|
||||
.map(|s| s.unwrap())
|
||||
.collect();
|
||||
assert_eq!(samples, vec![16383, 16383]); // mic folded into both channels
|
||||
let _ = std::fs::remove_dir_all(&dir);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn mic_bridge_pull_pads_with_silence_and_push_caps() {
|
||||
let bridge = MicBridge::shared();
|
||||
// Pull from empty -> all silence.
|
||||
assert_eq!(bridge.pull(3), vec![0.0, 0.0, 0.0]);
|
||||
// Push 5 with a cap of 3 -> only the newest 3 survive.
|
||||
bridge.push(&[1.0, 2.0, 3.0, 4.0, 5.0], 3);
|
||||
assert_eq!(bridge.pull(3), vec![3.0, 4.0, 5.0]);
|
||||
// Now empty again -> padded.
|
||||
assert_eq!(bridge.pull(2), vec![0.0, 0.0]);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn resampler_new_to_downsamples_between_arbitrary_rates() {
|
||||
// 44.1kHz -> 16kHz still lands on a passthrough when rates match…
|
||||
let mut same = Resampler::new_to(16_000, 16_000);
|
||||
assert_eq!(same.process(&[0.1, 0.2, 0.3]), vec![0.1, 0.2, 0.3]);
|
||||
// …and produces fewer samples when downsampling (48k -> 44.1k).
|
||||
let mut down = Resampler::new_to(48_000, 44_100);
|
||||
let out = down.process(&(0..480).map(|i| i as f32).collect::<Vec<_>>());
|
||||
assert!(out.len() >= 430 && out.len() <= 450, "got {}", out.len());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn read_wav_mono_16k_reads_a_normally_finalized_file() {
|
||||
let dir = std::env::temp_dir().join(format!("wa-test-{}", uuid::Uuid::new_v4()));
|
||||
@@ -1023,6 +1259,116 @@ mod tests {
|
||||
assert_eq!(ids.len(), devices.len());
|
||||
}
|
||||
|
||||
#[test]
|
||||
#[ignore = "captures real loopback while playing a sound; run with --ignored"]
|
||||
fn loopback_16bit_wav_captures_played_audio() {
|
||||
// Definitive check that the 32-bit-float -> 16-bit write path (FR-CAP-8)
|
||||
// records real loopback audio (not silence): start capture, play a known
|
||||
// WAV through the default device, stop, and assert the 16-bit WAV has
|
||||
// non-zero samples.
|
||||
let dir = std::env::temp_dir().join(format!("wa-loop-{}", uuid::Uuid::new_v4()));
|
||||
std::fs::create_dir_all(&dir).unwrap();
|
||||
let path = dir.join("audio.wav");
|
||||
let (frame_tx, _frame_rx) = std::sync::mpsc::sync_channel::<Vec<f32>>(64);
|
||||
let (event_tx, _event_rx) = std::sync::mpsc::sync_channel::<CaptureEvent>(8);
|
||||
|
||||
let handle = match WasapiCapture.start(&path, None, frame_tx, event_tx) {
|
||||
Ok(h) => h,
|
||||
Err(e) => {
|
||||
eprintln!("[loopback] could not open default render loopback: {e} (skipping)");
|
||||
return;
|
||||
}
|
||||
};
|
||||
// Play an audible system sound through the default render device so
|
||||
// loopback has something to capture. PlaySync blocks for its duration.
|
||||
let _ = std::process::Command::new("powershell")
|
||||
.args([
|
||||
"-NoProfile",
|
||||
"-Command",
|
||||
"(New-Object Media.SoundPlayer 'C:\\Windows\\Media\\Alarm01.wav').PlaySync()",
|
||||
])
|
||||
.status();
|
||||
std::thread::sleep(Duration::from_millis(500));
|
||||
let summary = WasapiCapture.stop(handle).unwrap();
|
||||
|
||||
let raw = std::fs::read(&path).unwrap();
|
||||
let data_at = raw.windows(4).position(|w| w == b"data").unwrap() + 8;
|
||||
let pcm = &raw[data_at..];
|
||||
let max_abs = pcm
|
||||
.chunks_exact(2)
|
||||
.map(|c| i16::from_le_bytes([c[0], c[1]]).unsigned_abs())
|
||||
.max()
|
||||
.unwrap_or(0);
|
||||
eprintln!(
|
||||
"[loopback] {}Hz {}ch, {} data bytes, peak i16={max_abs}",
|
||||
summary.sample_rate,
|
||||
summary.channels,
|
||||
pcm.len()
|
||||
);
|
||||
let _ = std::fs::remove_dir_all(&dir);
|
||||
assert!(
|
||||
max_abs > 0,
|
||||
"16-bit loopback WAV is silent despite playing audio — real conversion bug"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
#[ignore = "captures loopback+mic together while playing a sound; run with --ignored"]
|
||||
fn loopback_recording_with_mic_bridge_captures_played_audio() {
|
||||
// The exact production path when the mic is on: loopback + mic share a
|
||||
// MicBridge and the recording must still contain the loopback audio (mic
|
||||
// silence here just mixes in as 0). Proves the bridge doesn't zero or
|
||||
// deadlock the recording.
|
||||
let dir = std::env::temp_dir().join(format!("wa-mixrec-{}", uuid::Uuid::new_v4()));
|
||||
std::fs::create_dir_all(&dir).unwrap();
|
||||
let path = dir.join("audio.wav");
|
||||
let bridge = MicBridge::shared();
|
||||
let (loop_tx, _lr) = std::sync::mpsc::sync_channel::<Vec<f32>>(64);
|
||||
let (mic_tx, _mr) = std::sync::mpsc::sync_channel::<Vec<f32>>(64);
|
||||
let (ev_tx, _ev) = std::sync::mpsc::sync_channel::<CaptureEvent>(8);
|
||||
let (ev_tx2, _ev2) = std::sync::mpsc::sync_channel::<CaptureEvent>(8);
|
||||
|
||||
let loop_h =
|
||||
match WasapiCapture.start_loopback_recording(&path, None, loop_tx, ev_tx, bridge.clone())
|
||||
{
|
||||
Ok(h) => h,
|
||||
Err(e) => {
|
||||
eprintln!("[mixrec] no loopback device: {e} (skipping)");
|
||||
return;
|
||||
}
|
||||
};
|
||||
let mic_h = WasapiCapture
|
||||
.start_microphone_recording(None, mic_tx, ev_tx2, bridge)
|
||||
.ok();
|
||||
|
||||
let _ = std::process::Command::new("powershell")
|
||||
.args([
|
||||
"-NoProfile",
|
||||
"-Command",
|
||||
"(New-Object Media.SoundPlayer 'C:\\Windows\\Media\\Alarm01.wav').PlaySync()",
|
||||
])
|
||||
.status();
|
||||
std::thread::sleep(Duration::from_millis(500));
|
||||
let summary = WasapiCapture.stop(loop_h).unwrap();
|
||||
if let Some(m) = mic_h {
|
||||
let _ = WasapiCapture.stop(m);
|
||||
}
|
||||
|
||||
let raw = std::fs::read(&path).unwrap();
|
||||
let data_at = raw.windows(4).position(|w| w == b"data").unwrap() + 8;
|
||||
let peak = raw[data_at..]
|
||||
.chunks_exact(2)
|
||||
.map(|c| i16::from_le_bytes([c[0], c[1]]).unsigned_abs())
|
||||
.max()
|
||||
.unwrap_or(0);
|
||||
eprintln!(
|
||||
"[mixrec] {}Hz {}ch peak i16={peak}",
|
||||
summary.sample_rate, summary.channels
|
||||
);
|
||||
let _ = std::fs::remove_dir_all(&dir);
|
||||
assert!(peak > 0, "combined loopback+mic recording is silent");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn find_render_device_resolves_a_real_enumerated_id() {
|
||||
let devices = list_render_devices().unwrap();
|
||||
|
||||
+229
-9
@@ -296,17 +296,27 @@ pub async fn start_recording(
|
||||
// back to loopback-only (the mixer forwards loopback alone once its sink
|
||||
// drops).
|
||||
let (capture, mic_capture) = if settings.microphone_enabled {
|
||||
// The mixer sums both streams for the live transcript; the bridge carries
|
||||
// the mic into the loopback thread so the recorded WAV holds both sides
|
||||
// at native quality (FR-CAP-7).
|
||||
let bridge = crate::audio::MicBridge::shared();
|
||||
let (loop_sink, mic_sink) = crate::audio::spawn_mixer(frame_tx);
|
||||
let capture = WasapiCapture
|
||||
.start(
|
||||
.start_loopback_recording(
|
||||
&wav_path,
|
||||
settings.audio_output_device.as_deref(),
|
||||
loop_sink,
|
||||
event_tx.clone(),
|
||||
bridge.clone(),
|
||||
)
|
||||
.map_err(|e| WaError::new("audio", e.to_string()))?;
|
||||
let mic = WasapiCapture
|
||||
.start_microphone(settings.audio_input_device.as_deref(), mic_sink, event_tx)
|
||||
.start_microphone_recording(
|
||||
settings.audio_input_device.as_deref(),
|
||||
mic_sink,
|
||||
event_tx,
|
||||
bridge,
|
||||
)
|
||||
.map_err(|e| tracing::warn!("microphone capture unavailable: {e}"))
|
||||
.ok();
|
||||
(capture, mic)
|
||||
@@ -640,6 +650,58 @@ pub async fn stop_recording(
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Abandon the in-progress recording: stop capture, drop the transcript, and
|
||||
/// delete the meeting row + its working files entirely — for a recording that
|
||||
/// was started by mistake. Unlike `stop_recording`, nothing is finalized,
|
||||
/// transcribed further, diarized, retained, or synced.
|
||||
#[tauri::command]
|
||||
pub async fn cancel_recording(
|
||||
app: AppHandle,
|
||||
state: State<'_, AppState>,
|
||||
meeting_id: MeetingId,
|
||||
) -> WaResult<()> {
|
||||
let mut guard = state.session.lock().await;
|
||||
let session = match guard.take() {
|
||||
Some(s) if s.meeting_id == meeting_id => s,
|
||||
Some(s) => {
|
||||
*guard = Some(s);
|
||||
return Err(WaError::new(
|
||||
"recording",
|
||||
"meeting_id does not match the active recording",
|
||||
));
|
||||
}
|
||||
None => {
|
||||
return Err(WaError::new(
|
||||
"recording",
|
||||
"no meeting is currently recording",
|
||||
))
|
||||
}
|
||||
};
|
||||
drop(guard);
|
||||
|
||||
// Stop both captures so their frame sinks drop and the transcription worker
|
||||
// exits; join it so nothing is still touching the files we're about to delete.
|
||||
let _ = WasapiCapture.stop(session.capture);
|
||||
if let Some(mic) = session.mic_capture {
|
||||
let _ = WasapiCapture.stop(mic);
|
||||
}
|
||||
let _ = session.transcription_worker.join();
|
||||
|
||||
// Remove the DB row + the whole meeting folder (working audio.wav included).
|
||||
state
|
||||
.store
|
||||
.delete_meeting(&meeting_id)
|
||||
.await
|
||||
.map_err(|e| WaError::new("storage", e.to_string()))?;
|
||||
|
||||
crate::update_tray_tooltip(&app, "WhispAssist — idle");
|
||||
let _ = app.emit(
|
||||
"recording://state",
|
||||
serde_json::json!({ "meetingId": meeting_id, "state": "cancelled", "elapsedMs": 0 }),
|
||||
);
|
||||
Ok(())
|
||||
}
|
||||
|
||||
#[tauri::command]
|
||||
pub async fn pause_recording(
|
||||
app: AppHandle,
|
||||
@@ -1586,6 +1648,141 @@ pub async fn delete_meeting(state: State<'_, AppState>, meeting_id: MeetingId) -
|
||||
.map_err(|e| WaError::new("storage", e.to_string()))
|
||||
}
|
||||
|
||||
/// Return the `waaudio://` URL the in-app player loads for a meeting's recording
|
||||
/// (FR-REC-5). The bytes are decrypted in memory on demand by `serve_recording`
|
||||
/// — nothing plaintext is ever written to disk. Prechecks that a recording
|
||||
/// exists and (if sealed) the vault is unlocked, so the UI can show a clear
|
||||
/// error before playback; also clears any stale plaintext `audio.play.wav` left
|
||||
/// by the previous file-based player.
|
||||
#[tauri::command]
|
||||
pub async fn recording_playback_path(meeting_id: MeetingId) -> WaResult<String> {
|
||||
let id = meeting_id.clone();
|
||||
tauri::async_runtime::spawn_blocking(move || {
|
||||
let dir = meeting_dir(&id);
|
||||
let wav = dir.join("audio.wav");
|
||||
if !wav.exists() {
|
||||
return Err(WaError::new(
|
||||
"recording",
|
||||
"no saved recording for this meeting",
|
||||
));
|
||||
}
|
||||
// Cheap sealed check — read only the magic prefix, not the whole file.
|
||||
let mut head = [0u8; 8];
|
||||
let sealed = std::fs::File::open(&wav)
|
||||
.and_then(|mut f| {
|
||||
use std::io::Read;
|
||||
let n = f.read(&mut head)?;
|
||||
Ok(n)
|
||||
})
|
||||
.map(|n| crate::vault::is_sealed(&head[..n]))
|
||||
.unwrap_or(false);
|
||||
if sealed && !crate::vault::is_unlocked() {
|
||||
return Err(WaError::new(
|
||||
"recording",
|
||||
"unlock the vault to play this recording",
|
||||
));
|
||||
}
|
||||
// Drop any plaintext temp the old file-based player left behind.
|
||||
let _ = std::fs::remove_file(dir.join("audio.play.wav"));
|
||||
Ok(())
|
||||
})
|
||||
.await
|
||||
.map_err(|e| WaError::new("recording", e.to_string()))??;
|
||||
Ok(format!("http://waaudio.localhost/{meeting_id}"))
|
||||
}
|
||||
|
||||
/// Custom-scheme handler backing the `waaudio://` URL: reads the meeting's
|
||||
/// `audio.wav`, decrypts it in memory if sealed (T8.8), and streams the PCM to
|
||||
/// the `<audio>` element with byte-range support for seeking. The plaintext
|
||||
/// never touches the filesystem, so playback can't undermine encryption at rest.
|
||||
pub(crate) fn serve_recording(
|
||||
request: &tauri::http::Request<Vec<u8>>,
|
||||
) -> tauri::http::Response<Vec<u8>> {
|
||||
use tauri::http::{header, Response, StatusCode};
|
||||
let fail = |code: StatusCode| {
|
||||
Response::builder()
|
||||
.status(code)
|
||||
.body(Vec::new())
|
||||
.expect("static error response")
|
||||
};
|
||||
|
||||
// Guard the id against path traversal before joining it into a path.
|
||||
let id = request.uri().path().trim_start_matches('/').to_string();
|
||||
if id.is_empty() || !id.chars().all(|c| c.is_ascii_alphanumeric() || c == '-') {
|
||||
return fail(StatusCode::BAD_REQUEST);
|
||||
}
|
||||
let raw = match std::fs::read(meeting_dir(&id).join("audio.wav")) {
|
||||
Ok(b) => b,
|
||||
Err(_) => return fail(StatusCode::NOT_FOUND),
|
||||
};
|
||||
let plain = match crate::vault::open(&raw) {
|
||||
Ok(p) => p,
|
||||
Err(_) => return fail(StatusCode::FORBIDDEN), // sealed + vault locked
|
||||
};
|
||||
let total = plain.len();
|
||||
|
||||
let base = || {
|
||||
Response::builder()
|
||||
.header(header::CONTENT_TYPE, "audio/wav")
|
||||
.header(header::ACCEPT_RANGES, "bytes")
|
||||
};
|
||||
// Honour a single `Range` request so the player can seek.
|
||||
if let Some(range) = request
|
||||
.headers()
|
||||
.get(header::RANGE)
|
||||
.and_then(|v| v.to_str().ok())
|
||||
{
|
||||
if let Some((start, end)) = parse_byte_range(range, total) {
|
||||
return base()
|
||||
.status(StatusCode::PARTIAL_CONTENT)
|
||||
.header(header::CONTENT_RANGE, format!("bytes {start}-{end}/{total}"))
|
||||
.header(header::CONTENT_LENGTH, (end - start + 1).to_string())
|
||||
.body(plain[start..=end].to_vec())
|
||||
.unwrap_or_else(|_| fail(StatusCode::INTERNAL_SERVER_ERROR));
|
||||
}
|
||||
}
|
||||
base()
|
||||
.status(StatusCode::OK)
|
||||
.header(header::CONTENT_LENGTH, total.to_string())
|
||||
.body(plain)
|
||||
.unwrap_or_else(|_| fail(StatusCode::INTERNAL_SERVER_ERROR))
|
||||
}
|
||||
|
||||
/// Parse a single `bytes=start-end` range against `total`, returning inclusive
|
||||
/// clamped indices. Supports open-ended (`start-`) and suffix (`-n`) forms; only
|
||||
/// the common single-range case is handled (enough for an `<audio>` element).
|
||||
fn parse_byte_range(header: &str, total: usize) -> Option<(usize, usize)> {
|
||||
if total == 0 {
|
||||
return None;
|
||||
}
|
||||
let (s, e) = header.strip_prefix("bytes=")?.split_once('-')?;
|
||||
if s.is_empty() {
|
||||
let n: usize = e.parse().ok()?;
|
||||
return Some((total.saturating_sub(n), total - 1));
|
||||
}
|
||||
let start: usize = s.parse().ok()?;
|
||||
let end = if e.is_empty() {
|
||||
total - 1
|
||||
} else {
|
||||
e.parse::<usize>().ok()?.min(total - 1)
|
||||
};
|
||||
(start <= end && start < total).then_some((start, end))
|
||||
}
|
||||
|
||||
/// Remove any leftover plaintext `audio.play.wav` files from the previous
|
||||
/// file-based player, so no decrypted audio lingers on disk (T8.8).
|
||||
pub(crate) fn cleanup_playback_temp() {
|
||||
let Ok(entries) = std::fs::read_dir(crate::paths::meetings_dir()) else {
|
||||
return;
|
||||
};
|
||||
for entry in entries.flatten() {
|
||||
let play = entry.path().join("audio.play.wav");
|
||||
if play.exists() {
|
||||
let _ = std::fs::remove_file(play);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[tauri::command]
|
||||
pub async fn update_notes(
|
||||
state: State<'_, AppState>,
|
||||
@@ -2762,6 +2959,7 @@ async fn upload_job(
|
||||
target: &dyn crate::sync::SyncTarget,
|
||||
row: &crate::storage::SyncJobRow,
|
||||
encrypt: bool,
|
||||
mut on_progress: impl FnMut(u64, u64) + Send + 'static,
|
||||
) -> Result<u64, crate::sync::SyncError> {
|
||||
use crate::sync::SyncError;
|
||||
let remote_dir = row
|
||||
@@ -2771,6 +2969,22 @@ async fn upload_job(
|
||||
.unwrap_or("");
|
||||
target.ensure_dir(remote_dir).await?;
|
||||
let (tx, rx) = std::sync::mpsc::channel();
|
||||
// Forward live upload progress off the async task (the sink is a blocking
|
||||
// std channel): throttled so a large recording emits a steady stream of
|
||||
// `sync://job` updates without flooding the UI. Ends when `put` drops `tx`.
|
||||
let fallback_total = row.bytes_total.unwrap_or(0) as u64;
|
||||
let drain = std::thread::spawn(move || {
|
||||
let mut last = 0u64;
|
||||
let mut last_emit = std::time::Instant::now() - std::time::Duration::from_millis(500);
|
||||
while let Ok((sent, total)) = rx.recv() {
|
||||
last = sent;
|
||||
if last_emit.elapsed() >= std::time::Duration::from_millis(250) {
|
||||
on_progress(sent, total);
|
||||
last_emit = std::time::Instant::now();
|
||||
}
|
||||
}
|
||||
last
|
||||
});
|
||||
if encrypt {
|
||||
// Client-side encryption before upload (T9.12, FR-SYNC-10): seal to a
|
||||
// temp file (idempotent if already sealed at rest) so the destination
|
||||
@@ -2792,12 +3006,10 @@ async fn upload_job(
|
||||
.put(Path::new(&row.local_path), &row.remote_path, tx)
|
||||
.await?;
|
||||
}
|
||||
let sent = rx
|
||||
.try_iter()
|
||||
.last()
|
||||
.map(|(s, _)| s)
|
||||
.unwrap_or(row.bytes_total.unwrap_or(0) as u64);
|
||||
Ok(sent)
|
||||
// `tx` is dropped now that `put` returned, so the drain thread finishes and
|
||||
// hands back the final byte count it observed.
|
||||
let sent = drain.join().unwrap_or(fallback_total);
|
||||
Ok(if sent > 0 { sent } else { fallback_total })
|
||||
}
|
||||
|
||||
/// Drive all due jobs once: upload each, emit `sync://job` on every transition,
|
||||
@@ -2822,8 +3034,16 @@ pub(crate) async fn pump_sync(app: &AppHandle, store: &dyn crate::storage::Store
|
||||
let _ = store.update_sync_job(job.clone()).await;
|
||||
emit_sync_job(app, &job);
|
||||
|
||||
// Emit a live `sync://job` (status still "uploading") as bytes go out, so
|
||||
// the UI shows a moving per-item progress bar (FR-SYNC-11).
|
||||
let app_prog = app.clone();
|
||||
let mut prog_row = job.clone();
|
||||
let on_progress = move |sent: u64, _total: u64| {
|
||||
prog_row.bytes_sent = sent as i64;
|
||||
emit_sync_job(&app_prog, &prog_row);
|
||||
};
|
||||
let outcome = match crate::sync::build_sync_target(&target) {
|
||||
Ok(t) => upload_job(t.as_ref(), &job, target.encrypt_before_upload).await,
|
||||
Ok(t) => upload_job(t.as_ref(), &job, target.encrypt_before_upload, on_progress).await,
|
||||
Err(e) => Err(e),
|
||||
};
|
||||
match outcome {
|
||||
|
||||
@@ -88,6 +88,11 @@ pub fn run() {
|
||||
|
||||
tauri::Builder::default()
|
||||
.plugin(tauri_plugin_dialog::init())
|
||||
// In-memory streaming of recordings for the player (FR-REC-5): decrypts
|
||||
// on the fly so no plaintext audio is ever written to disk.
|
||||
.register_uri_scheme_protocol("waaudio", |_ctx, request| {
|
||||
commands::serve_recording(&request)
|
||||
})
|
||||
.manage(AppState {
|
||||
store,
|
||||
session: Mutex::new(None),
|
||||
@@ -127,6 +132,10 @@ pub fn run() {
|
||||
// Startup recovery + retention + reminder-reconcile pass (FR-REL-1,
|
||||
// FR-STORE-2, FR-CAL-5). Spawned so it never blocks the window from
|
||||
// showing (NFR-PERF-4); nothing here repeats on a timer (NFR-RES-1).
|
||||
// Sweep away any plaintext playback temp files left by the previous
|
||||
// file-based player (T8.8) — playback is now in-memory only.
|
||||
commands::cleanup_playback_temp();
|
||||
|
||||
let store = store_for_setup;
|
||||
let startup_app = app.handle().clone();
|
||||
tauri::async_runtime::spawn(async move {
|
||||
@@ -187,6 +196,8 @@ pub fn run() {
|
||||
.invoke_handler(tauri::generate_handler![
|
||||
commands::start_recording,
|
||||
commands::stop_recording,
|
||||
commands::cancel_recording,
|
||||
commands::recording_playback_path,
|
||||
commands::pause_recording,
|
||||
commands::resume_recording,
|
||||
commands::set_recording_retention,
|
||||
|
||||
+29
-10
@@ -241,21 +241,42 @@ impl SyncTarget for WebDavTarget {
|
||||
let url = self.url_for(remote_path);
|
||||
let resp = self
|
||||
.request(reqwest::Method::PUT, &url)?
|
||||
.body(bytes)
|
||||
.header(reqwest::header::CONTENT_LENGTH, total)
|
||||
.body(progress_body(bytes, prog))
|
||||
.send()
|
||||
.await
|
||||
.map_err(|e| SyncError::Upload(e.to_string()))?;
|
||||
match resp.status().as_u16() {
|
||||
code if (200..300).contains(&code) => {
|
||||
let _ = prog.send((total, total));
|
||||
Ok(())
|
||||
}
|
||||
code if (200..300).contains(&code) => Ok(()),
|
||||
401 | 403 => Err(SyncError::Auth),
|
||||
other => Err(SyncError::Upload(format!("PUT {other}"))),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Wrap in-memory bytes as a reqwest streaming body that reports cumulative
|
||||
/// bytes-sent to `prog` as the request drains it — the basis for live per-item
|
||||
/// upload progress (FR-SYNC-11). Chunked so the UI sees steady movement on a
|
||||
/// large recording instead of a single 0→100% jump at the end.
|
||||
#[cfg(feature = "sync")]
|
||||
fn progress_body(bytes: Vec<u8>, prog: ProgressSink) -> reqwest::Body {
|
||||
const CHUNK: usize = 64 * 1024;
|
||||
let total = bytes.len() as u64;
|
||||
let stream = futures_util::stream::unfold((bytes, 0usize), move |(bytes, pos)| {
|
||||
let prog = prog.clone();
|
||||
async move {
|
||||
if pos >= bytes.len() {
|
||||
return None;
|
||||
}
|
||||
let end = (pos + CHUNK).min(bytes.len());
|
||||
let chunk = bytes[pos..end].to_vec();
|
||||
let _ = prog.send((end as u64, total));
|
||||
Some((Ok::<Vec<u8>, std::io::Error>(chunk), (bytes, end)))
|
||||
}
|
||||
});
|
||||
reqwest::Body::wrap_stream(stream)
|
||||
}
|
||||
|
||||
/// Shared HTTP client (rustls TLS via reqwest's `rustls-tls` feature).
|
||||
#[cfg(feature = "sync")]
|
||||
fn http_client() -> Result<reqwest::Client, SyncError> {
|
||||
@@ -667,15 +688,13 @@ impl SyncTarget for OneDriveTarget {
|
||||
let resp = http_client()?
|
||||
.put(Self::item_url(remote_path, ":/content"))
|
||||
.bearer_auth(token)
|
||||
.body(bytes)
|
||||
.header(reqwest::header::CONTENT_LENGTH, total)
|
||||
.body(progress_body(bytes, prog))
|
||||
.send()
|
||||
.await
|
||||
.map_err(|e| SyncError::Upload(e.to_string()))?;
|
||||
match resp.status().as_u16() {
|
||||
code if (200..300).contains(&code) => {
|
||||
let _ = prog.send((total, total));
|
||||
Ok(())
|
||||
}
|
||||
code if (200..300).contains(&code) => Ok(()),
|
||||
401 | 403 => Err(SyncError::Auth),
|
||||
other => Err(SyncError::Upload(format!("PUT {other}"))),
|
||||
}
|
||||
|
||||
@@ -216,6 +216,12 @@ pub fn seal(plaintext: &[u8]) -> Result<Vec<u8>, VaultError> {
|
||||
}
|
||||
}
|
||||
|
||||
/// True if `data` begins with the vault's sealed-file magic (i.e. it's
|
||||
/// ciphertext at rest, not a plaintext/pre-vault file).
|
||||
pub fn is_sealed(data: &[u8]) -> bool {
|
||||
data.len() >= MAGIC.len() && &data[..MAGIC.len()] == MAGIC
|
||||
}
|
||||
|
||||
/// Inverse of `seal`. Plaintext (no magic) passes through unchanged; sealed data
|
||||
/// requires the vault to be unlocked.
|
||||
pub fn open(data: &[u8]) -> Result<Vec<u8>, VaultError> {
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
{
|
||||
"$schema": "https://schema.tauri.app/config/2",
|
||||
"productName": "WhispAssist",
|
||||
"version": "0.1.6",
|
||||
"version": "0.2.0",
|
||||
"identifier": "bet.dou.whispassist",
|
||||
"build": {
|
||||
"frontendDist": "../dist",
|
||||
@@ -21,7 +21,7 @@
|
||||
}
|
||||
],
|
||||
"security": {
|
||||
"csp": "default-src 'self'; connect-src 'self' http://localhost:* http://127.0.0.1:*; img-src 'self' data:; style-src 'self' 'unsafe-inline'"
|
||||
"csp": "default-src 'self'; connect-src 'self' http://localhost:* http://127.0.0.1:*; img-src 'self' data:; media-src 'self' http://waaudio.localhost; style-src 'self' 'unsafe-inline'"
|
||||
},
|
||||
"trayIcon": {
|
||||
"iconPath": "icons/tray.png",
|
||||
|
||||
@@ -0,0 +1,6 @@
|
||||
{
|
||||
"$schema": "gen/schemas/desktop-schema.json",
|
||||
"bundle": {
|
||||
"resources": ["vulkan-1.dll"]
|
||||
}
|
||||
}
|
||||
+34
-1
@@ -14,7 +14,7 @@
|
||||
import { api, type NoteTemplate } from "./lib/api";
|
||||
import { onMount } from "svelte";
|
||||
import ThemeToggle from "./lib/components/ThemeToggle.svelte";
|
||||
import { Circle, Square, Settings as SettingsIcon, AlertTriangle } from "@lucide/svelte";
|
||||
import { Circle, Square, Trash2, Settings as SettingsIcon, AlertTriangle } from "@lucide/svelte";
|
||||
|
||||
let showSettings = $state(false);
|
||||
let showConsent = $state(false);
|
||||
@@ -99,6 +99,12 @@
|
||||
else recording.stop();
|
||||
}
|
||||
|
||||
async function cancelRecording() {
|
||||
if (!confirm("Discard this recording? Its audio and transcript will be deleted.")) return;
|
||||
await recording.cancel();
|
||||
meetings.deselect();
|
||||
}
|
||||
|
||||
// Global shortcuts (T7.4, FR-UX-3): record start/stop, view toggles.
|
||||
function isEditableTarget(target: EventTarget | null): boolean {
|
||||
if (!(target instanceof HTMLElement)) return false;
|
||||
@@ -199,6 +205,14 @@
|
||||
<Square size={11} fill="currentColor" aria-hidden="true" />
|
||||
Stop
|
||||
</button>
|
||||
<button
|
||||
class="cancel-btn"
|
||||
onclick={cancelRecording}
|
||||
title="Discard this recording and delete it"
|
||||
>
|
||||
<Trash2 size={12} aria-hidden="true" />
|
||||
Cancel
|
||||
</button>
|
||||
<span class="rec">
|
||||
<span class="rec-dot" aria-hidden="true"></span>
|
||||
Recording…
|
||||
@@ -461,6 +475,25 @@
|
||||
.stop-btn:hover {
|
||||
background: var(--border);
|
||||
}
|
||||
.cancel-btn {
|
||||
display: inline-flex;
|
||||
align-items: center;
|
||||
gap: 0.35rem;
|
||||
padding: 0.35rem 0.7rem;
|
||||
border-radius: var(--radius-full);
|
||||
background: transparent;
|
||||
color: var(--muted);
|
||||
border: 1px solid var(--border);
|
||||
font-size: 0.85rem;
|
||||
cursor: pointer;
|
||||
transition:
|
||||
color 150ms ease-out,
|
||||
border-color 150ms ease-out;
|
||||
}
|
||||
.cancel-btn:hover {
|
||||
color: var(--danger);
|
||||
border-color: var(--danger);
|
||||
}
|
||||
.rec {
|
||||
display: inline-flex;
|
||||
align-items: center;
|
||||
|
||||
@@ -339,6 +339,11 @@ export const api = {
|
||||
}),
|
||||
listNoteTemplates: () => invoke<NoteTemplate[]>("list_note_templates"),
|
||||
stopRecording: (meetingId: MeetingId) => invoke<void>("stop_recording", { meetingId }),
|
||||
// Abandon an accidental recording: stop + delete files + drop the DB row.
|
||||
cancelRecording: (meetingId: MeetingId) => invoke<void>("cancel_recording", { meetingId }),
|
||||
// Absolute path to a playable audio.wav (decrypted if sealed), for convertFileSrc.
|
||||
recordingPlaybackPath: (meetingId: MeetingId) =>
|
||||
invoke<string>("recording_playback_path", { meetingId }),
|
||||
pauseRecording: (meetingId: MeetingId) => invoke<void>("pause_recording", { meetingId }),
|
||||
resumeRecording: (meetingId: MeetingId) => invoke<void>("resume_recording", { meetingId }),
|
||||
setRecordingRetention: (meetingId: MeetingId, record: boolean) =>
|
||||
|
||||
@@ -18,10 +18,14 @@ class RecordingStore {
|
||||
|
||||
async init() {
|
||||
await events.onRecordingState((p) => {
|
||||
const e = p as { state: "recording" | "paused" | "stopped"; elapsedMs: number };
|
||||
this.state = e.state === "stopped" ? "idle" : e.state;
|
||||
const e = p as {
|
||||
state: "recording" | "paused" | "stopped" | "cancelled";
|
||||
elapsedMs: number;
|
||||
};
|
||||
const ended = e.state === "stopped" || e.state === "cancelled";
|
||||
this.state = ended ? "idle" : (e.state as "recording" | "paused");
|
||||
this.elapsedMs = e.elapsedMs ?? this.elapsedMs;
|
||||
if (e.state === "stopped") {
|
||||
if (ended) {
|
||||
this.levelRms = 0;
|
||||
this.levelPeak = 0;
|
||||
}
|
||||
@@ -60,6 +64,17 @@ class RecordingStore {
|
||||
this.deviceNotice = null;
|
||||
}
|
||||
|
||||
/** Abandon an accidental recording: stop capture and delete it entirely. */
|
||||
async cancel() {
|
||||
if (this.meetingId) await api.cancelRecording(this.meetingId);
|
||||
this.meetingId = null;
|
||||
this.segments = [];
|
||||
this.state = "idle";
|
||||
this.levelRms = 0;
|
||||
this.levelPeak = 0;
|
||||
this.deviceNotice = null;
|
||||
}
|
||||
|
||||
/** Toggle audio retention mid-meeting (FR-REC-1); caller must have gated consent already. */
|
||||
async setRetention(record: boolean) {
|
||||
if (!this.meetingId) return;
|
||||
|
||||
@@ -26,6 +26,26 @@
|
||||
|
||||
onMount(() => calendar.load());
|
||||
|
||||
// ---- Recording playback (FR-REC-5) ----
|
||||
let audioSrc = $state<string | null>(null);
|
||||
let audioError = $state<string | null>(null);
|
||||
$effect(() => {
|
||||
const m = meetings.selected;
|
||||
audioSrc = null;
|
||||
audioError = null;
|
||||
if (m?.recorded) {
|
||||
const id = m.id;
|
||||
api
|
||||
.recordingPlaybackPath(id)
|
||||
.then((url) => {
|
||||
if (meetings.selected?.id === id) audioSrc = url;
|
||||
})
|
||||
.catch((e) => {
|
||||
if (meetings.selected?.id === id) audioError = errorMessage(e);
|
||||
});
|
||||
}
|
||||
});
|
||||
|
||||
// ---- Summary + action items (T5.4/T5.5/T5.6, FR-LLM-2/3/4) ----
|
||||
let llmStatus = $state<LlmStatus | null>(null);
|
||||
onMount(async () => {
|
||||
@@ -233,6 +253,26 @@
|
||||
</script>
|
||||
|
||||
<div class="wrap">
|
||||
{#if meetings.selected?.recorded}
|
||||
<h3><Mic2 size={14} aria-hidden="true" /> Recording</h3>
|
||||
{#if audioSrc}
|
||||
<!-- svelte-ignore a11y_media_has_caption -- a meeting recording has no caption track -->
|
||||
<!-- controlsList/contextmenu: no download affordance — the decrypted audio
|
||||
must not be savable to disk (would undermine encryption at rest). -->
|
||||
<audio
|
||||
class="player"
|
||||
controls
|
||||
controlsList="nodownload noplaybackrate"
|
||||
oncontextmenu={(e) => e.preventDefault()}
|
||||
src={audioSrc}
|
||||
></audio>
|
||||
{:else if audioError}
|
||||
<p class="muted">{audioError}</p>
|
||||
{:else}
|
||||
<p class="muted">Loading recording…</p>
|
||||
{/if}
|
||||
{/if}
|
||||
|
||||
{#if meetings.selected && settings.settings.sync_enabled}
|
||||
<h3><UploadCloud size={14} aria-hidden="true" /> Sync</h3>
|
||||
<button
|
||||
@@ -252,6 +292,7 @@
|
||||
<span class="artifact">{job.artifact}</span>
|
||||
<span class="job-status {job.status}">{job.status}</span>
|
||||
{#if job.status === "uploading" && job.bytesTotal}
|
||||
<progress class="job-bar" max={job.bytesTotal} value={job.bytesSent}></progress>
|
||||
<span class="muted">{Math.round((100 * job.bytesSent) / job.bytesTotal)}%</span>
|
||||
{/if}
|
||||
{#if job.status === "failed"}
|
||||
@@ -683,6 +724,27 @@
|
||||
.artifact {
|
||||
min-width: 5rem;
|
||||
}
|
||||
.job-bar {
|
||||
flex: 1;
|
||||
height: 0.4rem;
|
||||
min-width: 3rem;
|
||||
border: none;
|
||||
border-radius: var(--radius-full);
|
||||
overflow: hidden;
|
||||
accent-color: var(--accent, #2563eb);
|
||||
}
|
||||
.job-bar::-webkit-progress-bar {
|
||||
background: var(--bg-hover);
|
||||
border-radius: var(--radius-full);
|
||||
}
|
||||
.job-bar::-webkit-progress-value {
|
||||
background: var(--accent, #2563eb);
|
||||
border-radius: var(--radius-full);
|
||||
}
|
||||
.player {
|
||||
width: 100%;
|
||||
margin: 0.2rem 0 0.6rem;
|
||||
}
|
||||
.job-status {
|
||||
text-transform: capitalize;
|
||||
color: var(--muted);
|
||||
|
||||
@@ -21,6 +21,8 @@
|
||||
RefreshCw,
|
||||
MessageSquareText,
|
||||
NotebookPen,
|
||||
Eye,
|
||||
Pencil,
|
||||
} from "@lucide/svelte";
|
||||
|
||||
function speakerName(label: string, speakers: SpeakerInfo[] = []): string {
|
||||
@@ -28,6 +30,9 @@
|
||||
}
|
||||
|
||||
let notesText = $state("");
|
||||
// Notes is a single pane: raw markdown ("Editor") or the rendered result
|
||||
// ("Preview"), toggled by one button whose label flips to the other mode.
|
||||
let notesPreview = $state(false);
|
||||
let editorEl: HTMLTextAreaElement | undefined = $state();
|
||||
let saveTimer: ReturnType<typeof setTimeout> | undefined;
|
||||
let loadedForId: string | null = null;
|
||||
@@ -201,6 +206,20 @@
|
||||
>
|
||||
<ListChecks size={14} aria-hidden="true" />
|
||||
</button>
|
||||
<button
|
||||
class="toggle"
|
||||
onclick={() => (notesPreview = !notesPreview)}
|
||||
title={notesPreview ? "Edit the raw markdown" : "Render the markdown"}
|
||||
aria-pressed={notesPreview}
|
||||
>
|
||||
{#if notesPreview}
|
||||
<Pencil size={14} aria-hidden="true" />
|
||||
Editor
|
||||
{:else}
|
||||
<Eye size={14} aria-hidden="true" />
|
||||
Preview
|
||||
{/if}
|
||||
</button>
|
||||
<span class="spacer"></span>
|
||||
<button onclick={exportMd} title="Export notes as .md">
|
||||
<FileText size={13} aria-hidden="true" />
|
||||
@@ -220,14 +239,17 @@
|
||||
</button>
|
||||
</div>
|
||||
<div class="editor-preview">
|
||||
<textarea
|
||||
bind:this={editorEl}
|
||||
bind:value={notesText}
|
||||
oninput={scheduleSave}
|
||||
placeholder="Notes…"
|
||||
></textarea>
|
||||
<!-- eslint-disable-next-line svelte/no-at-html-tags -- sanitized via renderMarkdown() -->
|
||||
<div class="preview">{@html renderMarkdown(notesText)}</div>
|
||||
{#if notesPreview}
|
||||
<!-- eslint-disable-next-line svelte/no-at-html-tags -- sanitized via renderMarkdown() -->
|
||||
<div class="preview">{@html renderMarkdown(notesText)}</div>
|
||||
{:else}
|
||||
<textarea
|
||||
bind:this={editorEl}
|
||||
bind:value={notesText}
|
||||
oninput={scheduleSave}
|
||||
placeholder="Notes…"
|
||||
></textarea>
|
||||
{/if}
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
@@ -366,11 +388,14 @@
|
||||
}
|
||||
|
||||
.editor-preview {
|
||||
display: grid;
|
||||
grid-template-columns: 1fr 1fr;
|
||||
gap: 0.75rem;
|
||||
height: calc(100% - 2.5rem);
|
||||
}
|
||||
.toolbar .toggle {
|
||||
font-weight: 600;
|
||||
}
|
||||
.toolbar .toggle[aria-pressed="true"] {
|
||||
background: var(--bg-hover);
|
||||
}
|
||||
textarea {
|
||||
resize: none;
|
||||
width: 100%;
|
||||
@@ -388,8 +413,12 @@
|
||||
border-color: var(--accent);
|
||||
}
|
||||
.preview {
|
||||
height: 100%;
|
||||
box-sizing: border-box;
|
||||
overflow: auto;
|
||||
padding: 0.25rem 0.5rem;
|
||||
padding: 0.6rem;
|
||||
border: 1px solid var(--border);
|
||||
border-radius: var(--radius-md);
|
||||
font-size: 0.9rem;
|
||||
line-height: 1.5;
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user