Feature again here #13
@@ -3,88 +3,152 @@
|
||||
**A privacy-first, Windows-native meeting assistant that runs entirely on-device.**
|
||||
|
||||
WhispAssist captures system audio, transcribes it locally with Whisper-class models using
|
||||
on-device acceleration (NPU → GPU → CPU), structures the result into Markdown notes, and
|
||||
optionally augments them with a locally hosted LLM (Ollama). Audio and transcripts **never
|
||||
leave the machine** unless the user explicitly exports them.
|
||||
on-device acceleration (**NPU → GPU → CPU**), labels speakers, structures the result into
|
||||
Markdown notes, and optionally augments them with a locally hosted LLM (Ollama). Audio and
|
||||
transcripts **never leave the machine** unless you explicitly configure a destination.
|
||||
|
||||
> Status: **planning + scaffold**. This repository currently contains the full engineering
|
||||
> plan (`docs/`) and a compiling-intent skeleton (`src-tauri/`, `src/`). No feature code is
|
||||
> implemented yet. See [`docs/05-roadmap.md`](docs/05-roadmap.md) for the build order.
|
||||
> **Status: working application (v0.1.4).** Capture, transcription (CPU / Intel NPU / Vulkan
|
||||
> GPU), speaker diarization, storage + crash recovery, local-LLM summaries, opt-in recording,
|
||||
> at-rest encryption, and self-hosted sync are implemented and ship as signed **MSI + NSIS**
|
||||
> installers. Outlook `.pst`/calendar context and the coding-agent (MCP) handoff are in
|
||||
> progress. Build order and remaining tasks are in [`docs/05-roadmap.md`](docs/05-roadmap.md).
|
||||
|
||||
## Why WhispAssist
|
||||
## Why WhispAssist — Granola vs Meetily vs WhispAssist
|
||||
|
||||
| | Granola | Meetily | **WhispAssist** |
|
||||
| | **Granola** | **Meetily** | **WhispAssist** |
|
||||
|---|---|---|---|
|
||||
| Local transcription | ❌ cloud | ✅ | ✅ |
|
||||
| Notes/summaries stay local | ❌ cloud AI | ⚠️ optional | ✅ local-only by design |
|
||||
| NPU/GPU auto-acceleration | n/a | partial | ✅ NPU→GPU→CPU ladder |
|
||||
| Calendar + Outlook `.pst` context | ✅ (cloud) | ❌ | ✅ local |
|
||||
| Bot-free system-audio capture | ✅ | ✅ | ✅ |
|
||||
| License | proprietary | MIT | open source |
|
||||
| **Positioning** | Cloud AI notepad | Open-source self-hosted assistant | Local, Windows-native assistant |
|
||||
| **Platform** | macOS, Windows | macOS, Windows, Linux | Windows 10/11 |
|
||||
| **Bot-free system-audio capture** | ✅ | ✅ | ✅ WASAPI loopback |
|
||||
| **Transcription** | ☁️ cloud | ✅ local (Whisper) | ✅ local (whisper.cpp) |
|
||||
| **NPU / GPU auto-acceleration** | n/a (cloud) | ⚠️ CPU/GPU, manual | ✅ **NPU→NVIDIA→AMD→Intel→CPU** ladder, one binary |
|
||||
| **Speaker diarization** | ✅ cloud | ⚠️ limited | ✅ offline (sherpa-onnx) |
|
||||
| **Summaries / notes AI** | ☁️ cloud LLM | ✅ local (Ollama) / BYO | ✅ local (Ollama, localhost **or LAN**) + optional hosted |
|
||||
| **Default data egress** | ☁️ audio + notes to cloud | 🔒 local (cloud optional) | 🔒 **none** — everything off by default, allowlist-enforced |
|
||||
| **Calendar / Outlook `.pst` context** | ✅ cloud calendar | ❌ | ⚙️ local `.pst` — in progress |
|
||||
| **Opt-in recording + consent notice** | ⚠️ partial | ❌ | ✅ off by default, one-time consent |
|
||||
| **At-rest encryption** | ☁️ server-side | ❌ | ✅ vault: Argon2id + XChaCha20-Poly1305 |
|
||||
| **Self-hosted sync w/ client-side encryption** | ❌ | ⚠️ | ✅ WebDAV + OAuth, **encrypt-before-upload** |
|
||||
| **Coding-agent (MCP) handoff** | ❌ | ❌ | ⚙️ local MCP server — in progress |
|
||||
| **License** | Proprietary | Open source (MIT) | Open source (MIT / Apache-2.0) |
|
||||
| **Cost** | Subscription | Free | Free |
|
||||
|
||||
<sub>Comparison reflects each project's public positioning as of mid-2026. Granola and Meetily
|
||||
are independent products and their capabilities evolve — verify current details before relying
|
||||
on any row.</sub>
|
||||
|
||||
**The short version:** Granola is the polished cloud option (your audio and notes are processed
|
||||
on their servers). Meetily is the closest peer — open-source and self-hosted — but is
|
||||
cross-platform-generic and leans on manual setup. WhispAssist is the **Windows-native, hardware-
|
||||
accelerated, zero-egress-by-default** option: it exploits the NPU/GPU in modern laptops, keeps
|
||||
everything on the device unless you opt in, and adds Windows-specific context (Outlook) and a
|
||||
coding-agent handoff.
|
||||
|
||||
## What's built (v0.1.4)
|
||||
|
||||
- **Bot-free capture** — WASAPI loopback records the system mix (all participants) with no
|
||||
meeting bot and no per-app plumbing.
|
||||
- **Local transcription with a hardware ladder** — whisper.cpp via `whisper-rs` on CPU; the
|
||||
**Intel NPU** via ONNX Runtime + OpenVINO; **GPU via Vulkan** (a single binary that runs on
|
||||
NVIDIA, AMD, and Intel). WA detects the hardware, picks the best backend
|
||||
(**NPU → NVIDIA → AMD → Intel → CPU**), streams partial transcripts live, and shows the active
|
||||
backend in the UI.
|
||||
- **Speaker diarization** — `sherpa-onnx` (pyannote segmentation + speaker-embedding
|
||||
clustering), fully offline.
|
||||
- **Notes & summaries** — Markdown notes; local-LLM summaries via **Ollama** on `localhost`
|
||||
**or a private LAN endpoint** (RFC-1918), with a full advanced-parameter panel (system prompt,
|
||||
`think`, `keep_alive`, `num_ctx`, sampling/repetition/mirostat, etc.).
|
||||
- **Storage & crash recovery** — SQLite + on-disk audio/transcripts under
|
||||
`%LOCALAPPDATA%\WhispAssist`. Audio is the source of truth; notes and transcripts regenerate
|
||||
after a crash.
|
||||
- **Opt-in recording** — off by default; `.wav` retained only when you turn it on, after a
|
||||
one-time consent notice.
|
||||
- **At-rest encryption vault** — Argon2id key derivation + XChaCha20-Poly1305; transcripts,
|
||||
notes, summaries, and recordings sealed on disk; startup unlock gate; keys zeroized on lock.
|
||||
- **Self-hosted sync (optional, off by default)** — WebDAV (Nextcloud, ownCloud, Cloudreve,
|
||||
Seafile, Synology) plus OneDrive/Dropbox/Box (OAuth 2.0 PKCE); durable retry queue with
|
||||
backoff; **client-side encryption before upload** so the destination holds only ciphertext.
|
||||
Credentials live only in the OS credential store.
|
||||
- **Optional hosted AI** — Anthropic and OpenAI-compatible providers behind the same
|
||||
`LlmProvider` interface, off by default (third-party egress, keys in the OS credential store).
|
||||
- **Installers** — signed MSI and NSIS `-setup.exe`.
|
||||
|
||||
**In progress:** Outlook `.pst` + calendar context, the local **MCP server** that hands meeting
|
||||
context to your coding agents (Claude, Codex, Copilot, OpenCode), and MS Graph calendar.
|
||||
|
||||
## Technology
|
||||
|
||||
WhispAssist is built as a **Tauri 2** application: a small Rust core with a compiled
|
||||
**Svelte** web frontend rendered through the OS WebView2 (no bundled browser → low idle
|
||||
memory). The choice and its alternatives are recorded in [`docs/adr/`](docs/adr/).
|
||||
WhispAssist is a **Tauri 2** application: a small Rust core with a compiled **Svelte + TypeScript**
|
||||
frontend rendered through the OS WebView2 (no bundled browser → low idle memory). Every decision
|
||||
and its alternatives are recorded in [`docs/adr/`](docs/adr/) (ADR-0001–0011).
|
||||
|
||||
- **Shell / IPC:** Tauri 2 (Rust ⇄ WebView2)
|
||||
- **Audio capture:** WASAPI loopback (`wasapi` crate)
|
||||
- **Transcription:** `whisper-rs` (whisper.cpp: CPU/Vulkan/CUDA) + ONNX Runtime (`ort`) DirectML for the NPU path
|
||||
- **Diarization:** `sherpa-onnx` (pyannote segmentation + speaker-embedding clustering), fully offline
|
||||
- **Storage:** SQLite (`sqlx`/`rusqlite`) + on-disk audio/transcript files
|
||||
- **Local LLM:** Ollama HTTP API on `localhost:11434`
|
||||
- **Calendar / Outlook:** `outlook-pst` crate for `.pst`, OS notifications for reminders
|
||||
- **Optional recording:** opt-in (default off), saved as `.wav`, with a consent reminder (ADR-0009)
|
||||
- **Optional sync:** upload artifacts to your own server — WebDAV covers **Nextcloud, ownCloud, Cloudreve, Seafile, Synology**; **OneDrive/Dropbox/Box** via OAuth. Off by default (ADR-0010)
|
||||
- **Optional AI/agent integration:** hosted summary providers (Anthropic, OpenAI-compatible) behind the same provider model, **and** a local **MCP server** so your own coding agents (**Claude, Codex, Copilot, OpenCode, …**) can pull meeting context and "feature briefs" to start coding. Off by default; the MCP server is inbound/loopback only (ADR-0011)
|
||||
- **Audio capture:** WASAPI loopback
|
||||
- **Transcription:** `whisper-rs` (whisper.cpp: CPU / **Vulkan** / CUDA) + ONNX Runtime (`ort`)
|
||||
with the **OpenVINO** execution provider for the Intel NPU path
|
||||
- **Diarization:** `sherpa-onnx`, fully offline
|
||||
- **Storage:** SQLite + on-disk audio/transcript files
|
||||
- **Local LLM:** Ollama HTTP API (localhost or a private LAN endpoint)
|
||||
- **Encryption:** Argon2id + XChaCha20-Poly1305 envelope vault; secrets in the OS credential store
|
||||
- **Calendar / Outlook:** `outlook-pst` for `.pst`, OS notifications for reminders *(in progress)*
|
||||
- **Sync:** WebDAV primary set + OneDrive/Dropbox/Box via OAuth — off by default (ADR-0010)
|
||||
- **External AI / agents:** hosted providers behind `LlmProvider`; a loopback-only **MCP server**
|
||||
for coding-agent handoff — off by default (ADR-0011)
|
||||
|
||||
## Repository layout
|
||||
|
||||
```
|
||||
WhispAssist/
|
||||
├── docs/ # The engineering plan (read this first)
|
||||
│ ├── 00-overview.md Vision, goals, glossary
|
||||
│ ├── 01-requirements.md Functional + non-functional requirements (traceable IDs)
|
||||
│ ├── 02-architecture.md Components, data flow, threading model
|
||||
│ ├── 03-data-model.md SQLite schema, file layout, transcript JSON
|
||||
│ ├── 04-api-contracts.md Tauri commands/events + internal Rust service traits
|
||||
│ ├── 05-roadmap.md 8 phases, task breakdown, acceptance criteria
|
||||
│ ├── 06-test-strategy.md Test plan per phase + quality gates
|
||||
│ ├── 07-research-findings.md Validated stack with sources
|
||||
│ └── adr/ Architecture Decision Records (0001–0010)
|
||||
├── src-tauri/ # Rust core (service module skeletons)
|
||||
├── src/ # Svelte frontend skeleton
|
||||
├── scripts/ # Dev/model-download helper scripts
|
||||
└── tests/ # Test fixtures + integration test scaffolding
|
||||
│ ├── 00-overview.md … 07-research-findings.md
|
||||
│ └── adr/ Architecture Decision Records (0001–0011)
|
||||
├── src-tauri/ # Rust core — implemented service modules:
|
||||
│ └── src/{audio,transcription,diarization,storage,llm,calendar,
|
||||
│ hardware,notes,sync,mcp,vault}
|
||||
├── src/ # Svelte + TypeScript frontend (views, stores, API bindings)
|
||||
├── packaging/ # NPU/OpenVINO runtime bundle + release assets
|
||||
├── scripts/ # Dev / model-download helpers
|
||||
└── tests/ # Fixtures + cross-service integration tests
|
||||
```
|
||||
|
||||
## Getting started (for builders)
|
||||
|
||||
Prerequisites once implementation begins: Rust (stable), Node.js 20+, the Tauri CLI, and
|
||||
WebView2 runtime (preinstalled on Windows 11). Then:
|
||||
Prerequisites: **Rust** (stable), **Node.js 20+**, the **Tauri CLI**, and the WebView2 runtime
|
||||
(preinstalled on Windows 11). Native builds also need the **VS 2022 Build Tools** (load
|
||||
`vcvars64.bat` first).
|
||||
|
||||
```bash
|
||||
npm install
|
||||
npm run tauri dev # once src-tauri/Cargo.toml dependencies are filled in
|
||||
npm run tauri dev # CPU/NPU build
|
||||
```
|
||||
|
||||
The current skeleton intentionally does **not** compile end-to-end — modules contain typed
|
||||
stubs and `todo!()` markers that map 1:1 to roadmap tasks. Start at Phase 1 in
|
||||
[`docs/05-roadmap.md`](docs/05-roadmap.md).
|
||||
**GPU (Vulkan) build.** whisper.cpp's GPU backends are compiled in (not downloaded at runtime),
|
||||
so a GPU build needs a one-time toolchain setup — the **Vulkan SDK**, a **Ninja** generator, and
|
||||
a short target dir (to dodge Windows' 260-char path limit in the shader build):
|
||||
|
||||
```bash
|
||||
# after: Vulkan SDK installed, ninja.exe on PATH, vcvars64 loaded
|
||||
set VULKAN_SDK=C:\VulkanSDK\1.4.350.0
|
||||
set CMAKE_GENERATOR=Ninja
|
||||
set CARGO_TARGET_DIR=C:\wt
|
||||
npm run tauri build -- --features vulkan
|
||||
```
|
||||
|
||||
CUDA (NVIDIA-only, faster) is planned as an optional variant. The full, gotcha-annotated build
|
||||
recipe lives in the project notes.
|
||||
|
||||
## Privacy guarantee
|
||||
|
||||
WhispAssist originates **no outbound connection for audio or transcript content** except to
|
||||
destinations **you explicitly configure** — an LLM endpoint (local Ollama by default, or a hosted AI
|
||||
provider if you choose one) and any sync targets you enable — plus explicit model downloads.
|
||||
destinations **you explicitly configure** — an LLM endpoint (local Ollama by default, or a hosted
|
||||
AI provider if you choose one) and any sync targets you enable — plus explicit model downloads.
|
||||
Everything optional is **off by default**; with nothing configured, WA makes no content egress at
|
||||
all. The local **MCP server** (for handing meetings to your coding agents) is **inbound on loopback
|
||||
and adds no egress** — data only leaves via the agent's own provider, which WA discloses. The set of
|
||||
reachable hosts is an allowlist derived from your settings and enforced in the core (and verified by
|
||||
a CI network test). Recording is opt-in; sync/AI credentials live in the OS credential store, never
|
||||
in config files. See the privacy requirements (`FR-SEC-*`, `NFR-SEC-*`, `FR-SYNC-*`, `FR-MCP-*`) in
|
||||
[`docs/01-requirements.md`](docs/01-requirements.md) and ADRs 0009–0011.
|
||||
all. The local **MCP server** (for handing meetings to your coding agents) is **inbound on
|
||||
loopback and adds no egress** — data only leaves via the agent's own provider, which WA discloses.
|
||||
The set of reachable hosts is an allowlist derived from your settings and enforced in the core
|
||||
(and verified by a CI network test). Recording is opt-in; sync/AI credentials live in the OS
|
||||
credential store, never in config files. See the privacy requirements (`FR-SEC-*`, `NFR-SEC-*`,
|
||||
`FR-SYNC-*`, `FR-MCP-*`) in [`docs/01-requirements.md`](docs/01-requirements.md) and ADRs 0009–0011.
|
||||
|
||||
## License
|
||||
|
||||
|
||||
+1
-1
@@ -1,7 +1,7 @@
|
||||
{
|
||||
"name": "whispassist",
|
||||
"private": true,
|
||||
"version": "0.1.2",
|
||||
"version": "0.1.4",
|
||||
"type": "module",
|
||||
"description": "Privacy-first, fully local Windows meeting assistant.",
|
||||
"license": "MIT OR Apache-2.0",
|
||||
|
||||
Generated
+1
-1
@@ -5945,7 +5945,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "whispassist"
|
||||
version = "0.1.1"
|
||||
version = "0.1.4"
|
||||
dependencies = [
|
||||
"argon2",
|
||||
"async-trait",
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
[package]
|
||||
name = "whispassist"
|
||||
version = "0.1.2"
|
||||
version = "0.1.4"
|
||||
description = "Privacy-first, fully local Windows meeting assistant"
|
||||
authors = ["WhispAssist contributors"]
|
||||
license = "MIT OR Apache-2.0"
|
||||
@@ -36,9 +36,9 @@ reqwest = { version = "0.12", default-features = false, features = ["json", "str
|
||||
futures-util = "0.3"
|
||||
sha2 = "0.10" # skip-if-unchanged hashing (sync)
|
||||
|
||||
# At-rest encryption vault (Phase 8, T8.8, FR-SEC-3). Pure-Rust RustCrypto — no
|
||||
# At-rest encryption vault (Phase 8, T8.8, FR-SEC-3). Pure-Rust RustCrypto — no
|
||||
# C build. Argon2id derives the vault key from the password; XChaCha20-Poly1305
|
||||
# (24-byte nonce → safe random nonces) is the AEAD for artifacts + key wrapping.
|
||||
# (24-byte nonce → safe random nonces) is the AEAD for artifacts + key wrapping.
|
||||
argon2 = "0.5"
|
||||
chacha20poly1305 = "0.10"
|
||||
getrandom = "0.2"
|
||||
@@ -52,7 +52,7 @@ hound = { version = "3", optional = true } # WAV I/O (Phase 1)
|
||||
whisper-rs = { version = "0.16", optional = true } # whisper.cpp bindings (Phase 1)
|
||||
|
||||
# NPU transcription (Phase 3, T3.4): ONNX Runtime + OpenVINO EP. `load-dynamic`
|
||||
# dlopens the on-demand-downloaded runtime, so cargo compiles no C++/OpenVINO —
|
||||
# dlopens the on-demand-downloaded runtime, so cargo compiles no C++/OpenVINO —
|
||||
# keeps the CPU-only build untouched (NFR-MNT-4). `rustfft` powers the log-mel
|
||||
# front-end (detok is hand-rolled off serde_json, no `tokenizers`/C dep).
|
||||
ort = { version = "=2.0.0-rc.10", optional = true, default-features = false, features = ["load-dynamic", "openvino"] }
|
||||
@@ -60,7 +60,7 @@ rustfft = { version = "6", optional = true }
|
||||
sherpa-rs = { version = "0.6", optional = true, default-features = false, features = ["download-binaries"] } # sherpa-onnx bindings (Phase 4, ADR-0005)
|
||||
tauri-plugin-dialog = "2" # native Save/choose-folder (Phase 2 export)
|
||||
|
||||
# notes export (Phase 8, FR-NOTE-4) — pure-Rust, no external binary/cloud
|
||||
# notes export (Phase 8, FR-NOTE-4) — pure-Rust, no external binary/cloud
|
||||
# conversion service, consistent with the fully-local invariant.
|
||||
pulldown-cmark = "0.12"
|
||||
printpdf = "0.7"
|
||||
@@ -74,7 +74,7 @@ windows = { version = "0.58", features = [
|
||||
"Win32_Devices_DeviceAndDriverInstallation", # SetupAPI: NPU detection (Phase 3, T3.4)
|
||||
"Win32_System_Com",
|
||||
"Win32_UI_Shell", # SetCurrentProcessExplicitAppUserModelID (Phase 8, T8.6)
|
||||
"UI_Notifications", # scheduled toast reminders (Phase 8, T8.6, FR-CAL-5) — the
|
||||
"UI_Notifications", # scheduled toast reminders (Phase 8, T8.6, FR-CAL-5) — the
|
||||
"Data_Xml_Dom", # OS delivers these itself at the due time, no polling timer
|
||||
"Foundation",
|
||||
"Foundation_Collections", # IVectorView iteration (GetScheduledToastNotifications)
|
||||
@@ -96,14 +96,14 @@ vulkan = ["whisper-rs?/vulkan"] # whisper.cpp Vulkan
|
||||
npu = ["dep:ort", "dep:rustfft"] # ort + OpenVINO EP NPU path (T3.4)
|
||||
# Phase 4 / 6 (added when integrated)
|
||||
diarization = ["dep:sherpa-rs"] # sherpa-onnx
|
||||
pst = [] # shells out to readpst (libpst) — no crate dep, see ADR-0008 update
|
||||
pst = [] # shells out to readpst (libpst) — no crate dep, see ADR-0008 update
|
||||
# Phase 9
|
||||
sync = ["dep:keyring"] # remote upload (WebDAV + OAuth providers)
|
||||
# Phase 10
|
||||
mcp = ["dep:rmcp", "dep:keyring"] # WhispAssist as an MCP server + hosted-AI creds
|
||||
|
||||
[profile.release]
|
||||
opt-level = "z" # optimize for size — keep the binary small (NFR-RES-1)
|
||||
opt-level = "z" # optimize for size — keep the binary small (NFR-RES-1)
|
||||
lto = true
|
||||
codegen-units = 1
|
||||
strip = true
|
||||
|
||||
@@ -51,6 +51,7 @@ fn default_settings() -> Settings {
|
||||
llm_provider: "ollama".into(),
|
||||
llm_endpoint: "http://localhost:11434".into(),
|
||||
llm_model: "llama3".into(),
|
||||
llm_advanced: serde_json::Value::Null,
|
||||
preferred_backend: "auto".into(),
|
||||
whisper_model: crate::paths::DEFAULT_WHISPER_MODEL.to_string(),
|
||||
low_overhead: false,
|
||||
@@ -1550,6 +1551,7 @@ fn llm_provider_from_settings(settings: &Settings) -> Option<Box<dyn crate::llm:
|
||||
"ollama" => Some(Box::new(crate::llm::OllamaProvider {
|
||||
endpoint: settings.llm_endpoint.clone(),
|
||||
model: settings.llm_model.clone(),
|
||||
advanced: settings.llm_advanced.clone(),
|
||||
})),
|
||||
"custom" => Some(Box::new(crate::llm::OpenAiCompatProvider {
|
||||
endpoint: settings.llm_endpoint.clone(),
|
||||
|
||||
@@ -57,7 +57,7 @@ const RESPONSE_FORMAT_INSTRUCTIONS: &str = "Respond in Markdown with exactly thr
|
||||
are bullet lists (each line starting with \"- \"); write \"- None\" if a section has nothing \
|
||||
to report. Do not add any other top-level sections.";
|
||||
|
||||
fn build_messages(prompt: &Prompt) -> Vec<serde_json::Value> {
|
||||
fn build_messages(prompt: &Prompt, user_system: Option<&str>) -> Vec<serde_json::Value> {
|
||||
let mut user = String::new();
|
||||
if !prompt.metadata.is_empty() {
|
||||
user.push_str(&prompt.metadata);
|
||||
@@ -69,8 +69,19 @@ fn build_messages(prompt: &Prompt) -> Vec<serde_json::Value> {
|
||||
}
|
||||
user.push_str("Transcript:\n");
|
||||
user.push_str(&prompt.transcript);
|
||||
|
||||
// A user's custom system prompt (Ollama advanced config) is *combined with*,
|
||||
// not a replacement for, WA's output-format contract — otherwise summary /
|
||||
// action-item parsing would break.
|
||||
let mut system = String::new();
|
||||
if let Some(us) = user_system.map(str::trim).filter(|s| !s.is_empty()) {
|
||||
system.push_str(us);
|
||||
system.push_str("\n\n");
|
||||
}
|
||||
system.push_str(RESPONSE_FORMAT_INSTRUCTIONS);
|
||||
|
||||
vec![
|
||||
serde_json::json!({ "role": "system", "content": RESPONSE_FORMAT_INSTRUCTIONS }),
|
||||
serde_json::json!({ "role": "system", "content": system }),
|
||||
serde_json::json!({ "role": "user", "content": user }),
|
||||
]
|
||||
}
|
||||
@@ -218,6 +229,9 @@ async fn stream_lines(
|
||||
pub struct OllamaProvider {
|
||||
pub endpoint: String, // e.g. http://localhost:11434
|
||||
pub model: String,
|
||||
/// Advanced tuning (`Settings.llm_advanced`): `{ system?, think?, keep_alive?,
|
||||
/// options: {…} }`. Sparse — only non-default overrides. `Null` = all defaults.
|
||||
pub advanced: serde_json::Value,
|
||||
}
|
||||
|
||||
#[derive(Deserialize)]
|
||||
@@ -278,11 +292,33 @@ impl LlmProvider for OllamaProvider {
|
||||
}
|
||||
|
||||
async fn summarize(&self, prompt: Prompt, out: TokenSink) -> Result<Summary, LlmError> {
|
||||
let body = serde_json::json!({
|
||||
let user_system = self.advanced.get("system").and_then(|v| v.as_str());
|
||||
let mut body = serde_json::json!({
|
||||
"model": self.model,
|
||||
"messages": build_messages(&prompt),
|
||||
"messages": build_messages(&prompt, user_system),
|
||||
"stream": true,
|
||||
});
|
||||
// Fold in the user's advanced tuning. `advanced` already holds only
|
||||
// non-default values, so anything present is an intentional override.
|
||||
if let Some(adv) = self.advanced.as_object() {
|
||||
if let Some(think) = adv.get("think").filter(|v| !v.is_null()) {
|
||||
body["think"] = think.clone();
|
||||
}
|
||||
if let Some(keep_alive) = adv
|
||||
.get("keep_alive")
|
||||
.and_then(|v| v.as_str())
|
||||
.filter(|s| !s.trim().is_empty())
|
||||
{
|
||||
body["keep_alive"] = serde_json::json!(keep_alive);
|
||||
}
|
||||
if let Some(options) = adv
|
||||
.get("options")
|
||||
.and_then(|v| v.as_object())
|
||||
.filter(|o| !o.is_empty())
|
||||
{
|
||||
body["options"] = serde_json::Value::Object(options.clone());
|
||||
}
|
||||
}
|
||||
let resp = reqwest::Client::new()
|
||||
.post(format!("{}/api/chat", self.base()))
|
||||
.json(&body)
|
||||
@@ -409,7 +445,9 @@ impl LlmProvider for OpenAiCompatProvider {
|
||||
async fn summarize(&self, prompt: Prompt, out: TokenSink) -> Result<Summary, LlmError> {
|
||||
let body = serde_json::json!({
|
||||
"model": self.model,
|
||||
"messages": build_messages(&prompt),
|
||||
// Advanced tuning is Ollama-only; the OpenAI-compatible path uses no
|
||||
// custom system prompt (its own server/model config governs that).
|
||||
"messages": build_messages(&prompt, None),
|
||||
"stream": true,
|
||||
});
|
||||
let req = self.auth(
|
||||
|
||||
@@ -192,6 +192,10 @@ pub struct Settings {
|
||||
pub llm_provider: String, // ollama|custom|off
|
||||
pub llm_endpoint: String,
|
||||
pub llm_model: String,
|
||||
/// Advanced Ollama tuning (T5.2): `{ system?, think?, keep_alive?, options: {…} }`.
|
||||
/// Sparse — only user-overridden values are stored; `Null`/absent = all defaults.
|
||||
#[serde(default)]
|
||||
pub llm_advanced: serde_json::Value,
|
||||
pub preferred_backend: String, // auto|npu|nvidia|amd|intel|cpu
|
||||
pub whisper_model: String, // ModelInfo.id, e.g. "base.en-q5_1"
|
||||
pub low_overhead: bool,
|
||||
|
||||
@@ -296,4 +296,37 @@ mod tests {
|
||||
fn audio_ctx_is_capped_at_whispers_own_maximum() {
|
||||
assert_eq!(audio_ctx_for_window(60 * 16_000), 1500); // 60s window
|
||||
}
|
||||
|
||||
/// GPU spike (opt-in): times whisper.cpp on a chosen backend against a real
|
||||
/// model + wav. With `--features vulkan` and `WA_BACKEND=intel` (or nvidia/amd)
|
||||
/// whisper.cpp offloads to the GPU; `WA_BACKEND=cpu` is the baseline. Run:
|
||||
/// WA_WHISPER_MODEL=…ggml-base.en-q5_1.bin WA_TEST_WAV=…tts_test.wav \
|
||||
/// cargo test --features vulkan gpu_transcribes -- --ignored --nocapture
|
||||
#[test]
|
||||
#[ignore = "requires a whisper model + wav; GPU/CPU timing spike"]
|
||||
fn gpu_transcribes_and_times() {
|
||||
let model = std::env::var("WA_WHISPER_MODEL").expect("set WA_WHISPER_MODEL");
|
||||
let wav = std::env::var("WA_TEST_WAV").expect("set WA_TEST_WAV");
|
||||
let backend = match std::env::var("WA_BACKEND").as_deref() {
|
||||
Ok("cpu") => BackendId::Cpu,
|
||||
Ok("nvidia") => BackendId::Nvidia,
|
||||
Ok("amd") => BackendId::Amd,
|
||||
_ => BackendId::Intel, // use_gpu = true for any non-CPU backend
|
||||
};
|
||||
let t0 = std::time::Instant::now();
|
||||
let transcriber = WhisperTranscriber::load(Path::new(&model), backend).expect("load");
|
||||
let load_ms = t0.elapsed().as_millis();
|
||||
let t1 = std::time::Instant::now();
|
||||
let segments = transcriber
|
||||
.transcribe_file(Path::new(&wav))
|
||||
.expect("transcribe");
|
||||
let infer_ms = t1.elapsed().as_millis();
|
||||
let text = segments
|
||||
.iter()
|
||||
.map(|s| s.text.as_str())
|
||||
.collect::<Vec<_>>()
|
||||
.join(" ");
|
||||
eprintln!("[spike] backend={backend:?} load={load_ms}ms infer={infer_ms}ms text={text:?}");
|
||||
assert!(!text.trim().is_empty(), "transcript was empty");
|
||||
}
|
||||
}
|
||||
|
||||
@@ -433,16 +433,20 @@ mod tests {
|
||||
return;
|
||||
};
|
||||
let wav = std::env::var("WA_NPU_TEST_WAV").expect("set WA_NPU_TEST_WAV");
|
||||
let t0 = std::time::Instant::now();
|
||||
let t = OnnxNpuTranscriber::load(Path::new(&model_dir), BackendId::Npu)
|
||||
.expect("load NPU transcriber");
|
||||
let load_ms = t0.elapsed().as_millis();
|
||||
let t1 = std::time::Instant::now();
|
||||
let segs = t.transcribe_file(Path::new(&wav)).expect("transcribe");
|
||||
let infer_ms = t1.elapsed().as_millis();
|
||||
let text = segs
|
||||
.iter()
|
||||
.map(|s| s.text.as_str())
|
||||
.collect::<Vec<_>>()
|
||||
.join(" ")
|
||||
.to_lowercase();
|
||||
eprintln!("NPU transcript: {text:?}");
|
||||
eprintln!("[spike] backend=Npu load={load_ms}ms infer={infer_ms}ms text={text:?}");
|
||||
assert!(!text.trim().is_empty(), "transcript was empty");
|
||||
if let Ok(expect) = std::env::var("WA_NPU_EXPECT") {
|
||||
assert!(
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
{
|
||||
"$schema": "https://schema.tauri.app/config/2",
|
||||
"productName": "WhispAssist",
|
||||
"version": "0.1.2",
|
||||
"version": "0.1.4",
|
||||
"identifier": "bet.dou.whispassist",
|
||||
"build": {
|
||||
"frontendDist": "../dist",
|
||||
|
||||
@@ -253,6 +253,13 @@ export interface AppSettings {
|
||||
llm_provider: string;
|
||||
llm_endpoint: string;
|
||||
llm_model: string;
|
||||
/** Advanced Ollama tuning; sparse (only non-default overrides). */
|
||||
llm_advanced?: {
|
||||
system?: string;
|
||||
think?: string;
|
||||
keep_alive?: string;
|
||||
options?: Record<string, number | boolean | string[]>;
|
||||
} | null;
|
||||
preferred_backend: string;
|
||||
whisper_model: string;
|
||||
low_overhead: boolean;
|
||||
|
||||
@@ -0,0 +1,317 @@
|
||||
// Ollama advanced `options` parameters (T5.2) — the single source of truth that
|
||||
// drives the Settings ▸ AI advanced form: rendering, defaults, per-field reset,
|
||||
// and only-send-if-changed. System prompt / think / keep_alive are top-level
|
||||
// request fields handled separately in the UI; everything here goes in `options`.
|
||||
//
|
||||
// Defaults + ranges per the Ollama Modelfile/API reference.
|
||||
|
||||
export type OllamaGroup = "sampling" | "repetition" | "mirostat" | "context" | "hardware";
|
||||
|
||||
export interface OllamaOption {
|
||||
key: string;
|
||||
group: OllamaGroup;
|
||||
label: string;
|
||||
kind: "number" | "bool" | "enum" | "tags";
|
||||
/** Undefined = "auto" (no fixed default): the field is empty until set. */
|
||||
default?: number | boolean;
|
||||
min?: number;
|
||||
max?: number;
|
||||
step?: number;
|
||||
enumOptions?: { value: number; label: string }[];
|
||||
help: string;
|
||||
/** Dimmed unless Mirostat is enabled (mirostat != 0). */
|
||||
needsMirostat?: boolean;
|
||||
}
|
||||
|
||||
export const OLLAMA_OPTIONS: OllamaOption[] = [
|
||||
// ── Sampling ──────────────────────────────────────────────────────────────
|
||||
{
|
||||
key: "temperature",
|
||||
group: "sampling",
|
||||
label: "Temperature",
|
||||
kind: "number",
|
||||
default: 0.8,
|
||||
min: 0,
|
||||
max: 2,
|
||||
step: 0.05,
|
||||
help: "Randomness / creativity. Higher is more creative.",
|
||||
},
|
||||
{
|
||||
key: "top_k",
|
||||
group: "sampling",
|
||||
label: "Top K",
|
||||
kind: "number",
|
||||
default: 40,
|
||||
min: 0,
|
||||
max: 200,
|
||||
step: 1,
|
||||
help: "Sample only from the K most likely tokens.",
|
||||
},
|
||||
{
|
||||
key: "top_p",
|
||||
group: "sampling",
|
||||
label: "Top P",
|
||||
kind: "number",
|
||||
default: 0.9,
|
||||
min: 0,
|
||||
max: 1,
|
||||
step: 0.05,
|
||||
help: "Nucleus sampling cumulative probability.",
|
||||
},
|
||||
{
|
||||
key: "min_p",
|
||||
group: "sampling",
|
||||
label: "Min P",
|
||||
kind: "number",
|
||||
default: 0.0,
|
||||
min: 0,
|
||||
max: 1,
|
||||
step: 0.01,
|
||||
help: "Minimum probability relative to the top token.",
|
||||
},
|
||||
{
|
||||
key: "typical_p",
|
||||
group: "sampling",
|
||||
label: "Typical P",
|
||||
kind: "number",
|
||||
default: 1.0,
|
||||
min: 0,
|
||||
max: 1,
|
||||
step: 0.05,
|
||||
help: "Locally-typical sampling.",
|
||||
},
|
||||
{
|
||||
key: "tfs_z",
|
||||
group: "sampling",
|
||||
label: "Tail-free (tfs_z)",
|
||||
kind: "number",
|
||||
default: 1.0,
|
||||
min: 0,
|
||||
max: 2,
|
||||
step: 0.05,
|
||||
help: "Tail-free sampling. 1.0 disables it (legacy).",
|
||||
},
|
||||
{
|
||||
key: "seed",
|
||||
group: "sampling",
|
||||
label: "Seed",
|
||||
kind: "number",
|
||||
default: 0,
|
||||
min: 0,
|
||||
step: 1,
|
||||
help: "Fixed seed → reproducible output. 0 = random.",
|
||||
},
|
||||
{
|
||||
key: "num_predict",
|
||||
group: "sampling",
|
||||
label: "Max tokens (num_predict)",
|
||||
kind: "number",
|
||||
default: -1,
|
||||
min: -2,
|
||||
step: 1,
|
||||
help: "Max tokens to generate. -1 = unlimited, -2 = fill context.",
|
||||
},
|
||||
|
||||
// ── Repetition ────────────────────────────────────────────────────────────
|
||||
{
|
||||
key: "repeat_penalty",
|
||||
group: "repetition",
|
||||
label: "Repeat penalty",
|
||||
kind: "number",
|
||||
default: 1.1,
|
||||
min: 0,
|
||||
max: 2,
|
||||
step: 0.05,
|
||||
help: "How strongly to penalize repetition.",
|
||||
},
|
||||
{
|
||||
key: "repeat_last_n",
|
||||
group: "repetition",
|
||||
label: "Repeat lookback",
|
||||
kind: "number",
|
||||
default: 64,
|
||||
min: -1,
|
||||
step: 1,
|
||||
help: "Tokens to look back for repetition. 0 = off, -1 = num_ctx.",
|
||||
},
|
||||
{
|
||||
key: "presence_penalty",
|
||||
group: "repetition",
|
||||
label: "Presence penalty",
|
||||
kind: "number",
|
||||
default: 0.0,
|
||||
min: -2,
|
||||
max: 2,
|
||||
step: 0.1,
|
||||
help: "Penalize tokens that already appeared.",
|
||||
},
|
||||
{
|
||||
key: "frequency_penalty",
|
||||
group: "repetition",
|
||||
label: "Frequency penalty",
|
||||
kind: "number",
|
||||
default: 0.0,
|
||||
min: -2,
|
||||
max: 2,
|
||||
step: 0.1,
|
||||
help: "Penalize tokens by how often they appear.",
|
||||
},
|
||||
{
|
||||
key: "penalize_newline",
|
||||
group: "repetition",
|
||||
label: "Penalize newlines",
|
||||
kind: "bool",
|
||||
default: true,
|
||||
help: "Apply the repeat penalty to newline tokens.",
|
||||
},
|
||||
|
||||
// ── Mirostat ──────────────────────────────────────────────────────────────
|
||||
{
|
||||
key: "mirostat",
|
||||
group: "mirostat",
|
||||
label: "Mirostat",
|
||||
kind: "enum",
|
||||
default: 0,
|
||||
enumOptions: [
|
||||
{ value: 0, label: "Off" },
|
||||
{ value: 1, label: "Mirostat 1" },
|
||||
{ value: 2, label: "Mirostat 2" },
|
||||
],
|
||||
help: "Adaptive perplexity control.",
|
||||
},
|
||||
{
|
||||
key: "mirostat_tau",
|
||||
group: "mirostat",
|
||||
label: "Mirostat τ (tau)",
|
||||
kind: "number",
|
||||
default: 5.0,
|
||||
min: 0,
|
||||
max: 10,
|
||||
step: 0.1,
|
||||
help: "Balance of coherence vs diversity.",
|
||||
needsMirostat: true,
|
||||
},
|
||||
{
|
||||
key: "mirostat_eta",
|
||||
group: "mirostat",
|
||||
label: "Mirostat η (eta)",
|
||||
kind: "number",
|
||||
default: 0.1,
|
||||
min: 0,
|
||||
max: 1,
|
||||
step: 0.01,
|
||||
help: "How fast it adapts to feedback.",
|
||||
needsMirostat: true,
|
||||
},
|
||||
|
||||
// ── Context & session ─────────────────────────────────────────────────────
|
||||
{
|
||||
key: "num_ctx",
|
||||
group: "context",
|
||||
label: "Context size (num_ctx)",
|
||||
kind: "number",
|
||||
default: 4096,
|
||||
min: 256,
|
||||
step: 256,
|
||||
help: "Context window in tokens.",
|
||||
},
|
||||
{
|
||||
key: "num_keep",
|
||||
group: "context",
|
||||
label: "Keep tokens (num_keep)",
|
||||
kind: "number",
|
||||
default: 4,
|
||||
min: 0,
|
||||
step: 1,
|
||||
help: "Tokens kept from the start of the prompt when truncating.",
|
||||
},
|
||||
{
|
||||
key: "stop",
|
||||
group: "context",
|
||||
label: "Stop sequences",
|
||||
kind: "tags",
|
||||
help: "Comma-separated strings that end generation.",
|
||||
},
|
||||
|
||||
// ── Runtime & hardware (rarely needed) ────────────────────────────────────
|
||||
{
|
||||
key: "num_batch",
|
||||
group: "hardware",
|
||||
label: "Batch size (num_batch)",
|
||||
kind: "number",
|
||||
default: 512,
|
||||
min: 1,
|
||||
step: 1,
|
||||
help: "Prompt processing batch size.",
|
||||
},
|
||||
{
|
||||
key: "num_gpu",
|
||||
group: "hardware",
|
||||
label: "GPU layers (num_gpu)",
|
||||
kind: "number",
|
||||
min: 0,
|
||||
step: 1,
|
||||
help: "Layers to offload to the GPU. Blank = auto.",
|
||||
},
|
||||
{
|
||||
key: "main_gpu",
|
||||
group: "hardware",
|
||||
label: "Main GPU",
|
||||
kind: "number",
|
||||
default: 0,
|
||||
min: 0,
|
||||
step: 1,
|
||||
help: "Which GPU to use for a single-GPU offload.",
|
||||
},
|
||||
{
|
||||
key: "num_thread",
|
||||
group: "hardware",
|
||||
label: "CPU threads (num_thread)",
|
||||
kind: "number",
|
||||
min: 1,
|
||||
step: 1,
|
||||
help: "Threads for computation. Blank = auto.",
|
||||
},
|
||||
{
|
||||
key: "low_vram",
|
||||
group: "hardware",
|
||||
label: "Low VRAM",
|
||||
kind: "bool",
|
||||
default: false,
|
||||
help: "Reduce VRAM use at the cost of speed.",
|
||||
},
|
||||
{
|
||||
key: "numa",
|
||||
group: "hardware",
|
||||
label: "NUMA",
|
||||
kind: "bool",
|
||||
default: false,
|
||||
help: "Enable NUMA support.",
|
||||
},
|
||||
{
|
||||
key: "use_mmap",
|
||||
group: "hardware",
|
||||
label: "Use mmap",
|
||||
kind: "bool",
|
||||
default: true,
|
||||
help: "Memory-map the model file.",
|
||||
},
|
||||
{
|
||||
key: "use_mlock",
|
||||
group: "hardware",
|
||||
label: "Use mlock",
|
||||
kind: "bool",
|
||||
default: false,
|
||||
help: "Lock the model in RAM (no swap).",
|
||||
},
|
||||
];
|
||||
|
||||
export const THINK_OPTIONS = ["off", "low", "medium", "high", "max"] as const;
|
||||
|
||||
export const OLLAMA_GROUP_LABELS: Record<OllamaGroup, string> = {
|
||||
sampling: "Sampling",
|
||||
repetition: "Repetition",
|
||||
mirostat: "Mirostat",
|
||||
context: "Context & session",
|
||||
hardware: "Runtime & hardware",
|
||||
};
|
||||
+463
-18
@@ -21,7 +21,17 @@
|
||||
UploadCloud,
|
||||
ShieldCheck,
|
||||
Sparkles,
|
||||
RefreshCw,
|
||||
ChevronRight,
|
||||
RotateCcw,
|
||||
} from "@lucide/svelte";
|
||||
import {
|
||||
OLLAMA_OPTIONS,
|
||||
THINK_OPTIONS,
|
||||
OLLAMA_GROUP_LABELS,
|
||||
type OllamaOption,
|
||||
type OllamaGroup,
|
||||
} from "../llmParams";
|
||||
|
||||
let { onClose }: { onClose: () => void } = $props();
|
||||
|
||||
@@ -68,6 +78,93 @@
|
||||
}
|
||||
}
|
||||
|
||||
// ---- Advanced Ollama configuration (sparse; only overrides are stored) ----
|
||||
const OPTION_GROUPS: OllamaGroup[] = ["sampling", "repetition", "mirostat", "context"];
|
||||
type OptVal = number | boolean | string[];
|
||||
let advSystem = $state("");
|
||||
let advThink = $state("off");
|
||||
let advKeepAlive = $state("");
|
||||
let advOptions = $state<Record<string, OptVal>>({});
|
||||
let advSaving = $state(false);
|
||||
let advSaved = $state(false);
|
||||
$effect(() => {
|
||||
// (Re)load the form whenever persisted settings change.
|
||||
const a = settings.settings.llm_advanced ?? {};
|
||||
advSystem = a.system ?? "";
|
||||
advThink = a.think ?? "off";
|
||||
advKeepAlive = a.keep_alive ?? "";
|
||||
advOptions = { ...(a.options ?? {}) };
|
||||
});
|
||||
let advChangedCount = $derived(
|
||||
Object.keys(advOptions).length +
|
||||
(advSystem.trim() ? 1 : 0) +
|
||||
(advThink !== "off" ? 1 : 0) +
|
||||
(advKeepAlive.trim() ? 1 : 0),
|
||||
);
|
||||
function mirostatOff(): boolean {
|
||||
const m = advOptions["mirostat"];
|
||||
return (m === undefined ? 0 : (m as number)) === 0;
|
||||
}
|
||||
function optDisplay(opt: OllamaOption): string {
|
||||
const v = advOptions[opt.key];
|
||||
if (opt.kind === "tags") return Array.isArray(v) ? v.join(", ") : "";
|
||||
if (v !== undefined) return String(v);
|
||||
return opt.default !== undefined ? String(opt.default) : "";
|
||||
}
|
||||
function isOverridden(opt: OllamaOption): boolean {
|
||||
return opt.key in advOptions;
|
||||
}
|
||||
function commit(key: string, value: OptVal | undefined) {
|
||||
const next = { ...advOptions };
|
||||
if (value === undefined) delete next[key];
|
||||
else next[key] = value;
|
||||
advOptions = next;
|
||||
advSaved = false;
|
||||
}
|
||||
function setNumber(opt: OllamaOption, raw: string) {
|
||||
const t = raw.trim();
|
||||
if (t === "") return commit(opt.key, undefined);
|
||||
const n = Number(t);
|
||||
if (Number.isNaN(n)) return;
|
||||
commit(opt.key, opt.default !== undefined && n === opt.default ? undefined : n);
|
||||
}
|
||||
function setEnum(opt: OllamaOption, raw: string) {
|
||||
const n = Number(raw);
|
||||
commit(opt.key, opt.default !== undefined && n === opt.default ? undefined : n);
|
||||
}
|
||||
function setBool(opt: OllamaOption, checked: boolean) {
|
||||
commit(opt.key, opt.default !== undefined && checked === opt.default ? undefined : checked);
|
||||
}
|
||||
function setTags(opt: OllamaOption, raw: string) {
|
||||
const arr = raw
|
||||
.split(",")
|
||||
.map((s) => s.trim())
|
||||
.filter(Boolean);
|
||||
commit(opt.key, arr.length ? arr : undefined);
|
||||
}
|
||||
function resetAllAdvanced() {
|
||||
advSystem = "";
|
||||
advThink = "off";
|
||||
advKeepAlive = "";
|
||||
advOptions = {};
|
||||
advSaved = false;
|
||||
}
|
||||
async function saveAdvanced() {
|
||||
advSaving = true;
|
||||
advSaved = false;
|
||||
const adv: NonNullable<(typeof settings.settings)["llm_advanced"]> = {};
|
||||
if (advSystem.trim()) adv.system = advSystem.trim();
|
||||
if (advThink !== "off") adv.think = advThink;
|
||||
if (advKeepAlive.trim()) adv.keep_alive = advKeepAlive.trim();
|
||||
if (Object.keys(advOptions).length) adv.options = advOptions;
|
||||
try {
|
||||
await settings.patch({ llm_advanced: Object.keys(adv).length ? adv : null });
|
||||
advSaved = true;
|
||||
} finally {
|
||||
advSaving = false;
|
||||
}
|
||||
}
|
||||
|
||||
// ---- Calendar / .pst import (T6.1/T6.2/T6.3, FR-CAL-1/2) ----
|
||||
let pstPath = $state("");
|
||||
let pstPassword = $state("");
|
||||
@@ -698,25 +795,172 @@
|
||||
<button class="primary" onclick={saveLlm} disabled={settings.llmSaving}>
|
||||
{settings.llmSaving ? "Saving…" : "Save"}
|
||||
</button>
|
||||
<button onclick={() => settings.loadLlmStatus()}>Test / refresh</button>
|
||||
{#if settings.llmStatus}
|
||||
<span
|
||||
class="test"
|
||||
class:ok={settings.llmStatus.reachable}
|
||||
class:fail={!settings.llmStatus.reachable}
|
||||
>
|
||||
{#if settings.llmStatus.reachable}<Check size={14} aria-hidden="true" />{:else}<X
|
||||
size={14}
|
||||
aria-hidden="true"
|
||||
/>{/if}
|
||||
{settings.llmStatus.reachable ? "reachable" : "unreachable"}
|
||||
{#if settings.llmStatus.reachable && !settings.llmStatus.isLocal}· leaves your
|
||||
network{/if}
|
||||
</span>
|
||||
{/if}
|
||||
<button class="ghost" onclick={() => settings.loadLlmStatus()}>
|
||||
<RefreshCw size={14} aria-hidden="true" /> Test connection
|
||||
</button>
|
||||
</div>
|
||||
{#if settings.llmStatus?.reachable && settings.llmStatus.models.length}
|
||||
<p class="muted">Models: {settings.llmStatus.models.slice(0, 6).join(", ")}</p>
|
||||
{#if settings.llmStatus}
|
||||
{@const s = settings.llmStatus}
|
||||
<div class="status-card">
|
||||
<span class="status-dot" class:ok={s.reachable} class:bad={!s.reachable}></span>
|
||||
<span class="status-text">{s.reachable ? "Reachable" : "Unreachable"}</span>
|
||||
{#if s.reachable}
|
||||
<span class="muted"
|
||||
>· {s.models.length} model{s.models.length === 1 ? "" : "s"}</span
|
||||
>
|
||||
<span class="muted" class:warn={!s.isLocal}
|
||||
>· {s.isLocal ? "on your machine / LAN" : "leaves your network"}</span
|
||||
>
|
||||
{/if}
|
||||
</div>
|
||||
{#if s.reachable && s.models.length}
|
||||
<p class="muted small">
|
||||
{s.models.slice(0, 6).join(", ")}{s.models.length > 6 ? " …" : ""}
|
||||
</p>
|
||||
{/if}
|
||||
{/if}
|
||||
|
||||
{#if llmProvider === "ollama"}
|
||||
{#snippet paramRow(opt: OllamaOption)}
|
||||
{@const disabled = opt.needsMirostat && mirostatOff()}
|
||||
<div class="param" class:param-disabled={disabled}>
|
||||
<div class="param-label">
|
||||
<span class="param-name">{opt.label}</span>
|
||||
<span class="param-help"
|
||||
>{opt.help}{#if opt.default !== undefined}
|
||||
Default {opt.default}.{/if}</span
|
||||
>
|
||||
</div>
|
||||
<div class="param-control">
|
||||
{#if opt.kind === "bool"}
|
||||
<input
|
||||
type="checkbox"
|
||||
checked={(advOptions[opt.key] ?? opt.default) === true}
|
||||
{disabled}
|
||||
onchange={(e) => setBool(opt, e.currentTarget.checked)}
|
||||
/>
|
||||
{:else if opt.kind === "enum"}
|
||||
<select
|
||||
value={optDisplay(opt)}
|
||||
onchange={(e) => setEnum(opt, e.currentTarget.value)}
|
||||
>
|
||||
{#each opt.enumOptions ?? [] as o (o.value)}
|
||||
<option value={String(o.value)}>{o.label}</option>
|
||||
{/each}
|
||||
</select>
|
||||
{:else if opt.kind === "tags"}
|
||||
<input
|
||||
value={optDisplay(opt)}
|
||||
placeholder="e.g. User:, \n"
|
||||
onchange={(e) => setTags(opt, e.currentTarget.value)}
|
||||
/>
|
||||
{:else}
|
||||
<input
|
||||
type="number"
|
||||
value={optDisplay(opt)}
|
||||
min={opt.min}
|
||||
max={opt.max}
|
||||
step={opt.step}
|
||||
{disabled}
|
||||
onchange={(e) => setNumber(opt, e.currentTarget.value)}
|
||||
/>
|
||||
{/if}
|
||||
<button
|
||||
class="reset-field"
|
||||
class:hidden={!isOverridden(opt)}
|
||||
title="Reset to default"
|
||||
aria-label="Reset {opt.label}"
|
||||
onclick={() => commit(opt.key, undefined)}
|
||||
>
|
||||
<RotateCcw size={13} aria-hidden="true" />
|
||||
</button>
|
||||
</div>
|
||||
</div>
|
||||
{/snippet}
|
||||
|
||||
<details class="advanced">
|
||||
<summary>
|
||||
<ChevronRight size={15} aria-hidden="true" class="chevron" />
|
||||
<span>Advanced Ollama configuration</span>
|
||||
{#if advChangedCount}<span class="changed-badge">{advChangedCount} changed</span
|
||||
>{/if}
|
||||
</summary>
|
||||
<div class="adv-body">
|
||||
{#if advChangedCount}
|
||||
<button class="link reset-all" onclick={resetAllAdvanced}
|
||||
>Reset all to defaults</button
|
||||
>
|
||||
{/if}
|
||||
|
||||
<label class="wide adv-system"
|
||||
>System prompt
|
||||
<textarea
|
||||
rows="3"
|
||||
bind:value={advSystem}
|
||||
placeholder="e.g. You are a concise meeting summarizer."
|
||||
></textarea>
|
||||
</label>
|
||||
<p class="muted small">
|
||||
Added before WhispAssist's required output format, so your instructions can't
|
||||
break summary/action-item parsing.
|
||||
</p>
|
||||
|
||||
{#each OPTION_GROUPS as g (g)}
|
||||
<fieldset class="adv-group">
|
||||
<legend>{OLLAMA_GROUP_LABELS[g]}</legend>
|
||||
{#if g === "context"}
|
||||
<div class="param">
|
||||
<div class="param-label">
|
||||
<span class="param-name">Think</span>
|
||||
<span class="param-help">Reasoning effort (reasoning models only).</span>
|
||||
</div>
|
||||
<div class="param-control">
|
||||
<select bind:value={advThink}>
|
||||
{#each THINK_OPTIONS as t (t)}<option value={t}>{t}</option>{/each}
|
||||
</select>
|
||||
</div>
|
||||
</div>
|
||||
<div class="param">
|
||||
<div class="param-label">
|
||||
<span class="param-name">Keep alive</span>
|
||||
<span class="param-help"
|
||||
>How long the model stays in RAM. e.g. 5m, 1h, 0 (unload), -1 (forever).</span
|
||||
>
|
||||
</div>
|
||||
<div class="param-control">
|
||||
<input bind:value={advKeepAlive} placeholder="5m" />
|
||||
</div>
|
||||
</div>
|
||||
{/if}
|
||||
{#each OLLAMA_OPTIONS.filter((o) => o.group === g) as opt (opt.key)}
|
||||
{@render paramRow(opt)}
|
||||
{/each}
|
||||
</fieldset>
|
||||
{/each}
|
||||
|
||||
<details class="adv-nested">
|
||||
<summary>
|
||||
<ChevronRight size={14} aria-hidden="true" class="chevron" />
|
||||
<span>Runtime & hardware</span>
|
||||
<span class="muted small">— rarely needed</span>
|
||||
</summary>
|
||||
<fieldset class="adv-group">
|
||||
{#each OLLAMA_OPTIONS.filter((o) => o.group === "hardware") as opt (opt.key)}
|
||||
{@render paramRow(opt)}
|
||||
{/each}
|
||||
</fieldset>
|
||||
</details>
|
||||
|
||||
<div class="actions">
|
||||
<button class="primary" onclick={saveAdvanced} disabled={advSaving}>
|
||||
{advSaving ? "Saving…" : "Save advanced"}
|
||||
</button>
|
||||
{#if advSaved}<span class="test ok"
|
||||
><Check size={14} aria-hidden="true" /> Saved</span
|
||||
>{/if}
|
||||
</div>
|
||||
</div>
|
||||
</details>
|
||||
{/if}
|
||||
{:else}
|
||||
<div class="actions">
|
||||
@@ -1153,4 +1397,205 @@
|
||||
ul.hosts li {
|
||||
padding: 0.2rem 0;
|
||||
}
|
||||
|
||||
/* ---- AI provider: status + advanced config ---- */
|
||||
.ghost {
|
||||
display: inline-flex;
|
||||
align-items: center;
|
||||
gap: 0.35rem;
|
||||
background: transparent;
|
||||
border: 1px solid var(--border);
|
||||
color: var(--fg, inherit);
|
||||
}
|
||||
.ghost:hover {
|
||||
background: var(--hover, rgba(127, 127, 127, 0.08));
|
||||
}
|
||||
.status-card {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
flex-wrap: wrap;
|
||||
gap: 0.4rem;
|
||||
margin-top: 0.6rem;
|
||||
padding: 0.5rem 0.7rem;
|
||||
border: 1px solid var(--border);
|
||||
border-radius: 8px;
|
||||
font-size: 0.85rem;
|
||||
}
|
||||
.status-dot {
|
||||
width: 9px;
|
||||
height: 9px;
|
||||
border-radius: 50%;
|
||||
background: var(--muted);
|
||||
flex: none;
|
||||
}
|
||||
.status-dot.ok {
|
||||
background: var(--success, #16a34a);
|
||||
}
|
||||
.status-dot.bad {
|
||||
background: var(--danger, #dc2626);
|
||||
}
|
||||
.status-text {
|
||||
font-weight: 600;
|
||||
}
|
||||
.muted.warn {
|
||||
color: var(--danger, #dc2626);
|
||||
}
|
||||
|
||||
.advanced,
|
||||
.adv-nested {
|
||||
margin-top: 0.9rem;
|
||||
border: 1px solid var(--border);
|
||||
border-radius: 8px;
|
||||
}
|
||||
.adv-nested {
|
||||
margin-top: 0.6rem;
|
||||
}
|
||||
.advanced > summary,
|
||||
.adv-nested > summary {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
gap: 0.45rem;
|
||||
padding: 0.6rem 0.75rem;
|
||||
cursor: pointer;
|
||||
list-style: none;
|
||||
font-weight: 600;
|
||||
user-select: none;
|
||||
}
|
||||
.advanced > summary::-webkit-details-marker,
|
||||
.adv-nested > summary::-webkit-details-marker {
|
||||
display: none;
|
||||
}
|
||||
.advanced :global(.chevron),
|
||||
.adv-nested :global(.chevron) {
|
||||
transition: transform 0.18s ease;
|
||||
flex: none;
|
||||
color: var(--muted);
|
||||
}
|
||||
.advanced[open] > summary :global(.chevron),
|
||||
.adv-nested[open] > summary :global(.chevron) {
|
||||
transform: rotate(90deg);
|
||||
}
|
||||
@media (prefers-reduced-motion: reduce) {
|
||||
.advanced :global(.chevron),
|
||||
.adv-nested :global(.chevron) {
|
||||
transition: none;
|
||||
}
|
||||
}
|
||||
.changed-badge {
|
||||
margin-left: auto;
|
||||
font-size: 0.7rem;
|
||||
font-weight: 500;
|
||||
padding: 0.1rem 0.45rem;
|
||||
border-radius: 10px;
|
||||
background: var(--accent, #2563eb);
|
||||
color: var(--accent-fg, #fff);
|
||||
}
|
||||
.adv-body {
|
||||
padding: 0 0.75rem 0.85rem;
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
gap: 0.5rem;
|
||||
}
|
||||
.reset-all {
|
||||
align-self: flex-end;
|
||||
font-size: 0.78rem;
|
||||
}
|
||||
.adv-system textarea {
|
||||
width: 100%;
|
||||
resize: vertical;
|
||||
font-family: inherit;
|
||||
font-size: 0.85rem;
|
||||
padding: 0.4rem 0.5rem;
|
||||
border: 1px solid var(--border-strong, var(--border));
|
||||
border-radius: 6px;
|
||||
}
|
||||
.adv-group {
|
||||
border: 1px solid var(--border);
|
||||
border-radius: 8px;
|
||||
padding: 0.4rem 0.75rem 0.6rem;
|
||||
margin: 0;
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
}
|
||||
.adv-group legend {
|
||||
font-size: 0.72rem;
|
||||
font-weight: 700;
|
||||
letter-spacing: 0.04em;
|
||||
text-transform: uppercase;
|
||||
color: var(--muted);
|
||||
padding: 0 0.35rem;
|
||||
}
|
||||
.param {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
gap: 1rem;
|
||||
padding: 0.4rem 0;
|
||||
border-bottom: 1px solid var(--border);
|
||||
}
|
||||
.param:last-child {
|
||||
border-bottom: none;
|
||||
}
|
||||
.param-label {
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
gap: 0.1rem;
|
||||
min-width: 0;
|
||||
flex: 1;
|
||||
}
|
||||
.param-name {
|
||||
font-size: 0.85rem;
|
||||
font-weight: 500;
|
||||
}
|
||||
.param-help {
|
||||
font-size: 0.74rem;
|
||||
color: var(--muted);
|
||||
line-height: 1.35;
|
||||
}
|
||||
.param-control {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
gap: 0.3rem;
|
||||
flex: none;
|
||||
}
|
||||
.param-control input[type="number"],
|
||||
.param-control select {
|
||||
width: 6.5rem;
|
||||
text-align: right;
|
||||
font-variant-numeric: tabular-nums;
|
||||
padding: 0.3rem 0.4rem;
|
||||
border: 1px solid var(--border-strong, var(--border));
|
||||
border-radius: 6px;
|
||||
}
|
||||
.param-control select {
|
||||
text-align: left;
|
||||
}
|
||||
.param-control input:not([type="number"]):not([type="checkbox"]) {
|
||||
width: 9rem;
|
||||
padding: 0.3rem 0.4rem;
|
||||
border: 1px solid var(--border-strong, var(--border));
|
||||
border-radius: 6px;
|
||||
}
|
||||
.param-disabled {
|
||||
opacity: 0.45;
|
||||
}
|
||||
.reset-field {
|
||||
display: inline-flex;
|
||||
align-items: center;
|
||||
justify-content: center;
|
||||
width: 1.6rem;
|
||||
height: 1.6rem;
|
||||
padding: 0;
|
||||
border: none;
|
||||
background: transparent;
|
||||
color: var(--muted);
|
||||
cursor: pointer;
|
||||
border-radius: 5px;
|
||||
}
|
||||
.reset-field:hover {
|
||||
background: var(--hover, rgba(127, 127, 127, 0.1));
|
||||
color: var(--fg, inherit);
|
||||
}
|
||||
.reset-field.hidden {
|
||||
visibility: hidden;
|
||||
}
|
||||
</style>
|
||||
|
||||
Reference in New Issue
Block a user