Phase 4
This commit is contained in:
@@ -0,0 +1,16 @@
|
||||
{
|
||||
"hooks": {
|
||||
"SessionStart": [
|
||||
{
|
||||
"matcher": "startup",
|
||||
"hooks": [
|
||||
{
|
||||
"type": "command",
|
||||
"command": "memanto memory sync --project-dir .",
|
||||
"timeout": 30
|
||||
}
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,152 @@
|
||||
---
|
||||
name: memanto-memory
|
||||
description: Use this skill when you need to store or search MEMANTO persistent memories. It defines mandatory guidelines for best practices, memory types, confidence levels, tagging, and patterns for effective agent memory usage.
|
||||
---
|
||||
|
||||
# MEMANTO Memory Skill
|
||||
|
||||
Detailed reference for using MEMANTO persistent memory effectively.
|
||||
|
||||
## Memory Types: Decision Matrix
|
||||
|
||||
| Type | When to Use | Confidence | Example |
|
||||
|------|-------------|------------|---------|
|
||||
| `fact` | Verified information, project status | 0.9-1.0 | "MEMANTO uses PostgreSQL for metadata" |
|
||||
| `decision` | Architecture choices, approach selections | 0.9-1.0 | "Chose React over Vue for frontend" |
|
||||
| `instruction` | Standing rules, preferences, guidelines | 0.9-1.0 | "Always use type hints in Python" |
|
||||
| `commitment` | Promises, TODOs, obligations | 1.0 | "Will deploy monitoring by Friday" |
|
||||
| `preference` | User/team preferences | 0.8-1.0 | "User prefers dark mode" |
|
||||
| `goal` | Objectives, targets, milestones | 0.8-1.0 | "Launch CLI by end of March" |
|
||||
| `artifact` | Tool outputs, reports, file locations | 0.9-1.0 | "Report saved at ./reports/q1.md" |
|
||||
| `learning` | Knowledge acquired from experience | 0.7-0.9 | "Batch operations 100x faster" |
|
||||
| `event` | Important conversations, milestones | 0.8-0.95 | "Completed Phase 1 features" |
|
||||
| `relationship` | Team context, collaboration patterns | 0.85-0.95 | "Alice is lead backend engineer" |
|
||||
| `observation` | Patterns noticed, behaviors | 0.6-0.85 | "User prefers short responses" |
|
||||
| `error` | Failures, bugs, lessons learned | 0.95-1.0 | "Namespace format bug - use underscores" |
|
||||
| `context` | Session summaries, status updates | 0.9-1.0 | "Project 70% done, API complete" |
|
||||
|
||||
## Confidence Levels
|
||||
|
||||
- **1.0** — Explicit user statement, verified fact, standing instruction
|
||||
- **0.9-0.95** — Strong consensus, well-tested approach, clear team preference
|
||||
- **0.8-0.85** — Observed pattern (3+ times), indirect but supported preference
|
||||
- **0.7-0.75** — Emerging pattern (2 times), reasonable inference
|
||||
- **0.6-0.65** — Single observation, uncertain interpretation
|
||||
- **< 0.6** — Don't store. Too uncertain.
|
||||
|
||||
## Provenance Types
|
||||
|
||||
Always categorize the source of the memory. Valid options:
|
||||
- `explicit_statement` — Directly stated by user
|
||||
- `inferred` — Derived from behavior/context
|
||||
- `observed` — Seen in action
|
||||
- `corrected` — Updated after contradiction
|
||||
- `validated` — Confirmed/verified
|
||||
- `imported` — Brought in from an external source (file upload, sync, migration)
|
||||
|
||||
## Source Types
|
||||
|
||||
Always specify the tool or agent creating the memory.
|
||||
- For AI agents: Use the agent name (e.g., `--source claude_code` or `--source cursor`)
|
||||
- Valid base sources (if not using specific agent name): `user`, `agent`, `tool`, `system`
|
||||
|
||||
## Tagging Best Practices
|
||||
|
||||
Use 2-5 tags per memory. Tags make memories findable.
|
||||
|
||||
Good: `--tags "authentication,oauth,security"`
|
||||
Good: `--tags "bug-fix,namespace,commit-3f39351"`
|
||||
Bad: `--tags "important"` (too generic)
|
||||
Bad: `--tags "thing"` (not descriptive)
|
||||
|
||||
Conventions:
|
||||
- Lowercase with hyphens: `bug-fix` not `BugFix`
|
||||
- Be specific: `authentication-oauth` not `auth`
|
||||
- Include refs: `commit-abc123` for git references
|
||||
|
||||
## Patterns
|
||||
|
||||
### Session Start
|
||||
```bash
|
||||
# recall — load raw context (instructions, decisions, goals) to guide this session
|
||||
memanto recall "instructions decisions goals" --limit 20
|
||||
|
||||
# answer — get a direct synthesized summary of pending commitments
|
||||
memanto answer "What are my pending commitments?"
|
||||
```
|
||||
|
||||
### After Important Work
|
||||
```bash
|
||||
memanto remember "Implemented X using approach Y because Z. Commit abc123." --type decision --tags "feature-x" --confidence 0.95 --provenance "inferred" --source "claude_code"
|
||||
memanto remember "Learned that batch ops reduce API calls 100x." --type learning --tags "performance" --confidence 0.85 --provenance "observed" --source "claude_code"
|
||||
```
|
||||
|
||||
### When User Corrects You
|
||||
```bash
|
||||
memanto remember "User corrected: prefer pytest over unittest." --type learning --tags "correction,testing" --confidence 1.0 --provenance "corrected" --source "claude_code"
|
||||
```
|
||||
|
||||
### Choosing Between recall and answer
|
||||
|
||||
These are **equal-priority tools**. Pick the right one — do NOT always default to `recall`.
|
||||
|
||||
| Situation | Use |
|
||||
|-----------|-----|
|
||||
| Need raw memory chunks to read and apply as context | `recall` |
|
||||
| Need a direct synthesized answer to give (or act on) | `answer` |
|
||||
| Building context before a complex multi-step task | `recall` |
|
||||
| User asks "what did we decide / prefer / commit to?" | `answer` |
|
||||
| Comparing multiple matching memories | `recall` |
|
||||
| Need one grounded yes/no or summary response | `answer` |
|
||||
|
||||
**Decision rule**: If your next step is *"read these memories and act"* → `recall`. If your next step is *"answer this question directly"* → `answer`. Both save tokens equally — `answer` synthesizes so you don't have to.
|
||||
|
||||
```bash
|
||||
# Use recall — need raw context to work from
|
||||
memanto recall "authentication approach" --limit 10
|
||||
|
||||
# Use answer — need a direct synthesized answer
|
||||
memanto answer "What auth approach did we decide on and why?"
|
||||
```
|
||||
|
||||
## Pitfalls to Avoid
|
||||
|
||||
1. **Memory hoarding** — Ask "Will this matter in a week?" before storing
|
||||
2. **Vague content** — Bad: "better performance" → Good: "API response < 200ms"
|
||||
3. **No context** — Bad: "fixed bug" → Good: "Fixed OAuth expiry bug. Commit abc123."
|
||||
4. **Duplicates** — Search first (`memanto recall`), then store if not found
|
||||
5. **Missing tags** — Always include tags for retrieval
|
||||
|
||||
## recall vs answer: Choose the Right Tool
|
||||
|
||||
**Equal priority** — do NOT always default to `recall`. Pick based on what you need next:
|
||||
|
||||
| Use `recall` when... | Use `answer` when... |
|
||||
|---------------------|---------------------|
|
||||
| You need raw memory chunks as context | You need one direct synthesized response |
|
||||
| Building context before a complex task | User asks "what did we decide / prefer?" |
|
||||
| Comparing or reviewing multiple memories | Getting a grounded summary or yes/no |
|
||||
| Next step: *read these and act on them* | Next step: *deliver this as the answer* |
|
||||
|
||||
**Short rule**: need context to work from → `recall`. Need a ready answer → `answer`. Both save the agent tokens and time — `answer` synthesizes so you don't have to read and merge manually.
|
||||
|
||||
## Command Reference
|
||||
|
||||
```bash
|
||||
# Store memory
|
||||
memanto remember "content" --type TYPE --tags "tag1,tag2" --confidence 0.9 --provenance "inferred" --source "claude_code"
|
||||
|
||||
# Raw memory search (use for context-building, multi-step tasks)
|
||||
memanto recall "query" --limit 10 --type TYPE --min-similarity 0.8
|
||||
|
||||
# Temporal recall variants (no query needed)
|
||||
memanto recall --recent --limit 10 # newest first
|
||||
memanto recall --as-of "2026-01-15" # state at a point in time
|
||||
memanto recall --changed-since "last 7 days" # what changed since
|
||||
|
||||
# Synthesized answer (use for direct questions, "what did we decide about X?")
|
||||
memanto answer "question"
|
||||
|
||||
# Sync memories to project
|
||||
memanto memory sync --project-dir .
|
||||
```
|
||||
@@ -3,6 +3,101 @@
|
||||
This file orients an AI coding agent (or human) working in this repository. Read it before
|
||||
making changes.
|
||||
|
||||
<!-- MEMANTO-MANAGED-SECTION -->
|
||||
## MEMANTO - Your Active Memory Companion
|
||||
|
||||
**MEMANTO is not a passive store. It is an active companion agent that works alongside you.**
|
||||
Don't treat MEMANTO like a static blob you query once and forget. It's a teammate you keep
|
||||
talking to, every preference, decision, and correction flows through it. MEMANTO remembers,
|
||||
recalls, and answers so you hold context across sessions, honor prior decisions, and avoid
|
||||
repeating mistakes the user already corrected.
|
||||
|
||||
Every memory operation in this session goes through MEMANTO. There is no exception.
|
||||
|
||||
> **CRITICAL**: All `memanto` commands are **shell commands**. Always run them using the Bash tool.
|
||||
> Never simulate, describe, or "pretend to call" them. If you cannot run the shell, say so explicitly instead of inventing memory state.
|
||||
|
||||
### NON-NEGOTIABLE RULES
|
||||
|
||||
These are not suggestions. Follow each one on every turn.
|
||||
|
||||
1. **Read `MEMORY.md` before doing anything.** It is auto-synced at session start and holds
|
||||
the user's preferences, facts, goals, instructions, decisions, and commitments from every
|
||||
prior session. You MUST honor what is written there. If you act against it, you are
|
||||
breaking continuity the user is paying for.
|
||||
2. **Search memory before saying you don't know.** If the user asks about past context, an
|
||||
earlier decision, a preference, or anything you are unsure about, you MUST run `recall`
|
||||
or `answer` first. Saying "I don't have context" without searching is a failure.
|
||||
3. **Store proactively. Do not wait to be asked.** The moment a memory-worthy event happens
|
||||
— a preference stated, a decision made, a fact learned, an instruction given, a goal set,
|
||||
a mistake corrected — run `memanto remember` immediately, in the same turn.
|
||||
4. **Always pass full metadata to `remember`.** Every `memanto remember` call MUST include
|
||||
`--type`, `--confidence`, `--provenance`, and `--source <your_agent_name>`. Never let
|
||||
these default. Untyped, unsourced memories pollute the agent's recall quality.
|
||||
5. **One memory operation goes through MEMANTO. All of them do.** Do not keep mental notes,
|
||||
in-context scratch pads, or "I'll remember this for next time" promises. If it matters
|
||||
beyond this turn, it goes into MEMANTO. If it doesn't, drop it.
|
||||
|
||||
### Memory Operations — Use the Right One
|
||||
|
||||
MEMANTO gives you three primitives. They are equal-priority. Pick by intent, not by habit.
|
||||
|
||||
| You want to... | Use | Why |
|
||||
|---|---|---|
|
||||
| Read raw memory chunks and apply them as context | `memanto recall "query"` | Best for context-building, multi-step work, comparing options |
|
||||
| Get one synthesized, grounded answer to a direct question | `memanto answer "question"` | Best for "what did we decide / prefer / commit to?" — saves you reading and merging |
|
||||
| Persist something memory-worthy | `memanto remember "content" --type ... --confidence ... --provenance ... --source ...` | Every preference, decision, fact, instruction, goal, lesson |
|
||||
| See what changed since last time | `memanto recall --changed-since "last 7 days"` | Catching up after a break |
|
||||
| See the most recent memories | `memanto recall --recent` | Fast context refresh |
|
||||
|
||||
Do NOT always default to `recall`. If the user asked a direct question, `answer` is usually
|
||||
the right tool — it returns a grounded synthesis so you don't burn tokens re-reading raw
|
||||
chunks.
|
||||
|
||||
### When to Call `remember` (Examples — Run Immediately)
|
||||
|
||||
- User says *"I prefer tabs over spaces"*:
|
||||
`memanto remember "User prefers tabs over spaces for indentation" --type preference --confidence 1.0 --provenance explicit_statement --source <your_agent_name>`
|
||||
- You decide to use Library X for reason Y:
|
||||
`memanto remember "Chose Library X for reason Y; commit abc123" --type decision --confidence 0.95 --provenance inferred --source <your_agent_name>`
|
||||
- User corrects an approach:
|
||||
`memanto remember "User corrected: use pytest, not unittest" --type learning --confidence 1.0 --provenance corrected --source <your_agent_name>`
|
||||
- A failed approach taught you something:
|
||||
`memanto remember "Batch size > 100 fails with TimeoutError" --type error --confidence 0.95 --provenance observed --source <your_agent_name>`
|
||||
|
||||
### Command Reference
|
||||
|
||||
```bash
|
||||
# Store — ALWAYS pass full metadata
|
||||
memanto remember "content" --type <type> --confidence <0.0-1.0> --provenance <provenance> --source <agent_name>
|
||||
|
||||
# Recall raw context
|
||||
memanto recall "query" # semantic search
|
||||
memanto recall "query" --type <type> --limit 10 # filtered search
|
||||
memanto recall --recent --limit 10 # newest first, no query
|
||||
memanto recall --as-of "2026-01-15" # state at a point in time
|
||||
memanto recall --changed-since "last 7 days" # what changed since
|
||||
|
||||
# Synthesized answer (grounded RAG over memories)
|
||||
memanto answer "question"
|
||||
|
||||
# Re-sync MEMORY.md (project-local cache)
|
||||
memanto memory sync --project-dir .
|
||||
```
|
||||
|
||||
**Memory types** (use the closest fit, do not invent new ones):
|
||||
`fact`, `preference`, `instruction`, `decision`, `event`, `goal`, `commitment`,
|
||||
`observation`, `learning`, `relationship`, `context`, `artifact`, `error`.
|
||||
|
||||
**Provenance values**: `explicit_statement`, `inferred`, `observed`, `corrected`,
|
||||
`validated`, `imported`.
|
||||
|
||||
**Confidence**: `1.0` for explicit user statements; `0.9-0.95` for strong consensus;
|
||||
`0.8-0.85` for observed patterns (3+ times); `0.6-0.75` for emerging patterns.
|
||||
|
||||
> **Note**: The `memanto-memory` skill contains reference guidelines only (best practices, confidence levels, tagging). It is NOT executable — always use Bash for memanto commands.
|
||||
<!-- /MEMANTO-MANAGED-SECTION -->
|
||||
|
||||
## What this project is
|
||||
|
||||
WhispAssist (WA) is a **fully local, open-source, Windows-native** meeting assistant. The
|
||||
|
||||
@@ -58,6 +58,9 @@ CREATE TABLE speakers (
|
||||
display_name TEXT, -- user/participant name (nullable)
|
||||
participant_id TEXT REFERENCES participants(id), -- if mapped to a calendar attendee
|
||||
color TEXT, -- UI color hint
|
||||
merged_into TEXT, -- non-null: this label folds into another label's
|
||||
-- row at render/export time (T4.5, FR-SPK-3); segment
|
||||
-- speaker IDs in storage are never rewritten (FR-SPK-5)
|
||||
UNIQUE(meeting_id, label)
|
||||
);
|
||||
|
||||
|
||||
@@ -28,8 +28,9 @@ set_preferred_backend(input: { backend: BackendId | "auto" }): void
|
||||
// ---- Transcription / models ----
|
||||
reprocess_transcript(input: { meetingId: MeetingId; model: string }): void // batch mode (FR-TRX-3)
|
||||
list_models(): ModelInfo[]
|
||||
list_diarization_models(): ModelInfo[] // fixed seg+emb pair (T4.7, FR-MODEL-1)
|
||||
download_model(input: { kind: "whisper" | "diar-seg" | "diar-emb"; id: string }): void // emits progress events
|
||||
remove_model(input: { id: string }): void
|
||||
remove_model(input: { id: string }): void // disambiguated by id, not kind — ids never collide across catalogs
|
||||
|
||||
// ---- Speakers ----
|
||||
rename_speaker(input: { meetingId: MeetingId; label: string; name: string }): void
|
||||
|
||||
Generated
+287
-14
@@ -126,6 +126,29 @@ version = "1.8.3"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "2af50177e190e07a26ab74f8b1efbfe2ef87da2116221318cb1c2e82baf7de06"
|
||||
|
||||
[[package]]
|
||||
name = "bindgen"
|
||||
version = "0.69.5"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "271383c67ccabffb7381723dea0672a673f292304fcb45c01cc648c7a8d58088"
|
||||
dependencies = [
|
||||
"bitflags 2.13.0",
|
||||
"cexpr",
|
||||
"clang-sys",
|
||||
"itertools 0.12.1",
|
||||
"lazy_static",
|
||||
"lazycell",
|
||||
"log",
|
||||
"prettyplease",
|
||||
"proc-macro2",
|
||||
"quote",
|
||||
"regex",
|
||||
"rustc-hash 1.1.0",
|
||||
"shlex 1.3.0",
|
||||
"syn 2.0.118",
|
||||
"which",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "bindgen"
|
||||
version = "0.72.1"
|
||||
@@ -135,13 +158,13 @@ dependencies = [
|
||||
"bitflags 2.13.0",
|
||||
"cexpr",
|
||||
"clang-sys",
|
||||
"itertools",
|
||||
"itertools 0.13.0",
|
||||
"log",
|
||||
"prettyplease",
|
||||
"proc-macro2",
|
||||
"quote",
|
||||
"regex",
|
||||
"rustc-hash",
|
||||
"rustc-hash 2.1.2",
|
||||
"shlex 1.3.0",
|
||||
"syn 2.0.118",
|
||||
]
|
||||
@@ -257,6 +280,26 @@ dependencies = [
|
||||
"serde",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "bzip2"
|
||||
version = "0.4.4"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "bdb116a6ef3f6c3698828873ad02c3014b3c85cadb88496095628e3ef1e347f8"
|
||||
dependencies = [
|
||||
"bzip2-sys",
|
||||
"libc",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "bzip2-sys"
|
||||
version = "0.1.13+1.0.8"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "225bff33b2141874fe80d71e07d6eec4f85c5c216453dd96388240f96e1acc14"
|
||||
dependencies = [
|
||||
"cc",
|
||||
"pkg-config",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "cairo-rs"
|
||||
version = "0.18.5"
|
||||
@@ -695,13 +738,34 @@ dependencies = [
|
||||
"subtle",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "dirs"
|
||||
version = "5.0.1"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "44c45a9d03d6676652bcb5e724c7e988de1acad23a711b5217ab9cbecbec2225"
|
||||
dependencies = [
|
||||
"dirs-sys 0.4.1",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "dirs"
|
||||
version = "6.0.0"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "c3e8aa94d75141228480295a7d0e7feb620b1a5ad9f12bc40be62411e38cce4e"
|
||||
dependencies = [
|
||||
"dirs-sys",
|
||||
"dirs-sys 0.5.0",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "dirs-sys"
|
||||
version = "0.4.1"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "520f05a5cbd335fae5a99ff7a6ab8627577660ee5cfd6a94a6a929b52ff0321c"
|
||||
dependencies = [
|
||||
"libc",
|
||||
"option-ext",
|
||||
"redox_users 0.4.6",
|
||||
"windows-sys 0.48.0",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
@@ -712,7 +776,7 @@ checksum = "e01a3366d27ee9890022452ee61b2b63a67e6f13f58900b651ff5665f0bb1fab"
|
||||
dependencies = [
|
||||
"libc",
|
||||
"option-ext",
|
||||
"redox_users",
|
||||
"redox_users 0.5.2",
|
||||
"windows-sys 0.61.2",
|
||||
]
|
||||
|
||||
@@ -880,6 +944,16 @@ dependencies = [
|
||||
"typeid",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "errno"
|
||||
version = "0.3.14"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "39cab71617ae0d63f51a36d69f866391735b51691dbda63cf6f96d042b63efeb"
|
||||
dependencies = [
|
||||
"libc",
|
||||
"windows-sys 0.61.2",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "etcetera"
|
||||
version = "0.8.0"
|
||||
@@ -902,6 +976,16 @@ dependencies = [
|
||||
"pin-project-lite",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "eyre"
|
||||
version = "0.6.12"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "7cd915d99f24784cdc19fd37ef22b97e3ff0ae756c7e492e9fbfe897d61e2aec"
|
||||
dependencies = [
|
||||
"indenter",
|
||||
"once_cell",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "fastrand"
|
||||
version = "2.4.1"
|
||||
@@ -927,6 +1011,16 @@ dependencies = [
|
||||
"rustc_version",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "filetime"
|
||||
version = "0.2.29"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "5c287a33c7f0a620c38e641e7f60827713987b3c0f26e8ddc9462cc69cf75759"
|
||||
dependencies = [
|
||||
"cfg-if",
|
||||
"libc",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "find-msvc-tools"
|
||||
version = "0.1.9"
|
||||
@@ -1573,7 +1667,7 @@ dependencies = [
|
||||
"tokio",
|
||||
"tokio-rustls",
|
||||
"tower-service",
|
||||
"webpki-roots",
|
||||
"webpki-roots 1.0.8",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
@@ -1755,6 +1849,12 @@ dependencies = [
|
||||
"png 0.18.1",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "indenter"
|
||||
version = "0.3.4"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "964de6e86d545b246d84badc0fef527924ace5134f30641c203ef52ba83f58d5"
|
||||
|
||||
[[package]]
|
||||
name = "indexmap"
|
||||
version = "1.9.3"
|
||||
@@ -1793,6 +1893,15 @@ version = "2.12.0"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "d98f6fed1fde3f8c21bc40a1abb88dd75e67924f9cffc3ef95607bad8017f8e2"
|
||||
|
||||
[[package]]
|
||||
name = "itertools"
|
||||
version = "0.12.1"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "ba291022dbbd398a455acf126c1e341954079855bc60dfdda641363bd6922569"
|
||||
dependencies = [
|
||||
"either",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "itertools"
|
||||
version = "0.13.0"
|
||||
@@ -1938,6 +2047,12 @@ dependencies = [
|
||||
"spin",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "lazycell"
|
||||
version = "1.3.0"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "830d08ce1d1d941e6b30645f1a0eb5643013d835ce3779a5fc208261dbe10f55"
|
||||
|
||||
[[package]]
|
||||
name = "libappindicator"
|
||||
version = "0.9.0"
|
||||
@@ -2026,6 +2141,18 @@ dependencies = [
|
||||
"vcpkg",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "linux-raw-sys"
|
||||
version = "0.4.15"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "d26c52dbd32dccf2d10cac7725f8eae5296885fb5703b261f7d0a0739ec807ab"
|
||||
|
||||
[[package]]
|
||||
name = "linux-raw-sys"
|
||||
version = "0.12.1"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "32a66949e030da00e8c7d4434b251670a91556f4144941d37452769c25d58a53"
|
||||
|
||||
[[package]]
|
||||
name = "litemap"
|
||||
version = "0.8.2"
|
||||
@@ -2827,7 +2954,7 @@ dependencies = [
|
||||
"pin-project-lite",
|
||||
"quinn-proto",
|
||||
"quinn-udp",
|
||||
"rustc-hash",
|
||||
"rustc-hash 2.1.2",
|
||||
"rustls",
|
||||
"socket2",
|
||||
"thiserror 2.0.18",
|
||||
@@ -2847,7 +2974,7 @@ dependencies = [
|
||||
"lru-slab",
|
||||
"rand 0.9.4",
|
||||
"ring",
|
||||
"rustc-hash",
|
||||
"rustc-hash 2.1.2",
|
||||
"rustls",
|
||||
"rustls-pki-types",
|
||||
"slab",
|
||||
@@ -2975,6 +3102,17 @@ dependencies = [
|
||||
"bitflags 2.13.0",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "redox_users"
|
||||
version = "0.4.6"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "ba009ff324d1fc1b900bd1fdb31564febe58a8ccc8a6fdbb93b543d33b13ca43"
|
||||
dependencies = [
|
||||
"getrandom 0.2.17",
|
||||
"libredox",
|
||||
"thiserror 1.0.69",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "redox_users"
|
||||
version = "0.5.2"
|
||||
@@ -3073,7 +3211,7 @@ dependencies = [
|
||||
"wasm-bindgen-futures",
|
||||
"wasm-streams 0.4.2",
|
||||
"web-sys",
|
||||
"webpki-roots",
|
||||
"webpki-roots 1.0.8",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
@@ -3203,6 +3341,12 @@ dependencies = [
|
||||
"zeroize",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "rustc-hash"
|
||||
version = "1.1.0"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "08d43f7aa6b08d49f382cde6a7982047c3426db949b1424bc4b7ec9ae12c6ce2"
|
||||
|
||||
[[package]]
|
||||
name = "rustc-hash"
|
||||
version = "2.1.2"
|
||||
@@ -3218,12 +3362,39 @@ dependencies = [
|
||||
"semver",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "rustix"
|
||||
version = "0.38.44"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "fdb5bc1ae2baa591800df16c9ca78619bf65c0488b41b96ccec5d11220d8c154"
|
||||
dependencies = [
|
||||
"bitflags 2.13.0",
|
||||
"errno",
|
||||
"libc",
|
||||
"linux-raw-sys 0.4.15",
|
||||
"windows-sys 0.59.0",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "rustix"
|
||||
version = "1.1.4"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "b6fe4565b9518b83ef4f91bb47ce29620ca828bd32cb7e408f0062e9930ba190"
|
||||
dependencies = [
|
||||
"bitflags 2.13.0",
|
||||
"errno",
|
||||
"libc",
|
||||
"linux-raw-sys 0.12.1",
|
||||
"windows-sys 0.61.2",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "rustls"
|
||||
version = "0.23.41"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "6b92b125634d9b795e7beca796cc790df15a7fb38323bf3196fda83292d06b1f"
|
||||
dependencies = [
|
||||
"log",
|
||||
"once_cell",
|
||||
"ring",
|
||||
"rustls-pki-types",
|
||||
@@ -3359,7 +3530,7 @@ dependencies = [
|
||||
"phf",
|
||||
"phf_codegen",
|
||||
"precomputed-hash",
|
||||
"rustc-hash",
|
||||
"rustc-hash 2.1.2",
|
||||
"servo_arc",
|
||||
"smallvec",
|
||||
]
|
||||
@@ -3575,6 +3746,38 @@ dependencies = [
|
||||
"lazy_static",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "sherpa-rs"
|
||||
version = "0.6.8"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "81835d89a3fc44482a6829e3e2483ac83f7b4d1623c76f196c77c9ecf5c18203"
|
||||
dependencies = [
|
||||
"eyre",
|
||||
"hound",
|
||||
"sherpa-rs-sys",
|
||||
"tracing",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "sherpa-rs-sys"
|
||||
version = "0.6.8"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "591c9432b20f41d47f622a73c2888b188c321e313d820824df7d9fa51dbada43"
|
||||
dependencies = [
|
||||
"bindgen 0.69.5",
|
||||
"bzip2",
|
||||
"cmake",
|
||||
"dirs 5.0.1",
|
||||
"flate2",
|
||||
"glob",
|
||||
"lazy_static",
|
||||
"serde",
|
||||
"serde_json",
|
||||
"sha2",
|
||||
"tar",
|
||||
"ureq",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "shlex"
|
||||
version = "1.3.0"
|
||||
@@ -3634,6 +3837,17 @@ dependencies = [
|
||||
"windows-sys 0.61.2",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "socks"
|
||||
version = "0.3.4"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "f0c3dbbd9ae980613c6dd8e28a9407b50509d3803b57624d5dfe8315218cd58b"
|
||||
dependencies = [
|
||||
"byteorder",
|
||||
"libc",
|
||||
"winapi",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "softbuffer"
|
||||
version = "0.4.8"
|
||||
@@ -4058,6 +4272,17 @@ dependencies = [
|
||||
"syn 2.0.118",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "tar"
|
||||
version = "0.4.46"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "3f6221d9a6003c78398e3b239969f352578258df48c8eb051caadae0015bc840"
|
||||
dependencies = [
|
||||
"filetime",
|
||||
"libc",
|
||||
"xattr",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "target-lexicon"
|
||||
version = "0.12.16"
|
||||
@@ -4073,7 +4298,7 @@ dependencies = [
|
||||
"anyhow",
|
||||
"bytes",
|
||||
"cookie",
|
||||
"dirs",
|
||||
"dirs 6.0.0",
|
||||
"dunce",
|
||||
"embed_plist",
|
||||
"getrandom 0.3.4",
|
||||
@@ -4124,7 +4349,7 @@ checksum = "bc9ce40b16101cb6ea63d3e221567affd1c3a9205f95d7bc574941a10636b632"
|
||||
dependencies = [
|
||||
"anyhow",
|
||||
"cargo_toml",
|
||||
"dirs",
|
||||
"dirs 6.0.0",
|
||||
"glob",
|
||||
"heck 0.5.0",
|
||||
"json-patch",
|
||||
@@ -4744,7 +4969,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "65ba1e5f6b9ef9fd87e21b9c6f351554dbd717960089168fcfdef854686961dc"
|
||||
dependencies = [
|
||||
"crossbeam-channel",
|
||||
"dirs",
|
||||
"dirs 6.0.0",
|
||||
"libappindicator",
|
||||
"muda",
|
||||
"objc2",
|
||||
@@ -4857,6 +5082,22 @@ version = "0.9.0"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "8ecb6da28b8a351d773b68d5825ac39017e680750f980f3a1a85cd8dd28a47c1"
|
||||
|
||||
[[package]]
|
||||
name = "ureq"
|
||||
version = "2.12.1"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "02d1a66277ed75f640d608235660df48c8e3c19f3b4edb6a263315626cc3c01d"
|
||||
dependencies = [
|
||||
"base64 0.22.1",
|
||||
"log",
|
||||
"once_cell",
|
||||
"rustls",
|
||||
"rustls-pki-types",
|
||||
"socks",
|
||||
"url",
|
||||
"webpki-roots 0.26.11",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "url"
|
||||
version = "2.5.8"
|
||||
@@ -5160,6 +5401,15 @@ dependencies = [
|
||||
"system-deps",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "webpki-roots"
|
||||
version = "0.26.11"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "521bc38abb08001b01866da9f51eb7c5d647a19260e00054a8c7fd5f9e57f7a9"
|
||||
dependencies = [
|
||||
"webpki-roots 1.0.8",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "webpki-roots"
|
||||
version = "1.0.8"
|
||||
@@ -5205,6 +5455,18 @@ dependencies = [
|
||||
"windows-core 0.61.2",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "which"
|
||||
version = "4.4.2"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "87ba24419a2078cd2b0f2ede2691b6c66d8e47836da3b6db8265ebad47afbfc7"
|
||||
dependencies = [
|
||||
"either",
|
||||
"home",
|
||||
"once_cell",
|
||||
"rustix 0.38.44",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "whispassist"
|
||||
version = "0.0.0"
|
||||
@@ -5218,6 +5480,7 @@ dependencies = [
|
||||
"serde",
|
||||
"serde_json",
|
||||
"sha2",
|
||||
"sherpa-rs",
|
||||
"sqlx",
|
||||
"tauri",
|
||||
"tauri-build",
|
||||
@@ -5248,7 +5511,7 @@ version = "0.15.0"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "6986c0fe081241d391f09b9a071fbcbb59720c3563628c3c829057cf69f2a56f"
|
||||
dependencies = [
|
||||
"bindgen",
|
||||
"bindgen 0.72.1",
|
||||
"cfg-if",
|
||||
"cmake",
|
||||
"fs_extra",
|
||||
@@ -5944,7 +6207,7 @@ dependencies = [
|
||||
"block2",
|
||||
"cookie",
|
||||
"crossbeam-channel",
|
||||
"dirs",
|
||||
"dirs 6.0.0",
|
||||
"dom_query",
|
||||
"dpi",
|
||||
"dunce",
|
||||
@@ -5999,6 +6262,16 @@ dependencies = [
|
||||
"pkg-config",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "xattr"
|
||||
version = "1.6.1"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "32e45ad4206f6d2479085147f02bc2ef834ac85886624a23575ae137c8aa8156"
|
||||
dependencies = [
|
||||
"libc",
|
||||
"rustix 1.1.4",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "yoke"
|
||||
version = "0.8.3"
|
||||
|
||||
@@ -42,6 +42,7 @@ rmcp = { version = "0.16", optional = true, features = ["server"] } # MCP serve
|
||||
# feature-gated so the CPU-only build always compiles (NFR-MNT-4).
|
||||
hound = { version = "3", optional = true } # WAV I/O (Phase 1)
|
||||
whisper-rs = { version = "0.16", optional = true } # whisper.cpp bindings (Phase 1)
|
||||
sherpa-rs = { version = "0.6", optional = true, default-features = false, features = ["download-binaries"] } # sherpa-onnx bindings (Phase 4, ADR-0005)
|
||||
tauri-plugin-dialog = "2" # native Save/choose-folder (Phase 2 export)
|
||||
|
||||
[target.'cfg(windows)'.dependencies]
|
||||
@@ -53,7 +54,7 @@ windows = { version = "0.58", features = [
|
||||
wasapi = { version = "0.15", optional = true } # Phase 1
|
||||
|
||||
[features]
|
||||
default = ["audio", "cpu-transcription"]
|
||||
default = ["audio", "cpu-transcription", "diarization"]
|
||||
# Phase 1
|
||||
audio = ["dep:wasapi", "dep:hound"]
|
||||
cpu-transcription = ["dep:whisper-rs"] # whisper-rs CPU build
|
||||
@@ -62,7 +63,7 @@ cuda = ["whisper-rs?/cuda"] # whisper.cpp CUDA
|
||||
vulkan = ["whisper-rs?/vulkan"] # whisper.cpp Vulkan
|
||||
directml = [] # ort + DirectML NPU path (T3.4, not yet implemented)
|
||||
# Phase 4 / 6 (added when integrated)
|
||||
diarization = [] # sherpa-onnx
|
||||
diarization = ["dep:sherpa-rs"] # sherpa-onnx
|
||||
pst = [] # outlook-pst
|
||||
# Phase 9
|
||||
sync = ["dep:keyring"] # remote upload (WebDAV + OAuth providers)
|
||||
|
||||
@@ -0,0 +1,8 @@
|
||||
-- WhispAssist speaker-merge schema (Phase 4, T4.5, FR-SPK-3).
|
||||
-- Forward-only migration. Mirrors docs/03-data-model.md.
|
||||
PRAGMA foreign_keys = ON;
|
||||
|
||||
-- Folds an over-split speaker into a canonical label without touching segment
|
||||
-- speaker IDs in storage (FR-SPK-5): a non-null merged_into means this row's
|
||||
-- label resolves to `merged_into` at render/export time instead of its own.
|
||||
ALTER TABLE speakers ADD COLUMN merged_into TEXT;
|
||||
+351
-20
@@ -6,20 +6,25 @@
|
||||
//! else is still a typed `todo!()` stub mapped to its roadmap task.
|
||||
|
||||
use crate::audio::{AudioCapture, WasapiCapture};
|
||||
use crate::diarization::{Diarizer, SherpaDiarizer};
|
||||
use crate::error::WaResult;
|
||||
use crate::hardware::{HardwareDetector, WinHardwareDetector};
|
||||
use crate::models::*;
|
||||
use crate::notes::NotesRenderer;
|
||||
use crate::paths::{meeting_dir, settings_path, wa_root, whisper_model_file};
|
||||
use crate::paths::{
|
||||
diarization_embedding_model_file, diarization_segmentation_model_file, meeting_dir,
|
||||
settings_path, wa_root, whisper_model_file,
|
||||
};
|
||||
use crate::storage::{FinalizeMeeting, Meeting, NewMeeting};
|
||||
use crate::transcription::{
|
||||
models as model_catalog, run_streaming_worker, Transcriber, WhisperTranscriber,
|
||||
};
|
||||
use crate::{error::WaError, AppState, RecordingSession};
|
||||
use serde::Deserialize;
|
||||
use std::collections::HashMap;
|
||||
use std::path::PathBuf;
|
||||
use std::sync::{Arc, Mutex as StdMutex};
|
||||
use tauri::{AppHandle, Emitter, State};
|
||||
use tauri::{AppHandle, Emitter, Manager, State};
|
||||
|
||||
#[derive(Deserialize)]
|
||||
pub struct StartRecordingArgs {
|
||||
@@ -98,6 +103,54 @@ fn backend_for(settings: &Settings) -> BackendId {
|
||||
WinHardwareDetector.best(preferred).id
|
||||
}
|
||||
|
||||
/// Builds a `Diarizer` if both diarization models are installed — a fixed
|
||||
/// pair of well-known filenames, downloadable/removable via
|
||||
/// `diarization::models` and `list_diarization_models`/`download_model`/
|
||||
/// `remove_model` (T4.7). Returns `None` rather than erring — diarization is
|
||||
/// a provisional/refinement layer that a recording never depends on, same
|
||||
/// treatment as a missing hardware backend.
|
||||
fn diarizer_from_installed_models() -> Option<SherpaDiarizer> {
|
||||
let seg_model = diarization_segmentation_model_file();
|
||||
let emb_model = diarization_embedding_model_file();
|
||||
if !seg_model.exists() || !emb_model.exists() {
|
||||
return None;
|
||||
}
|
||||
match SherpaDiarizer::new(&seg_model, &emb_model) {
|
||||
Ok(d) => Some(d),
|
||||
Err(e) => {
|
||||
tracing::warn!("diarization models present but failed to load: {e}");
|
||||
None
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// The distinct speakers seen in `segments` so far, in first-appearance order,
|
||||
/// with any display names applied (T4.3/T4.4, FR-SPK-2/5). Falls back to the
|
||||
/// single pre-diarization "S1" placeholder if no segments exist yet.
|
||||
fn speaker_infos_from_segments(
|
||||
segments: &[TranscriptSegment],
|
||||
names: &HashMap<String, String>,
|
||||
) -> Vec<SpeakerInfo> {
|
||||
let mut seen = std::collections::HashSet::new();
|
||||
let mut out: Vec<SpeakerInfo> = segments
|
||||
.iter()
|
||||
.filter(|s| seen.insert(s.speaker.clone()))
|
||||
.map(|s| SpeakerInfo {
|
||||
label: s.speaker.clone(),
|
||||
display_name: names.get(&s.speaker).cloned(),
|
||||
participant_id: None,
|
||||
})
|
||||
.collect();
|
||||
if out.is_empty() {
|
||||
out.push(SpeakerInfo {
|
||||
label: "S1".to_string(),
|
||||
display_name: names.get("S1").cloned(),
|
||||
participant_id: None,
|
||||
});
|
||||
}
|
||||
out
|
||||
}
|
||||
|
||||
// ---- Recording lifecycle (Phase 1) ----
|
||||
|
||||
#[tauri::command]
|
||||
@@ -201,6 +254,79 @@ pub async fn start_recording(
|
||||
})
|
||||
.map_err(|e| WaError::new("transcription", e.to_string()))?;
|
||||
|
||||
// T4.3: cheap/provisional live diarization, skipped entirely (None) when
|
||||
// the diarization models aren't installed yet (T4.7) — never blocks
|
||||
// recording, same graceful-degradation treatment as a missing backend.
|
||||
// Loading the ONNX models is blocking I/O, so it runs off this async task.
|
||||
let diarizer: Option<Arc<dyn Diarizer>> =
|
||||
tauri::async_runtime::spawn_blocking(diarizer_from_installed_models)
|
||||
.await
|
||||
.ok()
|
||||
.flatten()
|
||||
.map(|d| Arc::new(d) as Arc<dyn Diarizer>);
|
||||
let speaker_names: Arc<StdMutex<HashMap<String, String>>> =
|
||||
Arc::new(StdMutex::new(HashMap::new()));
|
||||
|
||||
if let Some(diarizer) = diarizer.clone() {
|
||||
let app_for_diar = app.clone();
|
||||
let meeting_id_for_diar = meeting_id.clone();
|
||||
let wav_path_for_diar = wav_path.clone();
|
||||
let segments_for_diar = segments.clone();
|
||||
let names_for_diar = speaker_names.clone();
|
||||
tauri::async_runtime::spawn(async move {
|
||||
// ponytail: reprocesses the whole recording-so-far each tick
|
||||
// rather than incremental/windowed segmentation — sherpa-onnx's
|
||||
// offline Diarizer has no streaming primitive to build on, and
|
||||
// this is provisional preview only (the accurate pass runs once
|
||||
// at stop). Fine at meeting length and a 15s cadence; revisit
|
||||
// with real streaming segmentation if long meetings make it heavy.
|
||||
let mut ticker = tokio::time::interval(std::time::Duration::from_secs(15));
|
||||
ticker.tick().await; // interval's first tick fires immediately; skip it
|
||||
loop {
|
||||
ticker.tick().await;
|
||||
let still_active = app_for_diar
|
||||
.state::<AppState>()
|
||||
.session
|
||||
.lock()
|
||||
.await
|
||||
.as_ref()
|
||||
.is_some_and(|s| s.meeting_id == meeting_id_for_diar);
|
||||
if !still_active {
|
||||
break; // recording stopped (or a new one started) — nothing left to do
|
||||
}
|
||||
|
||||
let diarizer_for_pass = diarizer.clone();
|
||||
let wav_path = wav_path_for_diar.clone();
|
||||
let spans = tauri::async_runtime::spawn_blocking(move || {
|
||||
diarizer_for_pass.diarize(&wav_path)
|
||||
})
|
||||
.await;
|
||||
let spans = match spans {
|
||||
Ok(Ok(spans)) => spans,
|
||||
Ok(Err(e)) => {
|
||||
tracing::warn!("live diarization pass failed: {e}");
|
||||
continue;
|
||||
}
|
||||
Err(e) => {
|
||||
tracing::warn!("live diarization task failed: {e}");
|
||||
continue;
|
||||
}
|
||||
};
|
||||
|
||||
let speakers = {
|
||||
let mut segs = segments_for_diar.lock().unwrap_or_else(|e| e.into_inner());
|
||||
diarizer.assign(&mut segs, &spans);
|
||||
let names = names_for_diar.lock().unwrap_or_else(|e| e.into_inner());
|
||||
speaker_infos_from_segments(&segs, &names)
|
||||
};
|
||||
let _ = app_for_diar.emit(
|
||||
"diarization://updated",
|
||||
serde_json::json!({ "meetingId": meeting_id_for_diar, "speakers": speakers }),
|
||||
);
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
*guard = Some(RecordingSession {
|
||||
meeting_id: meeting_id.clone(),
|
||||
capture,
|
||||
@@ -211,6 +337,8 @@ pub async fn start_recording(
|
||||
segments,
|
||||
active_backend,
|
||||
model_id,
|
||||
diarizer,
|
||||
speaker_names,
|
||||
});
|
||||
drop(guard);
|
||||
|
||||
@@ -255,18 +383,35 @@ pub async fn stop_recording(
|
||||
// guarantee it has fully drained the audio before we act on retention.
|
||||
let _ = session.transcription_worker.join();
|
||||
|
||||
let segments = session
|
||||
let mut segments = session
|
||||
.segments
|
||||
.lock()
|
||||
.map(|g| g.clone())
|
||||
.unwrap_or_default();
|
||||
let segment_count = segments.len();
|
||||
// No diarization until Phase 4 — every segment is provisionally "S1".
|
||||
let speakers = vec![SpeakerInfo {
|
||||
label: "S1".to_string(),
|
||||
display_name: None,
|
||||
participant_id: None,
|
||||
}];
|
||||
|
||||
// T4.1/4.2: one authoritative diarization pass over the now-complete
|
||||
// recording (ADR-0005's "post-stop pass"), refining whatever the live
|
||||
// provisional passes (T4.3) produced. Skipped if diarization models
|
||||
// aren't installed — `speaker_infos_from_segments` then falls back to
|
||||
// the single pre-diarization "S1" placeholder, same as before Phase 4.
|
||||
if let Some(diarizer) = session.diarizer.clone() {
|
||||
let diarizer_for_task = diarizer.clone();
|
||||
let wav_path = session.wav_path.clone();
|
||||
match tauri::async_runtime::spawn_blocking(move || diarizer_for_task.diarize(&wav_path))
|
||||
.await
|
||||
{
|
||||
Ok(Ok(spans)) => diarizer.assign(&mut segments, &spans),
|
||||
Ok(Err(e)) => tracing::warn!("final diarization pass failed: {e}"),
|
||||
Err(e) => tracing::warn!("final diarization task failed: {e}"),
|
||||
}
|
||||
}
|
||||
let speaker_names = session
|
||||
.speaker_names
|
||||
.lock()
|
||||
.map(|g| g.clone())
|
||||
.unwrap_or_default();
|
||||
let speakers = speaker_infos_from_segments(&segments, &speaker_names);
|
||||
let backend_used = session
|
||||
.active_backend
|
||||
.lock()
|
||||
@@ -395,6 +540,120 @@ pub async fn acknowledge_recording_consent() -> WaResult<()> {
|
||||
save_settings(&settings)
|
||||
}
|
||||
|
||||
// ---- Speakers (Phase 4) ----
|
||||
|
||||
/// Re-renders and persists `notes.md` from a finalized meeting's current
|
||||
/// (post-rename/post-merge) segments+speakers, and tells the frontend what
|
||||
/// changed (T4.5/4.6, FR-SPK-3/5). This is what keeps `export_meeting` — which
|
||||
/// just copies the already-rendered `notes.md` — in sync with naming changes
|
||||
/// made after the meeting ends; `transcript.json` itself is untouched.
|
||||
async fn refresh_notes_and_notify(
|
||||
app: &AppHandle,
|
||||
state: &State<'_, AppState>,
|
||||
meeting_id: &MeetingId,
|
||||
) -> WaResult<Vec<SpeakerInfo>> {
|
||||
let meeting = state
|
||||
.store
|
||||
.get_meeting(meeting_id)
|
||||
.await
|
||||
.map_err(|e| WaError::new("storage", e.to_string()))?;
|
||||
let notes_md =
|
||||
crate::notes::MarkdownNotes.to_markdown(&meeting.segments, &meeting.speakers, None);
|
||||
let _ = std::fs::write(meeting_dir(meeting_id).join("notes.md"), notes_md);
|
||||
let _ = app.emit(
|
||||
"diarization://updated",
|
||||
serde_json::json!({ "meetingId": meeting_id, "speakers": meeting.speakers }),
|
||||
);
|
||||
Ok(meeting.speakers)
|
||||
}
|
||||
|
||||
/// Name a speaker; applies to that speaker's past & future segments (T4.4,
|
||||
/// FR-SPK-2). Segments only ever carry the internal label ("S1"…) — never
|
||||
/// rewritten — so persisting the label→name mapping here is enough to cover
|
||||
/// both past and future segments once names are resolved at render time
|
||||
/// (FR-SPK-5). Works whether the meeting is still recording or already
|
||||
/// finalized.
|
||||
#[tauri::command]
|
||||
pub async fn rename_speaker(
|
||||
app: AppHandle,
|
||||
state: State<'_, AppState>,
|
||||
meeting_id: MeetingId,
|
||||
label: String,
|
||||
name: String,
|
||||
) -> WaResult<()> {
|
||||
state
|
||||
.store
|
||||
.rename_speaker(&meeting_id, &label, &name)
|
||||
.await
|
||||
.map_err(|e| WaError::new("storage", e.to_string()))?;
|
||||
|
||||
let guard = state.session.lock().await;
|
||||
match guard.as_ref().filter(|s| s.meeting_id == meeting_id) {
|
||||
Some(session) => {
|
||||
// Still recording: transcript.json/notes.md don't exist on disk
|
||||
// yet, so there's nothing to re-render — just refresh the live
|
||||
// in-memory view (finalize builds notes.md from this same map
|
||||
// at stop, T4.3/4.4).
|
||||
let names = {
|
||||
let mut names = session
|
||||
.speaker_names
|
||||
.lock()
|
||||
.unwrap_or_else(|e| e.into_inner());
|
||||
names.insert(label, name);
|
||||
names.clone()
|
||||
};
|
||||
let segments = session
|
||||
.segments
|
||||
.lock()
|
||||
.map(|g| g.clone())
|
||||
.unwrap_or_default();
|
||||
drop(guard);
|
||||
let speakers = speaker_infos_from_segments(&segments, &names);
|
||||
let _ = app.emit(
|
||||
"diarization://updated",
|
||||
serde_json::json!({ "meetingId": meeting_id, "speakers": speakers }),
|
||||
);
|
||||
}
|
||||
None => {
|
||||
drop(guard);
|
||||
refresh_notes_and_notify(&app, &state, &meeting_id).await?;
|
||||
}
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Fold over-split speakers into one canonical label (T4.5, FR-SPK-3) — e.g.
|
||||
/// diarization split one person into "S2" and "S3"; merging them shows one
|
||||
/// name and one grouped paragraph in notes/export. Post-meeting only: while
|
||||
/// still recording, the live provisional pass (T4.3) re-clusters from scratch
|
||||
/// every tick, so a label merged now could mean something else by the next
|
||||
/// tick.
|
||||
#[tauri::command]
|
||||
pub async fn merge_speakers(
|
||||
app: AppHandle,
|
||||
state: State<'_, AppState>,
|
||||
meeting_id: MeetingId,
|
||||
from: Vec<String>,
|
||||
into: String,
|
||||
) -> WaResult<()> {
|
||||
let guard = state.session.lock().await;
|
||||
if guard.as_ref().is_some_and(|s| s.meeting_id == meeting_id) {
|
||||
return Err(WaError::new(
|
||||
"recording",
|
||||
"cannot merge speakers while this meeting is still recording — wait until it's stopped",
|
||||
));
|
||||
}
|
||||
drop(guard);
|
||||
|
||||
state
|
||||
.store
|
||||
.merge_speakers(&meeting_id, &from, &into)
|
||||
.await
|
||||
.map_err(|e| WaError::new("storage", e.to_string()))?;
|
||||
refresh_notes_and_notify(&app, &state, &meeting_id).await?;
|
||||
Ok(())
|
||||
}
|
||||
|
||||
// ---- Hardware + models (Phase 3) ----
|
||||
|
||||
#[tauri::command]
|
||||
@@ -431,34 +690,55 @@ pub async fn list_models() -> WaResult<Vec<ModelInfo>> {
|
||||
Ok(model_catalog::list(&model_id_for(&settings)))
|
||||
}
|
||||
|
||||
/// The fixed segmentation+embedding pair (T4.7, FR-MODEL-1) — a separate
|
||||
/// command rather than folding into `list_models` because they're a fixed
|
||||
/// installable pair, not an interchangeable-size catalog like whisper's.
|
||||
#[tauri::command]
|
||||
pub async fn list_diarization_models() -> WaResult<Vec<ModelInfo>> {
|
||||
Ok(crate::diarization::models::list())
|
||||
}
|
||||
|
||||
#[derive(Deserialize)]
|
||||
pub struct DownloadModelArgs {
|
||||
pub kind: String, // "whisper" — diar-seg/diar-emb land in Phase 4 (T4.7)
|
||||
pub kind: String, // "whisper" | "diar-seg" | "diar-emb"
|
||||
pub id: String,
|
||||
}
|
||||
|
||||
#[tauri::command]
|
||||
pub async fn download_model(app: AppHandle, args: DownloadModelArgs) -> WaResult<()> {
|
||||
if args.kind != "whisper" {
|
||||
return Err(WaError::new(
|
||||
"model",
|
||||
format!("model kind '{}' is not available yet", args.kind),
|
||||
));
|
||||
}
|
||||
let id = args.id;
|
||||
let id_for_progress = id.clone();
|
||||
model_catalog::download(&id, move |received, total| {
|
||||
let on_progress = move |received: u64, total: Option<u64>| {
|
||||
let _ = app.emit(
|
||||
"model://progress",
|
||||
serde_json::json!({ "id": id_for_progress, "receivedBytes": received, "totalBytes": total }),
|
||||
);
|
||||
})
|
||||
.await
|
||||
.map_err(|e| WaError::new("model", e.to_string()))
|
||||
};
|
||||
match args.kind.as_str() {
|
||||
"whisper" => model_catalog::download(&id, on_progress)
|
||||
.await
|
||||
.map_err(|e| WaError::new("model", e.to_string())),
|
||||
"diar-seg" | "diar-emb" => crate::diarization::models::download(&id, on_progress)
|
||||
.await
|
||||
.map_err(|e| WaError::new("model", e.to_string())),
|
||||
other => Err(WaError::new(
|
||||
"model",
|
||||
format!("unknown model kind '{other}'"),
|
||||
)),
|
||||
}
|
||||
}
|
||||
|
||||
#[tauri::command]
|
||||
pub async fn remove_model(id: String) -> WaResult<()> {
|
||||
// No `kind` in this command's contract — disambiguate by catalog
|
||||
// membership instead; whisper/diarization ids never collide.
|
||||
if crate::diarization::models::list()
|
||||
.iter()
|
||||
.any(|m| m.id == id)
|
||||
{
|
||||
return crate::diarization::models::remove(&id)
|
||||
.map_err(|e| WaError::new("model", e.to_string()));
|
||||
}
|
||||
let settings = load_settings();
|
||||
model_catalog::remove(&id, &model_id_for(&settings))
|
||||
.map_err(|e| WaError::new("model", e.to_string()))
|
||||
@@ -913,3 +1193,54 @@ pub async fn update_settings(patch: serde_json::Value) -> WaResult<Settings> {
|
||||
pub async fn privacy_self_check() -> WaResult<serde_json::Value> {
|
||||
todo!("Phase 7 — privacy_self_check")
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
fn segment(speaker: &str) -> TranscriptSegment {
|
||||
TranscriptSegment {
|
||||
id: 0,
|
||||
start_ms: 0,
|
||||
end_ms: 1000,
|
||||
speaker: speaker.to_string(),
|
||||
text: String::new(),
|
||||
confidence: None,
|
||||
interim: false,
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn speaker_infos_lists_distinct_speakers_in_first_appearance_order() {
|
||||
let segments = vec![segment("S2"), segment("S1"), segment("S2")];
|
||||
let infos = speaker_infos_from_segments(&segments, &HashMap::new());
|
||||
let labels: Vec<&str> = infos.iter().map(|s| s.label.as_str()).collect();
|
||||
assert_eq!(labels, vec!["S2", "S1"]);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn speaker_infos_applies_display_names_by_label() {
|
||||
let segments = vec![segment("S1"), segment("S2")];
|
||||
let mut names = HashMap::new();
|
||||
names.insert("S1".to_string(), "Alice".to_string());
|
||||
let infos = speaker_infos_from_segments(&segments, &names);
|
||||
assert_eq!(infos[0].display_name.as_deref(), Some("Alice"));
|
||||
assert_eq!(infos[1].display_name, None); // S2 was never named
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn speaker_infos_falls_back_to_s1_placeholder_when_no_segments_yet() {
|
||||
let infos = speaker_infos_from_segments(&[], &HashMap::new());
|
||||
assert_eq!(infos.len(), 1);
|
||||
assert_eq!(infos[0].label, "S1");
|
||||
assert!(infos[0].display_name.is_none());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn diarizer_is_none_when_models_are_not_installed() {
|
||||
// This test environment never has the fixed-path diarization models
|
||||
// installed (T4.7 will add real download/selection) — confirms the
|
||||
// graceful-degradation path a recording never blocks on (T4.3).
|
||||
assert!(diarizer_from_installed_models().is_none());
|
||||
}
|
||||
}
|
||||
|
||||
@@ -9,6 +9,8 @@
|
||||
use crate::models::{SpeakerSpan, TranscriptSegment};
|
||||
use std::path::Path;
|
||||
|
||||
pub mod models;
|
||||
|
||||
#[derive(Debug, thiserror::Error)]
|
||||
pub enum DiarError {
|
||||
#[error("model load failed: {0}")]
|
||||
@@ -24,17 +26,167 @@ pub trait Diarizer: Send + Sync {
|
||||
fn assign(&self, segments: &mut [TranscriptSegment], spans: &[SpeakerSpan]);
|
||||
}
|
||||
|
||||
/// Labels each segment with whichever span overlaps it most, in milliseconds
|
||||
/// (T4.2, FR-SPK-1). A segment with no overlapping span (e.g. it falls in a
|
||||
/// gap between spans) keeps its prior speaker label — the "S1" placeholder
|
||||
/// every segment starts with pre-diarization — rather than guessing.
|
||||
///
|
||||
/// Pure timestamp arithmetic, so it's engine-agnostic: every `Diarizer` impl
|
||||
/// can share it instead of reimplementing overlap math.
|
||||
pub fn assign_by_overlap(segments: &mut [TranscriptSegment], spans: &[SpeakerSpan]) {
|
||||
for segment in segments.iter_mut() {
|
||||
let best_span = spans
|
||||
.iter()
|
||||
.map(|span| {
|
||||
let overlap_start = segment.start_ms.max(span.start_ms);
|
||||
let overlap_end = segment.end_ms.min(span.end_ms);
|
||||
(overlap_end.saturating_sub(overlap_start), span)
|
||||
})
|
||||
.filter(|(overlap, _)| *overlap > 0)
|
||||
.max_by_key(|(overlap, _)| *overlap)
|
||||
.map(|(_, span)| span);
|
||||
|
||||
if let Some(span) = best_span {
|
||||
segment.speaker = span.speaker.clone();
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// sherpa-onnx-backed diarizer: pyannote segmentation + speaker-embedding +
|
||||
/// fast clustering (ADR-0005, T4.1). `Diarize::compute` needs `&mut self`; it's
|
||||
/// wrapped in a `Mutex` to satisfy `Diarizer: Sync` — diarization is a
|
||||
/// once-per-meeting post-pass (never a hot path), so lock contention is moot.
|
||||
#[cfg(feature = "diarization")]
|
||||
pub struct SherpaDiarizer;
|
||||
pub struct SherpaDiarizer {
|
||||
engine: std::sync::Mutex<sherpa_rs::diarize::Diarize>,
|
||||
}
|
||||
|
||||
#[cfg(feature = "diarization")]
|
||||
impl SherpaDiarizer {
|
||||
pub fn new(segmentation_model: &Path, embedding_model: &Path) -> Result<Self, DiarError> {
|
||||
let config = sherpa_rs::diarize::DiarizeConfig {
|
||||
// A meeting's speaker count isn't known ahead of time: <= 0 tells
|
||||
// sherpa-onnx to pick the cluster count itself from `threshold`
|
||||
// instead of forcing a fixed number of speakers.
|
||||
num_clusters: Some(-1),
|
||||
threshold: Some(0.5),
|
||||
..Default::default()
|
||||
};
|
||||
let engine = sherpa_rs::diarize::Diarize::new(segmentation_model, embedding_model, config)
|
||||
.map_err(|e| DiarError::Load(e.to_string()))?;
|
||||
Ok(Self {
|
||||
engine: std::sync::Mutex::new(engine),
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(feature = "diarization")]
|
||||
impl Diarizer for SherpaDiarizer {
|
||||
fn diarize(&self, _wav: &Path) -> Result<Vec<SpeakerSpan>, DiarError> {
|
||||
// T4.1: sherpa-onnx segmentation + embedding + clustering via FFI.
|
||||
todo!("Phase 4 — diarize")
|
||||
fn diarize(&self, wav: &Path) -> Result<Vec<SpeakerSpan>, DiarError> {
|
||||
let samples =
|
||||
crate::audio::read_wav_mono_16k(wav).map_err(|e| DiarError::Run(e.to_string()))?;
|
||||
let mut engine = self.engine.lock().unwrap_or_else(|e| e.into_inner());
|
||||
let segments = engine
|
||||
.compute(samples, None)
|
||||
.map_err(|e| DiarError::Run(e.to_string()))?;
|
||||
Ok(segments.into_iter().map(segment_to_span).collect())
|
||||
}
|
||||
fn assign(&self, _segments: &mut [TranscriptSegment], _spans: &[SpeakerSpan]) {
|
||||
// T4.2: timestamp-overlap alignment.
|
||||
todo!("Phase 4 — assign speakers to segments")
|
||||
|
||||
fn assign(&self, segments: &mut [TranscriptSegment], spans: &[SpeakerSpan]) {
|
||||
assign_by_overlap(segments, spans);
|
||||
}
|
||||
}
|
||||
|
||||
/// sherpa-onnx speaker indices are 0-based; WA's internal labels are 1-based ("S1"…).
|
||||
#[cfg(feature = "diarization")]
|
||||
fn segment_to_span(seg: sherpa_rs::diarize::Segment) -> SpeakerSpan {
|
||||
SpeakerSpan {
|
||||
start_ms: (seg.start.max(0.0) * 1000.0) as u64,
|
||||
end_ms: (seg.end.max(0.0) * 1000.0) as u64,
|
||||
speaker: format!("S{}", seg.speaker + 1),
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod overlap_tests {
|
||||
use super::*;
|
||||
|
||||
fn segment(start_ms: u64, end_ms: u64) -> TranscriptSegment {
|
||||
TranscriptSegment {
|
||||
id: 0,
|
||||
start_ms,
|
||||
end_ms,
|
||||
speaker: "S1".to_string(),
|
||||
text: String::new(),
|
||||
confidence: None,
|
||||
interim: false,
|
||||
}
|
||||
}
|
||||
|
||||
fn span(start_ms: u64, end_ms: u64, speaker: &str) -> SpeakerSpan {
|
||||
SpeakerSpan {
|
||||
start_ms,
|
||||
end_ms,
|
||||
speaker: speaker.to_string(),
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn assigns_the_span_that_overlaps_most() {
|
||||
let mut segments = vec![segment(0, 1000), segment(1000, 2000)];
|
||||
let spans = vec![span(0, 1000, "S1"), span(1000, 2000, "S2")];
|
||||
assign_by_overlap(&mut segments, &spans);
|
||||
assert_eq!(segments[0].speaker, "S1");
|
||||
assert_eq!(segments[1].speaker, "S2");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_segment_straddling_two_spans_picks_the_larger_overlap() {
|
||||
// 700ms in S1's span (300-1000), 300ms in S2's span (1000-1300).
|
||||
let mut segments = vec![segment(300, 1300)];
|
||||
let spans = vec![span(0, 1000, "S1"), span(1000, 2000, "S2")];
|
||||
assign_by_overlap(&mut segments, &spans);
|
||||
assert_eq!(segments[0].speaker, "S1");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_segment_with_no_overlapping_span_keeps_its_prior_label() {
|
||||
let mut segments = vec![segment(5000, 6000)];
|
||||
segments[0].speaker = "S9".to_string(); // distinct from any span below
|
||||
let spans = vec![span(0, 1000, "S1")];
|
||||
assign_by_overlap(&mut segments, &spans);
|
||||
assert_eq!(segments[0].speaker, "S9"); // untouched, not overwritten with a guess
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn no_spans_at_all_leaves_segments_untouched() {
|
||||
let mut segments = vec![segment(0, 1000)];
|
||||
assign_by_overlap(&mut segments, &[]);
|
||||
assert_eq!(segments[0].speaker, "S1");
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(all(test, feature = "diarization"))]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn segment_to_span_maps_0_based_speaker_index_to_1_based_label() {
|
||||
let seg = sherpa_rs::diarize::Segment {
|
||||
start: 1.5,
|
||||
end: 3.25,
|
||||
speaker: 0,
|
||||
};
|
||||
let span = segment_to_span(seg);
|
||||
assert_eq!(span.start_ms, 1500);
|
||||
assert_eq!(span.end_ms, 3250);
|
||||
assert_eq!(span.speaker, "S1");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn new_surfaces_a_load_error_for_missing_models_instead_of_panicking() {
|
||||
let result =
|
||||
SherpaDiarizer::new(Path::new("no-such-seg.onnx"), Path::new("no-such-emb.onnx"));
|
||||
assert!(matches!(result, Err(DiarError::Load(_))));
|
||||
}
|
||||
}
|
||||
|
||||
@@ -0,0 +1,134 @@
|
||||
//! Diarization model catalog + install/remove (Phase 4, T4.7, FR-MODEL-1).
|
||||
//!
|
||||
//! ponytail: two fixed models (segmentation + embedding, ADR-0005), not a
|
||||
//! fetched index — same lazy-correct call as the whisper catalog (T3.7).
|
||||
//! Mirrors the pyannote-segmentation-3.0 + 3D-Speaker ERes2Net (English)
|
||||
//! models sherpa-onnx's own docs use for offline speaker diarization.
|
||||
|
||||
use crate::models::ModelInfo;
|
||||
use crate::paths::{
|
||||
diarization_embedding_model_file, diarization_segmentation_model_file, models_dir,
|
||||
};
|
||||
use futures_util::StreamExt;
|
||||
use std::path::PathBuf;
|
||||
|
||||
struct Catalog {
|
||||
id: &'static str,
|
||||
label: &'static str,
|
||||
size_mb: u32,
|
||||
url: &'static str,
|
||||
dest: fn() -> PathBuf,
|
||||
}
|
||||
|
||||
const CATALOG: &[Catalog] = &[
|
||||
Catalog {
|
||||
id: "seg-pyannote-3.0",
|
||||
label: "Speaker segmentation (pyannote 3.0)",
|
||||
size_mb: 6,
|
||||
url: "https://huggingface.co/csukuangfj/sherpa-onnx-pyannote-segmentation-3-0/resolve/main/model.onnx",
|
||||
dest: diarization_segmentation_model_file,
|
||||
},
|
||||
Catalog {
|
||||
id: "spk-eres2net",
|
||||
label: "Speaker embedding (3D-Speaker ERes2Net, English)",
|
||||
size_mb: 27,
|
||||
url: "https://huggingface.co/csukuangfj/speaker-embedding-models/resolve/main/3dspeaker_speech_eres2net_sv_en_voxceleb_16k.onnx",
|
||||
dest: diarization_embedding_model_file,
|
||||
},
|
||||
];
|
||||
|
||||
fn find(id: &str) -> Option<&'static Catalog> {
|
||||
CATALOG.iter().find(|m| m.id == id)
|
||||
}
|
||||
|
||||
pub fn list() -> Vec<ModelInfo> {
|
||||
CATALOG
|
||||
.iter()
|
||||
.map(|m| ModelInfo {
|
||||
id: m.id.to_string(),
|
||||
label: m.label.to_string(),
|
||||
size_mb: m.size_mb,
|
||||
installed: (m.dest)().exists(),
|
||||
// Both models are always "active" once installed — diarization
|
||||
// has no interchangeable-size picker like whisper's (yet).
|
||||
active: true,
|
||||
})
|
||||
.collect()
|
||||
}
|
||||
|
||||
#[derive(Debug, thiserror::Error)]
|
||||
pub enum ModelError {
|
||||
#[error("unknown diarization model id: {0}")]
|
||||
UnknownId(String),
|
||||
#[error("download failed: {0}")]
|
||||
Download(String),
|
||||
#[error("io error: {0}")]
|
||||
Io(#[from] std::io::Error),
|
||||
}
|
||||
|
||||
/// Downloads a diarization model, reporting progress via `on_progress`.
|
||||
/// Writes to a `.part` file first so a crash/cancel never leaves a truncated
|
||||
/// model that `diarizer_from_installed_models` would treat as installed.
|
||||
pub async fn download(
|
||||
id: &str,
|
||||
mut on_progress: impl FnMut(u64, Option<u64>),
|
||||
) -> Result<(), ModelError> {
|
||||
let catalog = find(id).ok_or_else(|| ModelError::UnknownId(id.to_string()))?;
|
||||
std::fs::create_dir_all(models_dir())?;
|
||||
let dest = (catalog.dest)();
|
||||
let tmp = dest.with_extension("part");
|
||||
|
||||
let resp = reqwest::get(catalog.url)
|
||||
.await
|
||||
.map_err(|e| ModelError::Download(e.to_string()))?;
|
||||
if !resp.status().is_success() {
|
||||
return Err(ModelError::Download(format!("HTTP {}", resp.status())));
|
||||
}
|
||||
let total = resp.content_length();
|
||||
let mut received: u64 = 0;
|
||||
let mut file = std::fs::File::create(&tmp)?;
|
||||
let mut stream = resp.bytes_stream();
|
||||
while let Some(chunk) = stream.next().await {
|
||||
let chunk = chunk.map_err(|e| ModelError::Download(e.to_string()))?;
|
||||
std::io::Write::write_all(&mut file, &chunk)?;
|
||||
received += chunk.len() as u64;
|
||||
on_progress(received, total);
|
||||
}
|
||||
drop(file);
|
||||
std::fs::rename(&tmp, &dest)?;
|
||||
Ok(())
|
||||
}
|
||||
|
||||
pub fn remove(id: &str) -> Result<(), ModelError> {
|
||||
let catalog = find(id).ok_or_else(|| ModelError::UnknownId(id.to_string()))?;
|
||||
let path = (catalog.dest)();
|
||||
if path.exists() {
|
||||
std::fs::remove_file(path)?;
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn list_returns_exactly_the_segmentation_and_embedding_pair() {
|
||||
let models = list();
|
||||
assert_eq!(models.len(), 2);
|
||||
assert!(models.iter().any(|m| m.id == "seg-pyannote-3.0"));
|
||||
assert!(models.iter().any(|m| m.id == "spk-eres2net"));
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn download_rejects_an_unknown_id_without_touching_disk() {
|
||||
let err = download("no-such-model", |_, _| {}).await.unwrap_err();
|
||||
assert!(matches!(err, ModelError::UnknownId(_)));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn remove_rejects_an_unknown_id() {
|
||||
let err = remove("no-such-model").unwrap_err();
|
||||
assert!(matches!(err, ModelError::UnknownId(_)));
|
||||
}
|
||||
}
|
||||
@@ -56,6 +56,14 @@ pub struct RecordingSession {
|
||||
/// can pick a different model than `Settings.whisper_model`).
|
||||
pub active_backend: Arc<StdMutex<models::BackendId>>,
|
||||
pub model_id: String,
|
||||
/// `None` when diarization models aren't installed yet (T4.7) — live
|
||||
/// provisional turns and the final post-stop pass are both skipped, same
|
||||
/// graceful-degradation treatment as a missing hardware backend (T4.3).
|
||||
pub diarizer: Option<Arc<dyn diarization::Diarizer>>,
|
||||
/// label ("S1"…) -> user-given display name, settable mid-recording
|
||||
/// (T4.4, FR-SPK-2). Never rewritten onto segments (FR-SPK-5); resolved
|
||||
/// at render/finalize time instead.
|
||||
pub speaker_names: Arc<StdMutex<std::collections::HashMap<String, String>>>,
|
||||
}
|
||||
|
||||
/// Wraps the tray icon so it can be looked up from commands to update its
|
||||
@@ -120,6 +128,7 @@ pub fn run() {
|
||||
commands::hardware_status,
|
||||
commands::set_preferred_backend,
|
||||
commands::list_models,
|
||||
commands::list_diarization_models,
|
||||
commands::download_model,
|
||||
commands::remove_model,
|
||||
commands::reprocess_transcript,
|
||||
@@ -128,6 +137,8 @@ pub fn run() {
|
||||
commands::delete_meeting,
|
||||
commands::update_notes,
|
||||
commands::export_meeting,
|
||||
commands::rename_speaker,
|
||||
commands::merge_speakers,
|
||||
commands::llm_status,
|
||||
commands::set_llm_provider,
|
||||
commands::generate_summary,
|
||||
|
||||
@@ -43,3 +43,12 @@ pub fn whisper_model_file(id: &str) -> PathBuf {
|
||||
pub fn whisper_model_path() -> PathBuf {
|
||||
whisper_model_file(DEFAULT_WHISPER_MODEL)
|
||||
}
|
||||
|
||||
/// Fixed filenames pending T4.7 (diarization model management/selection in Settings).
|
||||
pub fn diarization_segmentation_model_file() -> PathBuf {
|
||||
models_dir().join("seg-pyannote-3.0.onnx")
|
||||
}
|
||||
|
||||
pub fn diarization_embedding_model_file() -> PathBuf {
|
||||
models_dir().join("spk-eres2net.onnx")
|
||||
}
|
||||
|
||||
+124
-13
@@ -10,6 +10,7 @@ use async_trait::async_trait;
|
||||
use serde::{Deserialize, Serialize};
|
||||
use sqlx::sqlite::{SqliteConnectOptions, SqlitePool, SqlitePoolOptions};
|
||||
use sqlx::Row;
|
||||
use std::collections::HashMap;
|
||||
use std::path::PathBuf;
|
||||
use std::time::{SystemTime, UNIX_EPOCH};
|
||||
|
||||
@@ -82,6 +83,24 @@ pub trait Store: Send + Sync {
|
||||
async fn get_meeting(&self, id: &MeetingId) -> Result<Meeting, StoreError>;
|
||||
async fn delete_meeting(&self, id: &MeetingId) -> Result<(), StoreError>;
|
||||
async fn update_notes(&self, id: &MeetingId, markdown: &str) -> Result<(), StoreError>;
|
||||
/// Set (or create) a speaker's display name; works whether or not the
|
||||
/// meeting has finalized yet (T4.4, FR-SPK-2/5).
|
||||
async fn rename_speaker(
|
||||
&self,
|
||||
id: &MeetingId,
|
||||
label: &str,
|
||||
name: &str,
|
||||
) -> Result<(), StoreError>;
|
||||
/// Fold over-split speaker labels into one canonical label (T4.5,
|
||||
/// FR-SPK-3). Segment speaker IDs in storage are never rewritten
|
||||
/// (FR-SPK-5) — `get_meeting` resolves `from` labels to `into` when it
|
||||
/// reads segments/speakers back.
|
||||
async fn merge_speakers(
|
||||
&self,
|
||||
id: &MeetingId,
|
||||
from: &[String],
|
||||
into: &str,
|
||||
) -> Result<(), StoreError>;
|
||||
/// Full-text search across transcripts + notes (Phase 8, FR-SEARCH-1).
|
||||
async fn search(&self, query: &str) -> Result<Vec<MeetingListItem>, StoreError>;
|
||||
/// Startup reconcile: meetings with audio but no finalized transcript (FR-REL-1).
|
||||
@@ -116,6 +135,42 @@ impl SqliteStore {
|
||||
}
|
||||
}
|
||||
|
||||
impl SqliteStore {
|
||||
async fn upsert_speaker(
|
||||
&self,
|
||||
meeting_id: &MeetingId,
|
||||
label: &str,
|
||||
display_name: Option<&str>,
|
||||
) -> Result<(), StoreError> {
|
||||
sqlx::query(
|
||||
"INSERT INTO speakers (id, meeting_id, label, display_name)
|
||||
VALUES (?, ?, ?, ?)
|
||||
ON CONFLICT(meeting_id, label) DO UPDATE SET display_name = excluded.display_name",
|
||||
)
|
||||
.bind(uuid::Uuid::new_v4().to_string())
|
||||
.bind(meeting_id)
|
||||
.bind(label)
|
||||
.bind(display_name)
|
||||
.execute(&self.pool)
|
||||
.await?;
|
||||
Ok(())
|
||||
}
|
||||
}
|
||||
|
||||
/// Follows a `label -> merged_into` chain to its canonical label. Capped at 8
|
||||
/// hops so a stale/cyclic mapping (shouldn't happen, but merges are
|
||||
/// user-driven data) can't loop forever; merges normally resolve in one hop.
|
||||
fn resolve_canonical<'a>(label: &'a str, merge_map: &'a HashMap<String, String>) -> &'a str {
|
||||
let mut current = label;
|
||||
for _ in 0..8 {
|
||||
match merge_map.get(current) {
|
||||
Some(next) if next != current => current = next.as_str(),
|
||||
_ => break,
|
||||
}
|
||||
}
|
||||
current
|
||||
}
|
||||
|
||||
fn now_unix() -> i64 {
|
||||
SystemTime::now()
|
||||
.duration_since(UNIX_EPOCH)
|
||||
@@ -194,17 +249,8 @@ impl Store for SqliteStore {
|
||||
.await?;
|
||||
|
||||
for speaker in &s.speakers {
|
||||
sqlx::query(
|
||||
"INSERT INTO speakers (id, meeting_id, label, display_name)
|
||||
VALUES (?, ?, ?, ?)
|
||||
ON CONFLICT(meeting_id, label) DO UPDATE SET display_name = excluded.display_name",
|
||||
)
|
||||
.bind(uuid::Uuid::new_v4().to_string())
|
||||
.bind(id)
|
||||
.bind(&speaker.label)
|
||||
.bind(&speaker.display_name)
|
||||
.execute(&self.pool)
|
||||
.await?;
|
||||
self.upsert_speaker(id, &speaker.label, speaker.display_name.as_deref())
|
||||
.await?;
|
||||
}
|
||||
|
||||
let transcript = TranscriptFile {
|
||||
@@ -261,13 +307,24 @@ impl Store for SqliteStore {
|
||||
.ok_or_else(|| StoreError::NotFound(id.clone()))?;
|
||||
|
||||
let speaker_rows = sqlx::query(
|
||||
"SELECT label, display_name FROM speakers WHERE meeting_id = ? ORDER BY label",
|
||||
"SELECT label, display_name, merged_into FROM speakers WHERE meeting_id = ? ORDER BY label",
|
||||
)
|
||||
.bind(id)
|
||||
.fetch_all(&self.pool)
|
||||
.await?;
|
||||
// T4.5: rows with merged_into set are folded away — only canonical
|
||||
// speakers are returned, but their raw label still resolves through
|
||||
// `merge_map` below so folded segments render under the right one.
|
||||
let merge_map: HashMap<String, String> = speaker_rows
|
||||
.iter()
|
||||
.filter_map(|r| {
|
||||
let merged_into: Option<String> = r.get("merged_into");
|
||||
merged_into.map(|into| (r.get::<String, _>("label"), into))
|
||||
})
|
||||
.collect();
|
||||
let speakers: Vec<SpeakerInfo> = speaker_rows
|
||||
.iter()
|
||||
.filter(|r| r.get::<Option<String>, _>("merged_into").is_none())
|
||||
.map(|r| SpeakerInfo {
|
||||
label: r.get("label"),
|
||||
display_name: r.get("display_name"),
|
||||
@@ -276,11 +333,19 @@ impl Store for SqliteStore {
|
||||
.collect();
|
||||
|
||||
let folder = paths::meeting_dir(id);
|
||||
let segments = std::fs::read_to_string(folder.join("transcript.json"))
|
||||
let mut segments = std::fs::read_to_string(folder.join("transcript.json"))
|
||||
.ok()
|
||||
.and_then(|s| serde_json::from_str::<TranscriptFile>(&s).ok())
|
||||
.map(|t| t.segments)
|
||||
.unwrap_or_default();
|
||||
if !merge_map.is_empty() {
|
||||
// Resolution only touches this in-memory copy — transcript.json
|
||||
// on disk keeps its raw labels, regenerable and non-destructive
|
||||
// (FR-SPK-5), same as display names.
|
||||
for seg in &mut segments {
|
||||
seg.speaker = resolve_canonical(&seg.speaker, &merge_map).to_string();
|
||||
}
|
||||
}
|
||||
let notes_markdown = std::fs::read_to_string(folder.join("notes.md")).unwrap_or_default();
|
||||
|
||||
Ok(Meeting {
|
||||
@@ -322,6 +387,52 @@ impl Store for SqliteStore {
|
||||
Ok(())
|
||||
}
|
||||
|
||||
async fn rename_speaker(
|
||||
&self,
|
||||
id: &MeetingId,
|
||||
label: &str,
|
||||
name: &str,
|
||||
) -> Result<(), StoreError> {
|
||||
self.upsert_speaker(id, label, Some(name)).await
|
||||
}
|
||||
|
||||
async fn merge_speakers(
|
||||
&self,
|
||||
id: &MeetingId,
|
||||
from: &[String],
|
||||
into: &str,
|
||||
) -> Result<(), StoreError> {
|
||||
// Ensure the canonical label has a row, without clobbering a name it
|
||||
// may already have (plain upsert_speaker would overwrite display_name
|
||||
// with `None` if `into` hasn't been named yet).
|
||||
sqlx::query(
|
||||
"INSERT INTO speakers (id, meeting_id, label) VALUES (?, ?, ?)
|
||||
ON CONFLICT(meeting_id, label) DO NOTHING",
|
||||
)
|
||||
.bind(uuid::Uuid::new_v4().to_string())
|
||||
.bind(id)
|
||||
.bind(into)
|
||||
.execute(&self.pool)
|
||||
.await?;
|
||||
|
||||
for label in from {
|
||||
if label == into {
|
||||
continue; // merging a label into itself is a no-op
|
||||
}
|
||||
sqlx::query(
|
||||
"INSERT INTO speakers (id, meeting_id, label, merged_into) VALUES (?, ?, ?, ?)
|
||||
ON CONFLICT(meeting_id, label) DO UPDATE SET merged_into = excluded.merged_into",
|
||||
)
|
||||
.bind(uuid::Uuid::new_v4().to_string())
|
||||
.bind(id)
|
||||
.bind(label)
|
||||
.bind(into)
|
||||
.execute(&self.pool)
|
||||
.await?;
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
async fn search(&self, _query: &str) -> Result<Vec<MeetingListItem>, StoreError> {
|
||||
todo!("Phase 8 — FTS5 search")
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user