iamdoubz
db244b116a
A phone that only rang no longer gets a coworker's recording, transcript or summary. The voicemail webhook leaves the transcript out. The saved Contrack token only goes to its own address. The probe trusts X-Forwarded-For from a local proxy only.
63 lines
3.6 KiB
Markdown
63 lines
3.6 KiB
Markdown
# 10 — Local AI
|
||
|
||
Owner, 2026-10-21: AI features that run on the PBX server (or the local network) only. No call
|
||
audio, transcript or prompt leaves it.
|
||
|
||
## 1. Endpoints
|
||
|
||
Three OpenAI-style APIs, set in Settings → AI (`ai.llm`, `ai.stt`, `ai.tts`), with `AI_*` env
|
||
values as the starting defaults (`services/settings.ts`):
|
||
|
||
| Setting | API used | Reference server |
|
||
|---|---|---|
|
||
| `ai.llm` {url, model} | `POST chat/completions` | Ollama (`/v1`) |
|
||
| `ai.stt` {url, model} | `POST audio/transcriptions` (multipart, `response_format=json`) | Speaches (Whisper) |
|
||
| `ai.tts` {url, model, voice} | `POST audio/speech` (`response_format=wav`) | Speaches (Kokoro) |
|
||
|
||
`ai/client.ts` refuses any address that isn't `localhost` or an IP in 127/8, 10/8, 172.16/12,
|
||
192.168/16, ::1, fc00::/7 or fe80::/10, written as an IP: no DNS, so no lookup can send data
|
||
elsewhere. Checked on save and before every request. Redirects are refused. CGNAT (100.64/10,
|
||
Tailscale) is refused too: it may cross the internet.
|
||
|
||
`POST /api/ai/test {kind, endpoint}` (admin) sends one tiny request (a chat, a second of silence,
|
||
one sentence of speech) with the values in the form, saved or not.
|
||
|
||
`deploy/ai/compose.yml` is its own stack: Speaches on 127.0.0.1:8000 with the GPU, and an
|
||
optional Ollama.
|
||
|
||
## 2. Voicemail transcripts (`ai.voicemailTranscripts`)
|
||
|
||
When a new message is indexed (04 §5.6), before the webhook, live event and email, the WAV goes to
|
||
`ai.stt` (180 s timeout). The text is stored in `voicemail_messages.transcript`, added to the
|
||
email ("What they said (automatic, may have mistakes)") and shown on the voicemail page. A failure
|
||
is logged and the message goes out without it.
|
||
|
||
## 3. Call transcripts and summaries (`ai.callTranscripts`, `ai.callSummaries`)
|
||
|
||
- Recording each side: `renderGlobals` writes `SP_REC_SIDES=1` while `ai.callTranscripts` is on and
|
||
`ai.stt` is set. `sp-sub-record` then runs `MixMonitor(<file>,br(<base>-in.wav)t(<base>-out.wav))`.
|
||
It always runs on the calling side, so `-in` is the caller and `-out` the other side
|
||
(`test/asterisk/rec-test.ts` checks it with a silent far side).
|
||
- The job (`ai/calls.ts`, every minute in `jobs.ts`, one recording at a time): recordings of the
|
||
last 2 days with `ai_done_at` null and made 20 s ago or more. No side files: done with
|
||
`ai_error`. Each side goes to `ai.stt` with `response_format=verbose_json`; the segments are
|
||
merged by start time into `recordings.transcript` [{side, at, text}]. With `ai.callSummaries` and
|
||
`ai.llm`: a summary (1–3 sentences, then "Action items:") from the transcript, cut at 12 000
|
||
characters, into `recordings.summary`. Then the side files are deleted. The AI stack unreachable
|
||
or timing out: the run stops and the call is retried next minute.
|
||
- API: `GET /api/recordings` has `summary`, `hasTranscript`, `aiError`, and `?q=` (ILIKE on the
|
||
transcript and summary, `%`/`_` taken literally). `GET /api/recordings/:id/transcript`: admins
|
||
and the people on the call (made or answered it, not only rang), like the audio. Deleting a recording, and retention, delete its side
|
||
files too.
|
||
|
||
## 4. Next (in order)
|
||
1. Typed prompts (Sounds): `ai.tts`, replacing the Piper plan in M12.
|
||
2. Type-to-speak in the browser phone: the api speaks the text, the browser mixes it into the
|
||
call.
|
||
3. AI receptionist: a destination type; chan_websocket (Asterisk 23.0+/22.6+) streams the call to
|
||
the api; utterance-at-a-time STT → LLM with tools (transfer, take a message, hours, hang up) →
|
||
TTS. Says it's an AI first; 911 never goes through it; any doubt falls back to the menu or a
|
||
person.
|
||
|
||
GPU sharing: live calls first; background transcripts one at a time.
|