0 Fork 0

Files

T

iamdoubz db244b116a security: recordings, webhooks, Contrack tokens, probe (08 §7 #3 #4 #6 #7 )

A phone that only rang no longer gets a coworker's recording, transcript or
summary. The voicemail webhook leaves the transcript out. The saved Contrack
token only goes to its own address. The probe trusts X-Forwarded-For from a
local proxy only.

2026-10-28 16:01:56 -05:00

3.6 KiB

Raw Blame History

10 — Local AI

Owner, 2026-10-21: AI features that run on the PBX server (or the local network) only. No call audio, transcript or prompt leaves it.

1. Endpoints

Three OpenAI-style APIs, set in Settings → AI (ai.llm, ai.stt, ai.tts), with AI_* env values as the starting defaults (services/settings.ts):

Setting	API used	Reference server
`ai.llm` {url, model}	`POST chat/completions`	Ollama (`/v1`)
`ai.stt` {url, model}	`POST audio/transcriptions` (multipart, `response_format=json`)	Speaches (Whisper)
`ai.tts` {url, model, voice}	`POST audio/speech` (`response_format=wav`)	Speaches (Kokoro)

ai/client.ts refuses any address that isn't localhost or an IP in 127/8, 10/8, 172.16/12, 192.168/16, ::1, fc00::/7 or fe80::/10, written as an IP: no DNS, so no lookup can send data elsewhere. Checked on save and before every request. Redirects are refused. CGNAT (100.64/10, Tailscale) is refused too: it may cross the internet.

POST /api/ai/test {kind, endpoint} (admin) sends one tiny request (a chat, a second of silence, one sentence of speech) with the values in the form, saved or not.

deploy/ai/compose.yml is its own stack: Speaches on 127.0.0.1:8000 with the GPU, and an optional Ollama.

2. Voicemail transcripts (`ai.voicemailTranscripts`)

When a new message is indexed (04 §5.6), before the webhook, live event and email, the WAV goes to ai.stt (180 s timeout). The text is stored in voicemail_messages.transcript, added to the email ("What they said (automatic, may have mistakes)") and shown on the voicemail page. A failure is logged and the message goes out without it.

3. Call transcripts and summaries (`ai.callTranscripts`, `ai.callSummaries`)

Recording each side: renderGlobals writes SP_REC_SIDES=1 while ai.callTranscripts is on and ai.stt is set. sp-sub-record then runs MixMonitor(<file>,br(<base>-in.wav)t(<base>-out.wav)). It always runs on the calling side, so -in is the caller and -out the other side (test/asterisk/rec-test.ts checks it with a silent far side).
The job (ai/calls.ts, every minute in jobs.ts, one recording at a time): recordings of the last 2 days with ai_done_at null and made 20 s ago or more. No side files: done with ai_error. Each side goes to ai.stt with response_format=verbose_json; the segments are merged by start time into recordings.transcript [{side, at, text}]. With ai.callSummaries and ai.llm: a summary (1–3 sentences, then "Action items:") from the transcript, cut at 12 000 characters, into recordings.summary. Then the side files are deleted. The AI stack unreachable or timing out: the run stops and the call is retried next minute.
API: GET /api/recordings has summary, hasTranscript, aiError, and ?q= (ILIKE on the transcript and summary, %/_ taken literally). GET /api/recordings/:id/transcript: admins and the people on the call (made or answered it, not only rang), like the audio. Deleting a recording, and retention, delete its side files too.

4. Next (in order)

Typed prompts (Sounds): ai.tts, replacing the Piper plan in M12.
Type-to-speak in the browser phone: the api speaks the text, the browser mixes it into the call.
AI receptionist: a destination type; chan_websocket (Asterisk 23.0+/22.6+) streams the call to the api; utterance-at-a-time STT → LLM with tools (transfer, take a message, hours, hang up) → TTS. Says it's an AI first; 911 never goes through it; any doubt falls back to the menu or a person.

GPU sharing: live calls first; background transcripts one at a time.

3.6 KiB Raw Blame History Unescape Escape