ai: call transcripts (each side recorded, merged by time) and summaries; search on Recordings
This commit is contained in:
1 parent
d9f9c6e444
commit
4d331fb70e
56 files changed
+5186
-37
No files matched your search
+22
-6
@@ -32,13 +32,29 @@ When a new message is indexed (04 §5.6), before the webhook, live event and ema
|
||||
email ("What they said (automatic, may have mistakes)") and shown on the voicemail page. A failure
|
||||
is logged and the message goes out without it.
|
||||
|
||||
## 3. Next (in order)
|
||||
1. Call recording transcripts and summaries: record each side (MixMonitor `r()`/`t()`), so who
|
||||
said what needs no speaker detection; summary and action items from `ai.llm`; search.
|
||||
2. Typed prompts (Sounds): `ai.tts`, replacing the Piper plan in M12.
|
||||
3. Type-to-speak in the browser phone: the api speaks the text, the browser mixes it into the
|
||||
## 3. Call transcripts and summaries (`ai.callTranscripts`, `ai.callSummaries`)
|
||||
|
||||
- Recording each side: `renderGlobals` writes `SP_REC_SIDES=1` while `ai.callTranscripts` is on and
|
||||
`ai.stt` is set. `sp-sub-record` then runs `MixMonitor(<file>,br(<base>-in.wav)t(<base>-out.wav))`.
|
||||
It always runs on the calling side, so `-in` is the caller and `-out` the other side
|
||||
(`test/asterisk/rec-test.ts` checks it with a silent far side).
|
||||
- The job (`ai/calls.ts`, every minute in `jobs.ts`, one recording at a time): recordings of the
|
||||
last 2 days with `ai_done_at` null and made 20 s ago or more. No side files: done with
|
||||
`ai_error`. Each side goes to `ai.stt` with `response_format=verbose_json`; the segments are
|
||||
merged by start time into `recordings.transcript` [{side, at, text}]. With `ai.callSummaries` and
|
||||
`ai.llm`: a summary (1–3 sentences, then "Action items:") from the transcript, cut at 12 000
|
||||
characters, into `recordings.summary`. Then the side files are deleted. The AI stack unreachable
|
||||
or timing out: the run stops and the call is retried next minute.
|
||||
- API: `GET /api/recordings` has `summary`, `hasTranscript`, `aiError`, and `?q=` (ILIKE on the
|
||||
transcript and summary, `%`/`_` taken literally). `GET /api/recordings/:id/transcript`: admins
|
||||
and the people on the call, like the audio. Deleting a recording, and retention, delete its side
|
||||
files too.
|
||||
|
||||
## 4. Next (in order)
|
||||
1. Typed prompts (Sounds): `ai.tts`, replacing the Piper plan in M12.
|
||||
2. Type-to-speak in the browser phone: the api speaks the text, the browser mixes it into the
|
||||
call.
|
||||
4. AI receptionist: a destination type; chan_websocket (Asterisk 23.0+/22.6+) streams the call to
|
||||
3. AI receptionist: a destination type; chan_websocket (Asterisk 23.0+/22.6+) streams the call to
|
||||
the api; utterance-at-a-time STT → LLM with tools (transfer, take a message, hours, hang up) →
|
||||
TTS. Says it's an AI first; 911 never goes through it; any doubt falls back to the menu or a
|
||||
person.
|
||||
|
||||
Reference in new issue
Block a user