Reference architecture
An AI-enabled contact center on Amazon Connect
The shape most telephony projects converge on, whatever the industry. Written for the engineer who has to build or inherit it, not for the person approving the budget.
The whole system, on one page
Two properties of this drawing matter more than the boxes. First, every path has a fallback that ends somewhere a caller can still be served — recognition failure returns to the keypad menu, fulfilment failure transfers to a person. Second, the boundary is drawn deliberately: the aggregators, partner APIs, and notification services outside it are the components you do not control, and every one of them will be unavailable at some point.
Component by component
- 01
Telephony ingress
A carrier DID or SIP trunk terminates on Amazon Connect. Two decisions matter here and both are reversible only with effort: who the carrier is, and whether numbers are ported or newly provisioned. Port in batches with a rollback per batch — a single big-bang port is the one step in a voice project with no undo. Keep a second carrier configured but idle if the line is revenue-critical.
- 02
Contact flow
Connect owns routing, hours of operation, holiday handling, queue selection, and recording. Keep flows small and composable rather than one flow with forty branches; a flow you cannot read is a flow nobody will change. Where an existing DTMF menu is being modernised, it stays in place as the failure path — every new branch must be able to fall back into it.
- 03
Speech and intent
Amazon Lex handles the intents worth modelling: those with a bounded set of slots and a clear success condition. Slot validation happens in Lex, not downstream. The confidence threshold is tuned against real call recordings — the default will either hand off constantly or confidently mishear addresses, and which one you get depends on your callers, not on the documentation. Recognition is per-language and the models are not equally good in all of them, so the keypad fallback matters more in some languages than others.
- 04
LLM fallback
Below the threshold, or for the long tail no one wants to model, an Amazon Bedrock agent takes the turn with tool access scoped to specific API calls. It gets a system prompt that states what it must never do, a token and time budget, and logging of prompts and responses after redaction. It does not get write access to anything financial or clinical without a confirmation step read back to the caller.
- 05
Fulfilment
Lambda is where the call meets your systems. Every handler is idempotent, because a caller who says nothing for four seconds may get retried. Timeouts are set below the caller's patience, not above it: a lookup that takes six seconds must return a holding response, not silence. Any dependency that can be slow gets a circuit breaker and a spoken fallback.
- 06
System of record
The interaction is written before the call ends, not queued for a batch afterwards. That means the record survives a dropped call, and the audit trail shows what the caller was told rather than what the system intended to tell them. Writes carry a correlation ID that also appears in the call recording metadata, so one identifier connects the audio, the transcript, and the row.
- 07
Downstream hand-off
Partner APIs, government portals, billing, and notifications sit outside the trust boundary. Hand-offs are batched on their schedule with retries, and — critically — rejections come back into a reconciliation queue with the reason attached. This is the component most often left out of the first version and most often responsible for the first month of manual work after launch.
- 08
Agent escalation
When a caller asks for a person, or the bot gives up, the transfer carries context: what was collected, what was attempted, and the transcript so far. An escalation that makes the caller repeat themselves costs more goodwill than the automation saved. Contact attributes set during the flow drive the screen pop in the agent desktop.
- 09
Data boundary
Everything inside the dashed line runs in your cloud account, in the region you choose. Recordings and transcripts land in your S3 with server-side encryption and a stated retention policy. Conversation logs are off or redacted before write. Nothing carrying regulated data reaches a third party without an agreement covering it. In healthcare this boundary is where the BAA applies; in payments it is where PCI scope ends; under GDPR it is where residency is argued.
- 10
Observability
Contact Lens for transcription and sentiment, CloudWatch for metrics and alarms. Alarm on leading indicators — queue depth, recognition failure rate, fulfilment error rate, downstream acknowledgement rate — rather than on abandoned calls, which only tell you the damage is already done. A containment dashboard makes the automation's value arguable with numbers instead of opinions.
Amazon Connect or Twilio
Neither is better in general and both are inexpensive to run. The question is whether humans answer queued calls. If they do, Connect saves months you would otherwise spend rebuilding a queue and an agent desktop. If voice is a feature inside your own product, Twilio gives you control that a flow editor will fight you for.
| Dimension | Amazon Connect | Twilio |
|---|---|---|
| Shape | A contact center product — queues, routing, agent desktop, recording, analytics arrive assembled | Programmable primitives you compose yourself, with nothing assumed about how you use them |
| Human agents | Built in: routing profiles, agent states, after-contact work, supervisor barge | You build or buy the queue and the desktop (Flex, or your own) |
| Programmability | Flow editor plus Lambda. Powerful, but complex logic in the editor gets unreadable fast | Your code, your language, your repository. No editor to fight |
| Embedded in a product | Works, but you are carrying contact center machinery you may not use | The natural fit — voice as a feature of your own application |
| Pricing model | Per minute of usage, so idle agents cost little | Per minute plus per-service charges, priced per component you use |
| Best fit | People answer queued calls | Software answers the call, or your product places it |
Lex or an LLM agent
The interesting answer is usually both, split by task. An LLM in front of everything is slower, costlier, and less predictable than an intent model for the requests that make up most of your volume.
| Situation | Use | Why |
|---|---|---|
| Bounded task with clear slots — order number, date, account ID | Lex | Deterministic, cheaper, and faster to first response. Validation belongs in the slot, not in a prompt |
| Caller phrases the same request forty different ways | Lex | This is what intent training is for. An LLM here adds latency and cost for no accuracy gain |
| Open question spanning several documents or policies | LLM agent | Retrieval plus synthesis, with citations back to the source so an answer can be checked |
| Multi-step request needing several system calls and judgement between them | LLM agent | Tool use with a scoped permission set. Set a step limit so a loop cannot run up the call |
| Anything irreversible — payment, cancellation, clinical instruction | Neither, alone | Explicit confirmation read back to the caller, or a transfer to a human. Automate the retrieval, not the commitment |
The latency budget
A phone call has no spinner. Every stage below happens between the caller finishing a sentence and hearing a reply, and the budget is spent whether or not it was planned.
| Stage | Budget | Note |
|---|---|---|
| Caller stops speaking → endpoint detected | 300–500 ms | The single largest slice, and the one most often left at its default |
| Speech to text, final result | 100–300 ms | Streaming recognition overlaps this with the caller still talking |
| Business lookup against your API | 100–400 ms | Prefetch on the likely intent while the caller is still speaking |
| Model response, first token | 200–900 ms | The wide range is why an LLM in the hot path needs a filler strategy |
| Text to speech, first audio | 100–300 ms | Stream audio out; do not wait for the whole utterance to synthesise |
| Total perceived gap | under 800 ms | Past roughly 800 ms the caller assumes the line has dropped and starts talking over it |
These stages overlap in a good implementation and stack in a bad one. Streaming recognition, prefetching the likely lookup, and streaming synthesis are what turn a 2-second gap into a 700ms one — not a faster model.
What moves the running cost
Platform rates change often enough that quoting them here would be misleading — take the current numbers from the AWS and Twilio pricing pages. What does not change is which levers matter, and per-minute pricing behaves very differently from per-seat licensing when volume is spiky.
| Driver | Effect |
|---|---|
| Talk minutes | The primary variable on a per-minute platform. Automation that shortens calls shows up here first |
| Agent idle time | Costs nothing per-minute, which is where Connect beats per-seat licensing for spiky volume |
| Recording and transcription | Storage plus per-minute analysis. Retention policy is a cost lever, not just a compliance one |
| Model usage | Charged per token, so prompt size matters as much as call volume. Cap context deliberately |
| Numbers and carrier | Per DID per month plus per-minute carrier charges — small, but it scales with the number inventory nobody audits |
| Idle infrastructure | Near zero on a serverless fulfilment path, and the reason to avoid always-on compute for bursty voice traffic |
Want this reviewed against your stack?
Send the current call flows and the systems they touch. A first pass usually finds the two or three decisions that will decide whether the project is straightforward or painful, and that conversation costs nothing.
Start a project