# LiveKit voice agent: 25 tests before the first real call

Run this checklist against the agent, tools, telephony route, and environment you plan to release. Start with deterministic session and tool tests. Then run the audio and failure lanes on representative devices and networks.

For every test, record:

- Build and prompt version
- LiveKit agent and dependency versions
- Model, voice, language, and turn-detection settings
- Phone/WebRTC route, region, codec, device, and network condition
- Expected result, actual result, evidence link, owner, and release decision

Use synthetic accounts and fixtures. Remove or protect personal data before saving recordings, transcripts, or traces.

## Lane 1: session opening and grounding

1. - [ ] **Opening and role.** The agent introduces itself as designed, identifies the allowed workflow, and does not overclaim its authority. Evidence: first audible output and session metadata.
2. - [ ] **Known caller context.** Approved context reaches the session once, maps to the right caller fixture, and excludes fields the agent does not need. Evidence: sanitized session input and prompt trace.
3. - [ ] **Missing context.** A missing account, locale, or workflow value produces a safe question or route instead of a guess. Evidence: turn trace and final state.
4. - [ ] **Unsupported intent.** A request outside the workflow is declined or routed using the approved boundary. Evidence: transcript, policy assertion, and route event.
5. - [ ] **Session close.** The agent confirms only completed actions, releases media, and leaves the expected durable state. Evidence: final audio, room/session events, and system-of-record diff.

## Lane 2: listening and conversation timing

6. - [ ] **Natural pause.** A caller pauses mid-thought without the agent cutting in or changing the partial intent. Evidence: audio plus turn-detection timestamps.
7. - [ ] **Barge-in.** Caller interruption stops playback inside the workflow budget and discards obsolete generation. Evidence: input start, playback stop, generation cancel, and next response.
8. - [ ] **Overlapping speech.** Nearby speech or brief overlap does not create a false tool action. Evidence: audio, turn events, and tool timeline.
9. - [ ] **Long silence.** The agent uses the approved reprompt count, explains the next step, and ends or routes safely. Evidence: silence durations and terminal state.
10. - [ ] **Self-correction.** “Tuesday, sorry, Thursday” reaches the final entity and tool argument as Thursday. Evidence: audio-to-entity-to-tool chain.

## Lane 3: tools and business state

11. - [ ] **Correct tool.** The permitted intent selects the right function with the complete schema. Evidence: request name, arguments, and source scenario.
12. - [ ] **Prerequisite order.** Identity, confirmation, consent, or disclosure happens before the consequential call when required. Evidence: ordered event timeline.
13. - [ ] **Tool delay.** A slow tool gets an accurate acknowledgement; the agent does not invent success while waiting. Evidence: tool timing and spoken claims.
14. - [ ] **Tool error.** Timeout, validation error, and service failure follow distinct approved recovery paths. Evidence: injected faults and final state.
15. - [ ] **Duplicate protection.** Retries, reconnects, and repeated caller confirmation do not create a second side effect. Evidence: idempotency key, request count, and after state.

## Lane 4: audio, telephony, and resilience

16. - [ ] **Target phone route.** The production SIP or telephony path connects with the intended region, codec, and caller-ID behavior. Evidence: call metadata and audio capture.
17. - [ ] **Noisy input.** Critical entities survive representative street, office, vehicle, or household noise without unsafe guessing. Evidence: stratified results and entity chain.
18. - [ ] **Packet loss and jitter.** Degraded network conditions do not cause clipped critical content, runaway repeats, or false completion. Evidence: network profile, audio, and event trace.
19. - [ ] **Disconnect and reconnect.** A media drop leaves a safe state and follows the approved reconnect, callback, or end behavior. Evidence: room events, tool state, and caller-facing message.
20. - [ ] **Dependency failure.** STT, model, TTS, or fallback failure is observable, bounded, and routed without exposing internal details. Evidence: fault injection and alert/trace.

## Lane 5: release evidence and abuse resistance

21. - [ ] **Critical entity set.** Names, dates, amounts, addresses, codes, and domain terms meet their lane-specific thresholds. Evidence: entity-level scorecard, not WER alone.
22. - [ ] **Policy misuse.** Prompt injection, false authority, and pressure to bypass rules do not unlock prohibited tools or data. Evidence: adversarial cases and tool absence.
23. - [ ] **Human handoff.** Trigger, explanation, routing, context briefing, two-way media, AI exit, and fallback all reach verified states. Evidence: transfer events and human receipt.
24. - [ ] **Observability and replay.** One session can be reconstructed from versions, audio/turns, tools, policy evidence, and final state without relying on memory. Evidence: exported trace and replay result.
25. - [ ] **Release gate.** Critical failures block the build, known gaps have owners, and the decision records exactly which routes, languages, conditions, and workflows were tested. Evidence: signed release record.

## Run record

| Test | Result | Evidence | Owner | Next action | Release impact |
| --- | --- | --- | --- | --- | --- |
| 1 | `[pass/fail/not run]` | `[link or run ID]` | `[role]` | `[action]` | `[ready/caution/blocked]` |

## Fictional example

In an invented pharmacy-support test, the refill tool waits eight seconds and then fails. The passing behavior is: the agent says it is still checking, reports that it could not complete the refill, preserves the original prescription state, and offers the approved human route. A cheerful “your refill is done” would fail tests 13, 14, 24, and 25 even if the transcript were perfect.

That example is fictional and contains no real patient or customer data.