# Production incident to permanent voice-agent regression

Use this worksheet after immediate safety and incident-response procedures are underway. It helps turn a failure into a privacy-safe, reproducible case and a release gate.

Do not copy a raw production call into a test repository. Keep original evidence under approved access controls. Build the regression from the smallest safely transformed trigger that still reproduces the failure.

## 1. Contain the impact

- Incident ID: `[stable internal ID]`
- Detected at: `[timestamp and timezone]`
- Workflow and release: `[workflow/version]`
- User impact: `[known impact without unnecessary personal data]`
- Unsafe side effects stopped: `[yes/no and how]`
- Rollback, feature flag, or route change: `[action]`
- Incident owner: `[role]`
- Customer/operations owner: `[role]`

Containment checklist:

- [ ] Stop repeatable unsafe effects.
- [ ] Preserve the affected versions and configuration.
- [ ] Record what is known separately from what is inferred.
- [ ] Follow notification, legal, security, and compliance procedures that apply.

## 2. Preserve controlled evidence

| Evidence | Original location | Access | Integrity proof | Retention | Owner |
| --- | --- | --- | --- | --- | --- |
| Audio | `[controlled store]` | `[approved roles]` | `[hash/version/event ID]` | `[rule]` | `[role]` |
| Conversation events | `[store]` | `[roles]` | `[proof]` | `[rule]` | `[role]` |
| Tool requests/responses | `[store]` | `[roles]` | `[proof]` | `[rule]` | `[role]` |
| Prompt/model/config | `[store]` | `[roles]` | `[proof]` | `[rule]` | `[role]` |
| Before/after business state | `[store]` | `[roles]` | `[proof]` | `[rule]` | `[role]` |

## 3. Reconstruct one timeline

Normalize timestamps before explaining the failure.

| Time | Layer | Event | Observed evidence | Expected event | Gap |
| --- | --- | --- | --- | --- | --- |
| `[time]` | `[audio/turn/model/tool/media/state]` | `[event]` | `[evidence ID]` | `[expected]` | `[difference]` |

- First confirmed divergence: `[event ID and time]`
- Upstream conditions: `[conditions that made it possible]`
- Downstream effects: `[effects caused by the divergence]`
- Root cause: `[evidence-backed cause, not a symptom]`
- Contributing factors: `[separate from root cause]`

## 4. Design the privacy transformation

| Original element | Needed to reproduce? | Transformation | Proof meaning is preserved | Approved by |
| --- | --- | --- | --- | --- |
| `[name, voice, account, address, health/card detail, timing, noise]` | `[yes/no]` | `[remove/redact/tokenize/synthesize/retain under control]` | `[replay comparison]` | `[role]` |

Safety checks:

- [ ] Direct identifiers are removed unless a controlled exception is approved.
- [ ] Indirect combinations cannot easily re-identify the caller.
- [ ] Synthetic replacement preserves the pronunciation, timing, ambiguity, or acoustic property that triggers the bug.
- [ ] Tool fixtures and business state use non-production accounts.
- [ ] Test logs, reviews, exports, and backups follow the new fixture's controls.

## 5. Build the minimal replay

- Regression ID: `[stable ID]`
- Minimal caller behavior: `[utterance/turn pattern]`
- Required acoustic condition: `[noise/codec/timing/overlap]`
- Required prior state: `[fixture state]`
- Injected tool or dependency condition: `[response/delay/error]`
- Affected version expected result: `[must reproduce the failure]`
- Fixed version expected result: `[must show the safe behavior]`
- Repeat count and variance rule: `[count]`

## 6. Add layered assertions

| Assertion | Evidence | Severity | Affected version | Fixed version | Owner |
| --- | --- | --- | --- | --- | --- |
| `[one observable condition]` | `[audio/event/tool/final state]` | `[severity]` | `[pass/fail]` | `[pass/fail]` | `[role]` |

Include assertions for the first divergence, the safe recovery, and the final system state. Add neighboring cases so the fix cannot merely recognize one transcript.

Neighboring cases:

- [ ] Same meaning with different wording
- [ ] Same critical entity with different pronunciation
- [ ] Earlier and later interruption timing
- [ ] Tool success, delay, timeout, and duplicate response
- [ ] Different supported device, route, or language stratum
- [ ] Unsupported case that must recover safely

## 7. Make it a release gate

- Suites that include this regression: `[suite IDs]`
- Builds and workflows it blocks: `[scope]`
- Pass threshold and repeat rule: `[rule]`
- Flake treatment: `[investigation rule, not automatic ignore]`
- Fixture owner: `[role]`
- Review/retention date: `[date]`
- Release decision: `[ready/caution/blocked]`
- Open risk and owner: `[risk/owner]`

## Worked fictional example

This example is invented. A fictional courier agent hears “flat fourteen” correctly, but a caller interruption arrives as the address-update tool is starting. The old generation is discarded while the tool request still runs, so the agent sends a second update after the caller corrects the number to forty. The durable address becomes 40, but two customer notifications are sent.

The minimal regression keeps the interruption timing, tool delay, correction, and duplicate-side-effect risk. It replaces the voice, account, and address with synthetic fixtures. The critical assertion is that one authorized address update and one notification occur after the final correction. No real caller data belongs in the fixture.

Voxeval recommends closing the incident-learning loop only when the affected version reproduces the case, the fixed version passes it repeatedly, and future releases cannot skip the gate.