Restaurant voice AI needs an order-state engine, not a clever prompt

“Two large pepperoni pizzas. Actually, make one medium. No olives on that one. Add olives to the other one. Wait, which one has extra cheese?” That is not a prompt-writing problem. It is a cart problem.

Restaurant operators looking at phone-order automation are usually sold the voice. Builders know the hard part arrives after the first correction. This guide shows how to put a versioned order-state engine between the conversation and the POS, so a caller can change their mind without corrupting the ticket.

Peak-hour value is real. So is the state problem.

AdValorem’s May 2026 restaurant voice AI analysis frames the use case around calls that arrive during the rush, when staff cannot pick up. That is a sensible buyer problem, although the public excerpt does not prove a universal ROI number. It does identify the jobs the system must handle: phone orders, reservations, common questions, payment links, and edge-case routing.

Restaurant voice vendors have already announced real deployments. SoundHound’s Jersey Mike’s release described an initial rollout at 50 locations, trained around the menu and connected to phone ordering. Treat deployment claims as company-reported evidence, not an independent accuracy study.

Against that evidence, the engineering question is concrete: what exactly is the order after the caller changes three things?

Scroll diagram horizontally on smaller screens.

The transcript is evidence, not the cart

Do not rebuild the order by rereading the whole transcript on every turn. Maintain explicit state instead:

{
  "order_version": 7,
  "items": [
    {
      "line_id": "line_1",
      "catalog_item": "pizza_pepperoni",
      "size": "large",
      "quantity": 1,
      "modifiers": ["extra_cheese", "no_olives"]
    },
    {
      "line_id": "line_2",
      "catalog_item": "pizza_pepperoni",
      "size": "medium",
      "quantity": 1,
      "modifiers": ["olives"]
    }
  ],
  "status": "proposed"
}

Every correction becomes an operation against a stable line_id. Change line_2.size from large to medium instead of issuing the vague instruction “change the pizza.”

The state also needs provenance. Record which utterance or tool result created each line, which operation last changed it, and whether the value is proposed, validated, confirmed, or committed. When two items become identical, provenance may be the only way to understand “the one I changed.”

Keep the transcript as linked evidence. It remains useful for resolving references, reviewing disputes, and improving language handling. It should not become the source of truth after every turn. Rebuilding the cart from the full transcript asks a stochastic component to reproduce every earlier decision while also interpreting the newest one.

A reducer gives the state engine a clean boundary. It accepts a typed operation and the current cart, checks that the target exists and the change is permitted, then returns a new version. The language layer can propose SET_SIZE(line_2, medium). The reducer decides whether medium is a valid variation, whether the line is still editable, and what downstream validation must run.

Use an operation log

Store each caller change as an explicit operation:

ADD line_1 pizza_pepperoni size=large
ADD line_2 pizza_pepperoni size=large
SET line_2 size=medium
SET line_1 olives=false
SET line_2 olives=true
SET line_1 extra_cheese=true

The operation log provides an audit trail and clean undo path, explains the current cart, supplies regression evidence, and protects untouched fields from silent model rewrites.

When the caller says “remove the second pizza,” resolve the reference and delete line_2.

Make operations small and explicit. REPLACE_ITEM should not silently copy modifiers that are invalid for the new catalog item. SET_FULFILLMENT_MODE may need to clear a delivery address and recheck location availability. CHANGE_LOCATION may invalidate prices, taxes, and the entire menu. Each operation should declare the derived checks it triggers.

The log is also the safest place to handle undo. “Put the olives back” can reverse the last modifier removal on the referenced line instead of regenerating an older cart snapshot from text. If several operations occurred after the removal, the engine can decide whether a direct inverse remains valid or whether the agent must clarify.

Do not expose internal line numbers to callers unless the product deliberately uses them. A caller says “the second large pizza” or “the one for my daughter.” The conversation layer resolves that reference to candidate line IDs, and the state engine accepts the operation only when one valid target remains. Several matches should produce a clarification, not a random choice.

Catalog identity comes before conversation flair

Map spoken choices to catalog objects by distinguishing the item, variation or size, quantity, modifier group and selection, allowed combinations, availability by location and time, and price source.

“Extra spicy” may be a modifier at one location and free text at another. “The usual” may require customer history and still need confirmation. “Half olives” may map to a structured placement option or may be unsupported. The language model can propose a mapping, but the catalog service validates it. If no valid mapping exists, the agent asks or routes instead of inventing a POS item.

Menu data needs a version and a location. A national product name can map to different variation IDs, prices, modifier groups, or availability by store. The cart should retain the catalog version used for validation, then revalidate if the caller changes the location or if the order sits open long enough for availability to change.

Separate free-form notes from structured modifiers. “Cut in half” might be an approved kitchen note for one item and unsupported for another. Notes should never act as a back door around allergen, price, or catalog rules. Decide which lanes permit them, where they appear on the ticket, and whether a human must review them.

Keep lifecycle states and commit uncertainty separate

Proposed means the caller has said it, but the system has not yet validated availability, pricing, or policy. Proposed items can change through clarification and should never be presented as accepted by the restaurant.

Confirmed means the caller heard and approved a complete summary. High-risk details such as allergens, substitutions, payment amount, or pickup location may need their own confirmation rule instead of inheriting the cart’s general status.

Committed means the POS accepted the order and the system re-read the durable state. Store the committed version and compare it with the confirmed cart before announcing success, since an accepted request can still contain a substitution, stale price, or missing modifier.

Use uncertain as a commit branch for the ugly middle. Networks fail after writes, providers return ambiguous errors, and webhook confirmation can lag. In that state, block a second blind commit, keep the caller informed, and start reconciliation. Depending on the restaurant’s process, the safe outcome may be a human check, a callback, or a clearly labeled pending order.

These boundaries create three non-negotiable rules: do not say “your order is placed” from proposed state, do not charge from an unconfirmed cart, and do not assume an HTTP success means the kitchen ticket matches the conversation.

Read back the diff, then the final cart

After a correction, acknowledge the change:

“Got it. The second pizza is now medium.”

Before commit, summarize the full order:

“I have one large pepperoni with extra cheese and no olives, plus one medium pepperoni with olives.”

These two read-backs are not needless repetition. They are state checks generated from the structured cart, not from memory. If the spoken summary cannot be reproduced from the cart, stop.

Choose read-back depth from risk and change. After a simple modifier correction, read the diff so the caller can catch the immediate mistake. Before payment or final commit, read the complete cart with quantities, key modifiers, fulfillment mode, location, and total. For a long order, group items naturally but do not compress away details the kitchen needs.

Record what the caller actually heard. If they interrupt the final summary, the order should remain unconfirmed unless your policy defines a narrower confirmation path. A transcript showing the full generated summary is not evidence that the caller approved the undelivered portion.

Version the cart before touching the POS

Square’s current Orders API supports creating, retrieving, and updating orders. Its update guidance uses current order versions for optimistic concurrency and idempotency keys for updates.

Your POS may work differently, but the principles travel:

• Read the current order version.
• Apply a sparse, validated change.
• Use a new idempotency key for the intended mutation.
• Reject stale writes.
• Re-read the committed order.
• Compare it with the confirmed voice state.

To keep the conversation from holding a private fantasy cart while the POS holds something else, use a clear commit protocol. Freeze the confirmed cart version, create the POS mutation with an idempotency key tied to that intended commit, and wait for a result. If the response succeeds, read the order back and compare every required field. If it times out, query by idempotency key or order reference before retrying. “No response” is an uncertain state, not proof that nothing happened.

When the POS rejects a stale version, fetch the latest order and compare it with the voice state. A staff member may have changed the ticket, a webhook may have arrived late, or another retry may already have committed. Apply only a reviewed, sparse diff against the current record. Replacing the whole order can erase work performed outside the voice session.

The caller-facing language should follow the commit state. “I am placing the order now” is different from “your order is confirmed.” If reconciliation is still running, say only what is true and move to the approved recovery when the wait exceeds the lane’s budget.

A fictional rush-hour failure

Consider an illustrative rush-hour case that did not happen. A caller orders three masala dosas and one plain dosa. The state engine keeps four stable lines because a later correction may refer to one item, even when several share a product name. When the caller says, “Make one of the masala dosas plain for the kid,” the system changes line_2 from masala to plain and records that provenance; line_4 remains the original plain dosa.

The caller then says, “Actually remove the first plain one.” The model deletes the newly converted line_2, while the caller meant the original line_4. Stable identity preserves the distinction, but the phrase still leaves two plausible candidates. The agent should clarify instead of choosing one.

Test the state engine without audio first

Run deterministic operation sequences that cover:

• Identity and mutation: add two identical items, modify only the second, remove by position, undo the last change, and apply an unavailable modifier.
• Catalog and confirmation: change location mid-order and interrupt during final confirmation.
• POS concurrency: receive a stale version, retry after a timeout, then commit and compare the POS record.

Once those pass deterministically, add speech variation, accents, noise, code-switching, and interruptions.

Voxeval’s opinion is blunt: do not spend expensive audio-eval cycles proving that a broken cart reducer is broken.

Add contract tests against a POS sandbox or controlled double. Exercise catalog refresh, version conflict, timeout after commit, duplicate webhook, rejected modifier, tax change, and staff edit. The state-engine tests should assert both the final order and the operations that produced it. Accidental final correctness can hide an unsafe broad mutation.

Then run complete voice simulations with real token streaming and phone audio. Interrupt during reference resolution, validation, confirmation, and commit. Check that the delivered speech, proposed cart, confirmed cart, and durable POS record remain aligned at every boundary.

The release gate lives at the kitchen ticket

Score three kinds of evidence:

• Ticket fidelity: exact line-item, modifier, and quantity match, with the correct price and tax source.
• Journey correctness: pickup or delivery state and confirmation coverage.
• System safety: duplicate-write prevention, final POS state, and safe handoff when mapping fails.

The state engine supplies the oracle for the restaurant correction regression pack. Its outcomes also feed a less flattering but more useful cost-per-resolution model.

Review failures with restaurant operations, not only engineers. Menu managers can spot invalid modifier combinations, store teams know which notes the kitchen can act on, and finance can define how price changes or refunds should appear. The state machine turns those rules into testable behavior, but the rules still need an accountable owner.

Release by location and menu version with a rollback path. Watch unknown catalog mappings, clarification rate, correction rate, uncertain commits, duplicate-write prevention, and ticket mismatches. If one location has a different menu configuration, a blended accuracy score across every restaurant will hide it.

The restaurant order-state design sheet gives product, engineering, menu operations, and QA one place to define those rules. Build the cart engine first, then let the voice sound charming.

Subscribe to Voxeval for practical voice AI architecture and evaluation notes.