Skip to content
All posts
Product 6 min read

AI agents that know when to stop

Deflection rate is the wrong metric. What to measure instead, and how to build for escalation.

At

AI team

Engineering ·

Every AI support demo optimises for deflection rate. It is the wrong number to lead with, and optimising for it produces agents that are confidently unhelpful.

Why deflection rate misleads

An agent that answers everything has a perfect deflection rate and may be wrong half the time. The customer who gave up and stopped replying counts as deflected. So does the one who got a plausible answer that was incorrect.

Better metrics

  • Resolution rate — did the customer's problem actually go away, measured by whether they came back.
  • Escalation quality — when it handed over, did the human have what they needed?
  • Confidence calibration — when the agent was confident, was it right?
  • Repeat contact rate within 48 hours — the honest signal that an answer did not land.

Building for escalation

An agent should hand over early and with context. Ours carries the full conversation, the tools it called, what they returned, and why it stopped. The human picks up mid-thought rather than starting over — which is the difference between escalation feeling like progress or like being passed around.

Grounding reduces confident wrongness. It does not eliminate it.

Set a confidence threshold, watch what your agent says in the first weeks, and treat a low escalation rate as something to investigate rather than celebrate.

Where AI genuinely wins

Questions with a factual answer that already exists in your systems. Order status, stock, booking availability, opening hours. The agent is not being clever — it is being fast and available at 2am, which is enough.

Get the next one

One email a month with new posts and notable product changes.

More posts