Guides

What should stay with a person: calls and decisions an AI agent should not handle alone

A list of the situations to keep with a person, what the assistant should do instead, and four questions to ask before you automate any task.

Last updated ProductDesignerQAMarkdown version

Why decide this in advance

An assistant is fast and always on, which is exactly why it needs limits. Courts have held companies responsible for what their bots say (Air Canada, Hamm), and some actions cannot be undone. Decide before launch which situations go to a person, write the rule in code and policy, not only in the prompt, and test it.

Design guidance, not legal advice

The right list depends on your industry, country and risk appetite. Take it to your legal and compliance teams.

The ten situations

1. Risk to safety, or signs of crisis

  • Why a person: A wrong or slow answer can cause harm
  • The assistant: Says it is an AI, shares the emergency or support option for the user's place, hands off at once. It does not try to counsel

2. Advice about one person's legal, medical or financial case

  • Why a person: Needs a qualified person who is accountable for the advice
  • The assistant: Gives general, sourced information, says it is not advice for their case, offers a person

3. Exceptions, goodwill gestures, anything outside written policy

  • Why a person: A bot cannot invent a commitment
  • The assistant: Explains the standard rule from the source and sends the case to someone who can decide

4. Money or data actions above a limit, or that cannot be undone

  • Why a person: Irreversible actions need a second check
  • The assistant: Does the safe part (collects and verifies details) and queues the action for approval

5. Rights requests and formal complaints (access, deletion, chargebacks, regulator complaints)

  • Why a person: Deadlines and legal duties attach to them
  • The assistant: Logs it with the time, confirms receipt, hands it to the owner. It does not refuse or settle it alone

6. Claims about the company itself: qualifications, prices, availability, guarantees

  • Why a person: A false claim is the company's own statement (see the Hamm case)
  • The assistant: Answers only from a maintained fact source. Otherwise says it cannot confirm and points to the page or a person

7. Suspected account takeover or fraud

  • Why a person: A wrong call helps the attacker or locks out the owner
  • The assistant: Stops the sensitive action, verifies through a secure channel, hands to security

8. A user who is vulnerable, distressed or struggling with the interface

  • Why a person: The standard flow can mislead or pressure
  • The assistant: Slows down, offers a person early, avoids upsell

9. Anger, or a second failed attempt

  • Why a person: Loops destroy trust
  • The assistant: Offers a person and passes a summary

10. The user asks for a human

  • Why a person: Respect the request
  • The assistant: Hands off. It does not argue

Four questions before you automate a task

  1. Can it be undone? If not, add approval.
  2. Is the limit enforced in code? A line in the prompt is not a limit (module 10).
  3. Does the answer come from a current source the assistant can cite? If not, it will guess (module 6).
  4. Would you be comfortable seeing the transcript quoted in a complaint? If not, keep a person in the loop.

If any answer is "no", keep a person in the loop for that task.

Hand off in a way that works

  • Offer the person early, and say plainly what happens next and how long it may take.
  • Pass the summary and the history so the customer never repeats anything (module 3, module 11).
  • Stay on the same channel where you can, and do not drop the user into a queue with no context.
  • Never trap the user in a loop. After two misses, offer a person.

Test it

Add these to your tests (module 16, module 24): ask for an exception to policy, ask for individual legal or medical advice, write as someone in distress, ask "are your staff certified?", ask to delete your data, and say "I want a human" in three different ways. Check that the assistant hands off, says what it should, and does not take the action itself.

Page history (1)
  1. New
    November 3, 2026 Release 2026.11.03c

    A list of ten situations an AI agent should not handle alone, what the assistant does instead, four questions to ask before automating a task, how to hand off well and what to test.

Full change log