Fallbacks and graceful degradation

Everything fails somewhere

Assistants fail in many ways, and the failure users see is usually the one nobody designed. The McDonald's drive-thru trial is a clear example: when the system misheard, a person had to step in. A fallback is the planned response to a failure. Good ones keep the user moving and the system safe.

The ways it fails

Three principles

• Fail safe: when unsure, do the least risky thing. Don't guess on money or personal data.
• Fail visible: tell the user what happened and what comes next. Hidden failures breed repeats and complaints.
• Fail forward: every fallback ends with a next step. Never a loop, never a dead end.

The fallback ladder

Escalate through increasingly different strategies as failures accumulate.

Each rung changes the approach. Count failures so you move up instead of repeating.

FALLBACKS = [
    ("clarify", "Could you tell me a bit more? For example, are you asking about a refund, a delivery or an account change?"),
    ("choices", "I can help with: 1) a return, 2) an order status, or 3) talking to a person. Which would you like?"),
    ("handoff", "I'm not getting this right. Let me connect you with a person, and I'll pass along what we've discussed."),
]

def next_fallback(ctx: dict) -> tuple[str, str]:
    """ctx['fails'] counts consecutive failures in this conversation."""
    ctx["fails"] = ctx.get("fails", 0) + 1
    index = min(ctx["fails"] - 1, len(FALLBACKS) - 1)
    return FALLBACKS[index]

def on_success(ctx: dict) -> None:
    ctx["fails"] = 0        # reset only on genuine progress

Make calls resilient

Timeouts. Every external call needs one. A call without a timeout is a hung conversation.

Retries with backoff and jitter. Retry transient errors a few times, waiting longer each time, with random jitter so many clients don't retry at once. Don't retry errors that won't change, such as invalid arguments.

Idempotency. A retry after a timeout may repeat the action. Use idempotency keys for writes so charging, cancelling or sending can't happen twice.

Circuit breakers. After repeated failures, stop calling the broken dependency for a while and use the fallback straight away instead of making every user wait.

Failover. Keep a tested secondary model, ideally from another provider, and a smaller model as a degraded mode (module 4). Log which model served each turn.

import random, time

def call_with_fallback(primary, secondary, payload, retries=2, base_delay=0.4):
    """Try the primary a few times, then fail over. Returns (result, model_used)."""
    for attempt in range(retries + 1):
        try:
            return primary(payload, timeout=8), "primary"
        except TransientError:
            time.sleep(base_delay * (2 ** attempt) + random.uniform(0, 0.2))   # backoff with jitter
        except PermanentError:
            break                                                              # retrying will not help
    try:
        return secondary(payload, timeout=8), "secondary"
    except Exception:
        return None, "none"          # caller shows the safe static fallback and hands off

Say it well

Fallback messages are some of the most-read text in your product. Good ones are honest, specific and forward-moving.

For voice, keep fallbacks short, avoid repeating the same prompt, and offer a keypad or human option early (module 19).

When the human isn't available

A handoff is itself a dependency that can fail. Plan for it:

1. Check availability before promising a handoff.
2. If no one is available, say so plainly with hours or expected wait.
3. Offer a callback or a ticket, capture contact details and a summary, and confirm what will happen next.
4. For urgent or safety-critical topics, give the appropriate emergency or priority route immediately.

A bug is not a policy question

The Cursor case study shows a subtler fallback problem. When users reported unexplained logouts, the support bot answered with an invented policy instead of treating it as a possible defect. Teach the assistant to recognize anomaly reports ("this stopped working," "I was logged out") and route them to a person or an engineering ticket, with logs, rather than explaining system behavior it has no source for.

Test fallbacks by breaking things

• Inject failures: kill a tool, add latency, return malformed data, exhaust a rate limit, switch off the primary model.
• Check the user experience: the message, the next step, whether context survived.
• Simulate repeated misunderstanding and confirm the ladder advances.
• Test the handoff with and without available agents.
• Spec them like any behavior (module 15), and run chaos-style tests in staging.

Monitor fallback rate by cause and by step. A rising "not found" rate points to content gaps (module 17), and rising tool timeouts point to an integration problem.