# Fallbacks and graceful degradation

> Design what happens when understanding, retrieval, tools, models or channels fail: a fallback ladder, resilient calls, honest wording, handoff when no one is available and tests that break things on purpose.

Source: https://conversationalaidesign.com/modules/11-fallbacks.html. Author: Nelson Abacherli Kawakami. Reviewed October 27, 2026.

## What you will learn
- Classify the ways an assistant fails and match each to a fallback
- Design a fallback ladder that avoids loops and dead ends
- Make model, tool and retrieval calls resilient with retries, timeouts and failover
- Write honest, useful fallback messages for chat and voice
- Test fallbacks by injecting failures


## Everything fails somewhere

Assistants fail in many ways, and the failure users see is usually the one nobody designed. The McDonald's drive-thru trial is a clear example: when the system misheard, a person had to step in. A **fallback** is the planned response to a failure. Good ones keep the user moving and the system safe.

## The ways it fails

| Failure | Example | Fallback direction |
|---|---|---|
| **Not understood** | "Sorry, I didn't get that" | Clarify, offer choices, show examples |
| **Low confidence** | Two likely intents | Ask a disambiguating question |
| **Nothing retrieved** | The knowledge base has no answer | Say so, point to the right source, offer a person |
| **Tool error or timeout** | The order system is down | Retry safely, explain, offer a callback |
| **Model error or outage** | Rate limit or provider incident | Retry, fail over to another model, degrade |
| **Policy block** | A guardrail stopped the reply | Safe alternative wording, scoped help |
| **User frustration** | Repeated "agent", anger | Hand off promptly |
| **Unsupported language** | The user switches language | Detect, confirm, switch or offer a person |
| **Channel failure** | Noisy audio, failed upload | Switch modality or channel (module 21) |

## Three principles

- **Fail safe:** when unsure, do the least risky thing. Don't guess on money or personal data.
- **Fail visible:** tell the user what happened and what comes next. Hidden failures breed repeats and complaints.
- **Fail forward:** every fallback ends with a next step. Never a loop, never a dead end.

## The fallback ladder

Escalate through increasingly different strategies as failures accumulate.

- Retry safely → Clarify with a specific question → Offer two or three choices → Switch mode: simpler flow or another channel → Static or cached answer → Human handoff → Callback or ticket

```python
FALLBACKS = [
    ("clarify", "Could you tell me a bit more? For example, are you asking about a refund, a delivery or an account change?"),
    ("choices", "I can help with: 1) a return, 2) an order status, or 3) talking to a person. Which would you like?"),
    ("handoff", "I'm not getting this right. Let me connect you with a person, and I'll pass along what we've discussed."),
]

def next_fallback(ctx: dict) -> tuple[str, str]:
    """ctx['fails'] counts consecutive failures in this conversation."""
    ctx["fails"] = ctx.get("fails", 0) + 1
    index = min(ctx["fails"] - 1, len(FALLBACKS) - 1)
    return FALLBACKS[index]

def on_success(ctx: dict) -> None:
    ctx["fails"] = 0        # reset only on genuine progress
```

## Make calls resilient

**Timeouts.** Every external call needs one. A call without a timeout is a hung conversation.

**Retries with backoff and jitter.** Retry transient errors a few times, waiting longer each time, with random jitter so many clients don't retry at once. Don't retry errors that won't change, such as invalid arguments.

**Idempotency.** A retry after a timeout may repeat the action. Use idempotency keys for writes so charging, cancelling or sending can't happen twice.

**Circuit breakers.** After repeated failures, stop calling the broken dependency for a while and use the fallback straight away instead of making every user wait.

**Failover.** Keep a tested secondary model, ideally from another provider, and a smaller model as a degraded mode (module 4). Log which model served each turn.

```python
import random, time

def call_with_fallback(primary, secondary, payload, retries=2, base_delay=0.4):
    """Try the primary a few times, then fail over. Returns (result, model_used)."""
    for attempt in range(retries + 1):
        try:
            return primary(payload, timeout=8), "primary"
        except TransientError:
            time.sleep(base_delay * (2 ** attempt) + random.uniform(0, 0.2))   # backoff with jitter
        except PermanentError:
            break                                                              # retrying will not help
    try:
        return secondary(payload, timeout=8), "secondary"
    except Exception:
        return None, "none"          # caller shows the safe static fallback and hands off
```

## Say it well

Fallback messages are some of the most-read text in your product. Good ones are honest, specific and forward-moving.

**Nothing found: honest and useful**
> **User:** Can I return a jacket I bought in a store?
> **Bot:** I only have the policy for online orders, so I can't answer that reliably. Store returns are handled by the store policy, and I can connect you to someone who can confirm it. Would you like that?

**A tool is down**
> **User:** Where is order A1234?
> **Bot:** I can't reach our order system right now. I haven't lost your request. I can text you the status as soon as it's back, or connect you to an agent. Which would you prefer?

For voice, keep fallbacks short, avoid repeating the same prompt, and offer a keypad or human option early (module 19).

## When the human isn't available

A handoff is itself a dependency that can fail. Plan for it:

1. Check availability before promising a handoff.
2. If no one is available, say so plainly with hours or expected wait.
3. Offer a callback or a ticket, capture contact details and a summary, and confirm what will happen next.
4. For urgent or safety-critical topics, give the appropriate emergency or priority route immediately.

## A bug is not a policy question

The Cursor case study shows a subtler fallback problem. When users reported unexplained logouts, the support bot answered with an invented policy instead of treating it as a possible defect. Teach the assistant to recognize **anomaly reports** ("this stopped working," "I was logged out") and route them to a person or an engineering ticket, with logs, rather than explaining system behavior it has no source for.

## Test fallbacks by breaking things

- **Inject failures:** kill a tool, add latency, return malformed data, exhaust a rate limit, switch off the primary model.
- **Check the user experience:** the message, the next step, whether context survived.
- **Simulate repeated misunderstanding** and confirm the ladder advances.
- **Test the handoff** with and without available agents.
- **Spec them** like any behavior (module 15), and run chaos-style tests in staging.

Monitor **fallback rate by cause and by step**. A rising "not found" rate points to content gaps (module 17), and rising tool timeouts point to an integration problem.


## Key takeaways
- Plan for failure first: every step needs a safe next step.
- Fail safe, fail visible and fail forward: never a loop or a dead end.
- Retry what is safe to retry, and make writes idempotent.
- A human handoff needs its own fallback for when no one is available.

## Sources
- Case study: McDonald's AI drive-thru: https://conversationalaidesign.com/case-studies/mcdonalds-ai-drive-thru.html
- Case study: Cursor's support bot: https://conversationalaidesign.com/case-studies/cursor-support-bot.html
- Langfuse: tracing: https://langfuse.com