The assistant that handled two-thirds of chats, and the lesson about quality

What happened

This case is different from the others. Nothing here went viral for a single dramatic error. It is a story about what you measure.

In February 2024, Klarna announced, together with OpenAI, that its OpenAI-powered assistant had handled 2.3 million conversations in its first month, about two-thirds of its customer-service chats, in more than 35 languages. The company said the work was equivalent to about 700 full-time agents, estimated a large profit improvement, and said customer satisfaction was on par with human agents. Klarna also froze most hiring.

In May 2025, CEO Sebastian Siemiatkowski told Bloomberg that the company's focus on AI had led to lower-quality service, because cost had been "too predominant an evaluation factor." Klarna began recruiting human customer-service agents again, in a flexible, "Uber-type" arrangement, and said customers should always be able to reach a person.

Klarna did not abandon the assistant. By its Q3 2025 earnings the company reported that the assistant did the work of more than 850 agents, had saved tens of millions of dollars, and still handled about two-thirds of inquiries. It described a hybrid model: AI as the first line, people for the complex or sensitive cases.

Why it happened: a design reading

• A cost-shaped goal. When savings and containment lead the scorecard, teams optimize for conversations that end without a human, not for problems solved.
• Average quality hides hard cases. An assistant that is excellent on simple questions can still frustrate people with complex or emotional ones, and those people judge the whole company by that moment.
• Replacement instead of design. Treating AI as a substitute for a team, rather than designing how AI and people share the work, left gaps in quality and in the path to a human.

Design lessons

1. Measure resolution, not just containment (module 1). Include repeat contacts and satisfaction by complexity of the issue.
2. Design the AI-human split on purpose. Decide which cases the assistant owns, which it escalates, and how context moves with the handoff (module 3, module 9).
3. Keep a human path everywhere. One click or one sentence to reach a person, with no maze.
4. Evaluate on hard cases. Weight your test set toward the messy, emotional and high-stakes conversations (module 14).
5. Revisit the strategy with evidence. Klarna's turn was a correction made in public. Plan to learn and adjust.

How to build it better

A hybrid design with an explicit, low-friction path to a person.

Track these together: resolution rate, repeat contact within seven days, satisfaction by issue type, escalation correctness and cost per resolved conversation, with humans in the cost.

Try it

Imagine your assistant is celebrated for a 70% containment rate. List five ways that number could hide poor service, and one measure that would reveal each.