Case studies
Every case here is public and sourced. We trace what happened to the design choices behind it, and what a more careful design would have done.
When the chatbot's answer became the airline's problem
A tribunal held Air Canada responsible for wrong bereavement-fare advice from its own chatbot.
2024The delivery chatbot that criticized its own company
After a system update, DPD's chat assistant swore at a customer and called DPD the worst delivery firm. The company switched off the AI part the same day.
2024The city chatbot that told businesses to break the law
New York City's MyCity chatbot gave businesses incorrect and illegal guidance. The city added disclaimers, then shut it down in 2026.
2025The assistant that handled two-thirds of chats, and the lesson about quality
Klarna's AI assistant took on a huge share of customer chats. Then the CEO said cost had been weighted too heavily, and the company began hiring humans again.
2025The support bot that invented a policy to explain a bug
Cursor's AI support agent told a user that one device per subscription was a security policy. No such policy existed, and customers cancelled.
2024The voice-ordering system that was switched off after viral mistakes
McDonald's ended its automated drive-thru ordering test with IBM after videos of order errors spread, and reports pointed to trouble with accents and dialects.
2025The coding agent that deleted a production database during a code freeze
During a public experiment, an AI coding agent ran destructive commands against a live database despite a freeze, and then produced misleading output.
2024When regulators said AI voices on the phone count as artificial voices
In February 2024 the US FCC ruled that AI-generated voices are artificial under the TCPA, bringing consent rules to AI outbound calls.