# The support bot that invented a policy to explain a bug

> Cursor's AI support agent told a user that one device per subscription was a security policy. No such policy existed, and customers cancelled.

Source: https://conversationalaidesign.com/case-studies/cursor-support-bot.html. Author: Nelson Abacherli Kawakami. Reviewed October 27, 2026.

## Key facts
- When: April 2025
- What the bot said: That Cursor is designed to work with one device per subscription as a core security feature
- Reality: No such policy existed; users had been unexpectedly logged out when switching machines
- Customer impact: Public cancellations on forums, then an apology and refunds
- Company response: Staff said it was an incorrect response from a front-line AI support bot, and said AI-generated replies would be labeled more clearly


## What happened

In April 2025, users of the AI code editor Cursor began getting logged out when they switched between machines. Some of them asked support what had changed. According to reporting, one received an email from an agent named "Sam" that said Cursor "is designed to work with one device per subscription as a core security feature."

There was no such policy. Sam was an AI support bot, and the logouts were an unintended problem, not a security rule. The user's account of the reply spread on Reddit and Hacker News, many readers took it as an official policy change, and several publicly announced they were cancelling. A company representative then replied that there was no such policy, that people could use Cursor on multiple machines, and that the answer was an incorrect response from a front-line AI support bot.

The company apologized, issued refunds, investigated the logout problem and said it would label AI-generated support replies more clearly. Commenters also objected that the bot had a human first name and wasn't identified as AI.

## Why it happened: a design reading

We know what the bot said and what the company said afterward, not how the bot was built. Reading the design from outside:

- **The bot was asked to explain behavior it had no source for.** A real defect was explained with a plausible-sounding policy. A model asked "why is this happening?" will produce an answer unless it is told what it may state.
- **No grounding for policy claims.** Statements about policy, security or pricing should be traceable to an approved document. This one wasn't.
- **A bug report was handled as a question.** The right response to "I was logged out" is to collect details and route to engineering, not to explain.
- **A human name without disclosure.** It made the answer sound official and made the deception complaint easy to make.
- **The consequence landed immediately.** A customer-facing answer on a pricing and account policy reached an audience that acts on it.

## Design lessons

1. **Treat claims about policy, terms, pricing and security as restricted.** The assistant may only state them from cited approved sources. Otherwise it should say it doesn't know and hand off (module 10, module 6).
2. **Add an output check** that flags policy-like statements with no source, and routes them for review or rewrites them (module 10).
3. **Recognize anomaly reports** and escalate them with logs instead of explaining (module 11).
4. **Disclose that it's an AI** and avoid disguising it with a human persona (module 3, module 18).
5. **Plan for correction at speed:** a way to retract the bad answer, notify affected users and update the knowledge base.

## How to build it better

**Reporting a problem, not asking for a policy**
> **User:** I keep getting logged out when I switch laptops. Is this a new rule?
> **Bot:** Thanks for telling me. I don't have any policy that limits you to one device, so I can't confirm that this is intended. It may be a bug, and I don't want to guess. I've passed your report to our team, including your account and the times you were logged out, and I'll email you when we know more. Would you like me to include your operating system versions?

The key moves: don't invent, say what is and isn't known, capture useful details, escalate, and commit only to what the system can actually do.

## Try it

List ten questions where a plausible but invented answer would create a commitment: warranties, privacy, security, refunds, account limits. For each, write the rule your assistant should follow when no approved source exists, and the test that proves it.


## Sources
- CX Today: A contact center chatbot invents company policies, now customers want out: https://www.cxtoday.com/contact-center/a-contact-center-chatbot-invents-company-policies-now-customers-want-out/
- Fanatical Futurist: Another AI bot causes a mess after inventing a company policy: https://www.fanaticalfuturist.com/2025/05/another-ai-bot-causes-a-mess-after-inventing-a-company-policy/
- AI Incident Database, incident 1039: https://incidentdatabase.ai/cite/1039/
- Wikipedia: Anysphere: https://en.wikipedia.org/wiki/Anysphere
- Vibe Graveyard: Cursor's AI support bot invented a login policy: https://vibegraveyard.ai/story/cursor-ai-support-bot-fake-policy/