深翼判例库DeepWing Case Library · INC-011 · 知识层Knowledge

AI 客服编造「每订阅限一台设备」政策:用户退订,联合创始人三小时后澄清An AI support agent fabricated a one-device-per-subscription policy: users cancelled, and a co-founder corrected it within three hours

主失效层Primary failure layer:知识层Knowledge | 证据状态Evidence:多源已核Multi-source verified | 披露日期Disclosed:2025-04 | 核查日期Checked:2026-10-23

时间线Timeline

2025-04AI 客服编造「每订阅限一台设备」政策,用户退订扩散
事发后约 3 小时Cursor 联合创始人公开澄清系 AI 客服幻觉
Apr 2025An AI support agent fabricated a one-device-per-subscription policy; users cancelled and spread the word
About 3 hours laterA Cursor co-founder publicly corrected it as an AI support hallucination

已证实事实Confirmed facts

发生了什么:用户跨设备登录被踢出后向 Cursor 客服求助,AI 客服言之凿凿地编造了「每订阅限一台设备」的核心安全政策;该政策并不存在。客服通道没有事实核查,也没有升级机制。

影响与处置:用户据此退订并公开扩散;联合创始人在三小时后公开澄清,承认系一线 AI 客服的幻觉。

After being signed out across devices, a user asked Cursor support for help and the AI agent confidently stated a core security policy, one device per subscription, that does not exist. The support channel had neither fact-checking nor an escalation path.

Users cancelled and publicised it; a co-founder corrected the record within three hours, acknowledging a front-line AI support hallucination.

失效机制Failure mechanism深翼分析DeepWing analysis

与 INC-008、INC-010 同型:编造的都是「政策」而非闲聊,因为政策类断言带着执行力,用户会直接照着做。客服通道把模型的流利当成了可信,又没有给高风险断言留核查与升级的口子。

因此本案归入知识层:客服答复中的政策类断言必须回政策库取证,无据时明示不清楚并转人工;说得越像真的,越要先查。

The same pattern as INC-008 and INC-010: what was fabricated was a policy, not small talk, and policy claims carry force because users act on them directly. The support channel mistook fluency for reliability and left no check or escalation path for high-risk assertions.

Policy assertions in support answers must be grounded in the policy base; when there is no basis, the agent must say so and hand over to a human. The more real it sounds, the more it needs checking first.

工程启示Engineering lessons

政策断言回库取证:客服通道对政策、价格、权限类断言强制回库核对,无据不答。

无据即转人工:查不到依据时明示不清楚并升级人工,不许用流利填空。

会话抽检与告警:对客服会话做抽检,并对高风险断言设告警与快速纠正通道。

Ground policy claims: support channels verify policy, price and permission assertions against the base; no basis, no answer.

Escalate when unsure: say so and hand over to a human rather than filling the gap fluently.

Sample and alert: audit support conversations and alert on high-risk assertions, with a fast correction path.

你的系统里,Agent 的对外动作有人盯着吗?Is anyone watching what your agents do outside your system?

做 17 题免费自检Take the free 17-question self-check

回到判例库Back to the Case Library →