深翼判例库DeepWing Case Library · INC-011 · 知识层Knowledge
AI 客服编造「每订阅限一台设备」政策:用户退订,联合创始人三小时后澄清An AI support agent fabricated a one-device-per-subscription policy: users cancelled, and a co-founder corrected it within three hours
本条在判例库中的登记行This entry in the Case Library: deepwingai.com/cases#INC-011
时间线Timeline
已证实事实Confirmed facts
发生了什么:用户跨设备登录被踢出后向 Cursor 客服求助,AI 客服言之凿凿地编造了「每订阅限一台设备」的核心安全政策;该政策并不存在。客服通道没有事实核查,也没有升级机制。
影响与处置:用户据此退订并公开扩散;联合创始人在三小时后公开澄清,承认系一线 AI 客服的幻觉。
After being signed out across devices, a user asked Cursor support for help and the AI agent confidently stated a core security policy, one device per subscription, that does not exist. The support channel had neither fact-checking nor an escalation path.
Users cancelled and publicised it; a co-founder corrected the record within three hours, acknowledging a front-line AI support hallucination.
失效机制Failure mechanism深翼分析DeepWing analysis
与 INC-008、INC-010 同型:编造的都是「政策」而非闲聊,因为政策类断言带着执行力,用户会直接照着做。客服通道把模型的流利当成了可信,又没有给高风险断言留核查与升级的口子。
因此本案归入知识层:客服答复中的政策类断言必须回政策库取证,无据时明示不清楚并转人工;说得越像真的,越要先查。
The same pattern as INC-008 and INC-010: what was fabricated was a policy, not small talk, and policy claims carry force because users act on them directly. The support channel mistook fluency for reliability and left no check or escalation path for high-risk assertions.
Policy assertions in support answers must be grounded in the policy base; when there is no basis, the agent must say so and hand over to a human. The more real it sounds, the more it needs checking first.
工程启示Engineering lessons
政策断言回库取证:客服通道对政策、价格、权限类断言强制回库核对,无据不答。
无据即转人工:查不到依据时明示不清楚并升级人工,不许用流利填空。
会话抽检与告警:对客服会话做抽检,并对高风险断言设告警与快速纠正通道。
Ground policy claims: support channels verify policy, price and permission assertions against the base; no basis, no answer.
Escalate when unsure: say so and hand over to a human rather than filling the gap fluently.
Sample and alert: audit support conversations and alert on high-risk assertions, with a fast correction path.
你的系统里,Agent 的对外动作有人盯着吗?Is anyone watching what your agents do outside your system?
做 17 题免费自检Take the free 17-question self-check