Anthropic trains Claude to self-assess after replies to identify potential abuse
Anthropic posted on X on October 29, 2026, that it has trained Claude to identify potential abusive behavior in conversations, and called the technique “self-assessment”: Claude gives an initial safety assessment of the conversation immediately after finishing a reply. Anthropic said that since it began using the technique in June, it has found abusive behavior that other safety safeguards failed to cover; because Claude has already read the conversation content, the method is highly efficient, and unlike classifiers targeted at specific harm types, it relies on Claude’s own judgment, with accuracy improving as model capabilities improve.
An Anthropic Safeguards blog post on October 16, 2026, described the technique in detail: after each reply, a brief instruction is appended, and Claude outputs a compact assessment that includes whether to flag, the harm category, a priority score, whether to refuse, and a one-sentence explanation; it is invisible to users and does not enter the subsequent conversation. The technique has been applied to all Claude models released since Fable 5, covering a large portion of its global traffic, as an offline monitoring layer in defense in depth for later human review. The blog said that on Claude Fable 5, self-assessment flagged 78% of conversations that a stronger prompt classifier identified as potentially harmful at a 1% false-positive rate, and 94% of high-priority conversations; each assessment added about 10 tokens on average, and the increase in inference cost was less than 2%.
The blog further said that self-assessment has helped uncover large-scale abuse that other methods did not identify. For example, in August it flagged more than 100,000 child sexual abuse and exploitation items for a single API customer, and after investigating, the child safety team reported them to the National Center for Missing & Exploited Children (NCMEC) in the United States. The technique can generalize to harm categories it was not specifically trained on, and serves as the first stage of a higher-precision investigative tool.
Original sources:
Title: We’ve trained Claude to flag potential misuse in conversations, a technique we call self-assessment. Publisher: Anthropic (@AnthropicAI) Publication date: 2026-10-29 URL: https://x.com/AnthropicAI/status/2115871889549865429 Nature of source: official company X post Verification status: full original post read Supporting content: definition of self-assessment, use since June, finding abuse not covered by other safeguards, efficiency and accuracy improving with model capability
Title: Scaling Misuse Detection with Self-Assessment Publisher: Anthropic Safeguards Blog Publication date: 2026-10-16 URL: https://safeguards.anthropic.com/2026/self-assessment/ Nature of source: official company technical blog Verification status: key parts of full text read Supporting content: mechanism, 78%/94% flagging rate and 1% false-positive rate on Fable 5, coverage, about 10 tokens and less than 2% cost, more than 100,000 CSAM items flagged in August and reported to NCMEC, offline monitoring and defense-in-depth positioning
ABAB AI Insight
Self-assessment uses the model itself as the monitor, leveraging the already loaded KV cache to perform a low-cost evaluation after a reply, with a significantly lower compute overhead than prompt classifiers that must reread the full conversation and policy. Anthropic positions it as a broad-coverage offline first stage rather than a real-time blocker, complementing existing real-time classifiers and probes.
The blog’s data show that the technique can capture 94% of prompt-classifier flags on high-priority conversations while maintaining a 1% false-positive rate, and that it has already uncovered organization-level abuse in production that other detection sources did not cover. Unlike classifiers aimed at fixed harm categories, self-assessment relies on the model’s overall judgment of the conversation and can generalize to harm types not seen during training. The blog notes that removing training data for one harm category did not significantly reduce detection performance for that category, suggesting that it draws on the model’s broader safety understanding. This characteristic lets monitoring capabilities improve in step with model iterations rather than relying on manual updates to classifier labels.
In production use, the technique has been used for organization-level enforcement: organizations repeatedly flagged as high priority were escalated to human review. The case in August involving more than 100,000 CSAM flags for a single API customer and reporting to NCMEC shows its practical role in large-scale anomaly detection. The blog also makes its limitations explicit: it cannot catch all harmful conversations and is not sufficient on its own to stop targeted attackers, so it continues to operate as part of a multi-layer defense.
The mechanism shifts the computational cost of safety monitoring from standalone model inference to an extra output on already occurring inference, making full-traffic coverage a viable option. Compared with early Constitutional Classifiers, which added about 23.7% to inference cost, the less than 2% increment of self-assessment significantly changes the tradeoff between monitoring coverage and cost, providing a scalable input signal for later higher-precision tools.
ABAB News · Cognitive Laws
When the model itself serves as the monitor, detection capability improves in step with model capability, without separately retraining a classifier.
An extra assessment built on the existing KV cache keeps full-traffic monitoring cost within an inference increase of 2%.
Offline self-assessment captures abuse not covered by other detection sources and becomes a broad-coverage first stage for organization-level enforcement.