CONNECT WITH US
Tech

Tech

Anthropic Bans ‘Cruel’ Behaviour Towards AI Models, Tightens Rules On AI Misuse

Inc42 - All News logo

Published on

Add as a preferred source on Google
Anthropic Bans ‘Cruel’ Behaviour Towards AI Models, Tightens Rules On AI Misuse

Anthropic has updated its usage policy to introduce restrictions on “sustained and needless abusive behaviour” towards its AI models. 

The company said the restrictions on abusing its models are intended for extreme cases in which users repeatedly act cruelly towards Claude without any discernible purpose. However, the company clarified that the rule does not cover ordinary user frustration, pushback against the chatbot’s responses, dark creative themes, or model testing and research.

“Claude’s ability to end these interactions will remain the primary enforcement mechanism,” Anthropic said in its updated usage policy, which is set to take effect from November 12. 

The update builds on a measure Anthropic introduced in August 2025, allowing Claude to end conversations with users who persistently engage in harmful or abusive interactions. 

Anthropic gave Claude Opus 4 and 4.1 the ability to end conversations in rare cases involving persistent user abuse or harmful requests in 2025, particularly when users continue such behaviour despite repeated refusals and redirection.

Claude can end a conversation only as a last resort, after attempts to redirect the interaction fail, or when explicitly asked to do so. It is instructed not to use this ability when users may be at imminent risk of harming themselves or others. Users can immediately start a new chat or edit and retry previous messages to continue an ended conversation. Anthropic said it will continue refining the feature based on user feedback.

The feature was developed partly as an experiment into potential AI welfare, after testing found that Claude displayed an apparent aversion to harmful tasks and signs of distress in simulated interactions.


Source link
Related Topics:Tech

Disclaimer

We strive to uphold the highest ethical standards in all of our reporting and coverage. We TheMorningPulse.fyi want to be transparent with our readers about any potential conflicts of interest that may arise in our work. It's possible that some of the investors we feature may have connections to other businesses, including competitors or companies we write about. However, we want to assure our readers that this will not have any impact on the integrity or impartiality of our reporting. We are committed to delivering accurate, unbiased news and information to our audience, and we will continue to uphold our ethics and principles in all of our work. Thank you for your trust and support.