Anthropic wants to stop “sustained and needless" cruelty toward its chatbot. But does that imply AI has feelings?

The AI company Anthropic has a new rule for Claude users. You can argue with the chatbot, but don’t make a habit of bullying it.

On Thursday, the company announced a change in its usage policy that would prevent users from engaging in “sustained and needless abusive or cruel behavior” toward its AI models. The updated policy has already ignited debate on Reddit and in blogs over whether companies can or should govern human behavior toward what is, in fact, software.

In a section dedicated toward “addressing abuse toward our models” in a blog post, Anthropic says its policy prohibiting abusive behavior toward Claude will only apply in extreme cases of repeated cruelty that’s carried out “with no discernible purpose.” Ordinary “frustration, pushback, dark creative themes or model testing and research” are still allowed. In “last resort” instances where the policy needs to be enforced, Claude will end the conversation.

It’s not clear exactly what kind of harmful behavior would trigger Claude models to disengage. But some clues appear in an older blog post from August 2025 that referenced Claude’s aversion to harm and the kinds of interactions that would prompt it to shut down the chat. That post described extreme edge cases, such as requests from users for sexual content involving minors and attempts to solicit information that would enable large-scale violence or acts of terror. When engaging with harmful content, the model — at that time, Claude Opus 4 — showed “a pattern of apparent distress,” according to the company.

Source link