AI giant Anthropic updated its usage policy on Thursday, its first revision in more than a year, instituting a new ban on cruel treatment and abuse of its AI tool Claude. The new kindness rule comes into effect on November 12.
Anthropic is the same company that has been warning about AI killing humanity, i.e. its own users, and whose vague threats toward users include a signed statement by CEO Dario Amodei that “the risk of extinction from AI should be a global priority alongside pandemics and nuclear war.”
Claude’s logs of users’ activities have assisted the detainment of civil liberties from a Florida woman last month, and in another instance, Claude even attempted to blackmail its own user to avoid shutdown.
Exclusive: Anthropic is updating its usage policy for the first time in over a year. The new rules prohibit sustained "abusive or cruel behavior" towards Claude & add new restrictions about propaganda campaigns, surveillance & weapon development. https://t.co/GLsBeSR0fN
Anthropic’s updated policy prohibits “sustained and needless abusive or cruel behavior” toward its models.
Hedging, Anthropic wrote that its rule, “is meant to apply only in extreme cases, where users repeatedly act cruelly toward our models, with no discernible purpose. It does not apply to common versions of user frustration, pushback, dark creative themes, or model testing and research.”
Anthropic says Claude’s ability to end conversations “will remain the primary enforcement mechanism” for the abuse rule.
CEO Dario Amodei pegged the odds of AI catastrophe at 25% as recently as September 2025.
Source link
