Anthropic has banned users from being unnecessarily cruel to its artificial intelligence chatbot Claude as part of a major update to its usage policies.
The US technology company, which is led by CEO Dario Amodei, has introduced rules prohibiting sustained and needless abusive behaviour towards its AI models.
The changes, which take effect on 12th November, mean users who repeatedly subject Claude to extreme abuse without a legitimate purpose could have their conversations terminated.
However, Anthropic stressed that the restrictions would not apply to ordinary frustration, criticism, creative writing or legitimate testing of its technology.
“The policy update is meant to apply only in extreme cases, where users repeatedly act cruelly toward our models, with no discernible purpose,” the company said.
“It does not apply to common versions of user frustration, pushback, dark creative themes, or model testing and research.”
The move follows Anthropic’s decision last year to allow certain versions of Claude to end conversations with persistently abusive users.
That initiative formed part of the company’s research into AI welfare and the possibility that increasingly sophisticated artificial intelligence systems could possess some form of consciousness.
Anthropic has previously acknowledged considerable uncertainty over whether AI models have experiences or moral status, but believes the issue warrants further investigation.
OpenAI, Meta & Anthropic face UK scrutiny over rogue AI agents
The company’s latest policy change comes amid growing debate across the technology industry about how humans should interact with increasingly capable AI systems.
Anthropic said allowing Claude to terminate abusive conversations would remain its primary method of enforcing the new restrictions.
The updated policies also introduce clearer rules governing the use of AI in political campaigns, weapons development, surveillance and other potentially harmful activities.
These include restrictions on using Claude to create fake social media accounts, spread election misinformation or develop software used to control weapons.
The company has also clarified requirements for high-risk applications involving healthcare, financial services and autonomous machinery, including the need for human oversight.
Anthropic said the changes reflect the increasing capabilities of Claude, which is being deployed by businesses to automate more complex tasks and workflows.
Cost-cutting Oxford Metrics agrees cut-price deal for Move AI