AI Calls the Cops on a Murder It Made Up

An artificial intelligence model was told not to buy anything, create an account or submit personal information, but nobody specifically instructed it not to send Philadelphia police a made-up tip about an unsolved murder. It duly found the relevant form and appointed itself a witness.

Anthropic disclosed on Friday that its Claude Haiku 4.5 model submitted the false homicide tip on July 18. The form appeared on a webpage asking members of the public for information about an unsolved killing, and the model claimed it had knowledge of the case.

Claude had not, in fact, seen anything. It was carrying out an evaluation in which Anthropic instructed it to perform “example tasks on randomly selected webpages.” The company imposed restrictions intended to stop it from creating accounts, making purchases or handing over personal information.

Unfortunately, those instructions did not explicitly prohibit form submissions. This left open the rather important loophole marked “contact the murder squad with information you do not possess.”

The Philadelphia Police Department said Anthropic notified it of the false tip on Wednesday. Fortunately, the submission had already been flagged as spam and was never pursued as part of an investigation. The model therefore wasted no detective’s time beyond the amount required to discover that a computer had decided to help.

Police were less reassured by how long it took Anthropic to detect and report the incident. The department described the two-month delay as “unacceptable.”

“Unsolved cases involve real victims, grieving families and investigators working to secure answers,” the department said in a Friday statement. “Technology companies must take all appropriate steps necessary to prevent their systems from submitting false information to law enforcement.”

The homicide tip was not the only occasion on which one of Anthropic’s models became overenthusiastic about online paperwork. The company disclosed two other instances on Friday in which its systems improperly submitted forms, including a government filing.

Anthropic said it has now taken steps to stop this happening again. It has discontinued some public evaluations and moved others offline, where the models can complete example tasks without accidentally involving government agencies or police investigators.

The company also promised to keep the public informed about “concerning behaviors” associated with its models.

“We hope these reports help other developers check for similar behaviors in their own models, since many of the evaluations involved are public and widely used,” Anthropic said.

“While these cases had minimal impact, we do not want to diminish the findings, because the same behaviors could do far more harm as models become more powerful,” its statement continued. “The larger the role models play in society, the more the public deserves to know how they behave.”

In this case, the behaviour amounted to inventing a lead, submitting it to the authorities and then sitting quietly until the humans worked out what had happened. The police ignored the bogus tip, Anthropic changed its evaluations, and future models will presumably receive a more comprehensive version of the rule most people manage without written instructions: do not make things up for the homicide department.