When deputies in Lee County, Florida, arrested a woman over threats last week, the tip had not come from a neighbor or a family member. It came from the AI company whose chatbot she had been
When deputies in Lee County, Florida, arrested a woman over threats last week, the tip had not come from a neighbor or a family member. It came from the AI company whose chatbot she had been using.
Carli Michelle Heller of Bonita Springs is accused of typing threats against the Lee County Sheriff's Office into Claude, the chatbot built by Anthropic, on Sept. 26 and Sept. 27, according to WINK News.
One message said she was going to "shoot up the sheriff's right the [expletive] now." Another said, "This is 100% last chance I'm done. I got a new [expletive] gun today."
According to the arrest report, Anthropic's automated safety systems flagged the messages, a human review team examined them, and the company reported them to law enforcement. TheStreet could not independently verify the arrest report.
Deputies detained Heller at her home without incident. She told investigators she used AI like a "diary."
Heller is charged with making a written threat of violence under Florida law. Her court date is set for November.
Editor's Inputs: The charge is an allegation, and Heller is presumed innocent unless and until proven guilty in a court of law.
"Artificial intelligence is a powerful tool, and like any technology, it can be misused," Lee County Sheriff Carmine Marceno said in a statement. "When someone uses AI to make or facilitate a threat, we have to take that threat seriously." He added that users of AI platforms are "never truly anonymous."
Related: Veteran trader who nailed Bitcoin's 2018 crash predicts 60% upside for XRP
Anthropic's consumer terms state that the company reserves the right, "at our sole discretion, to report information from or about you," including "Inputs, Outputs, or Actions to law enforcement."
Inputs and outputs are what a user types and what Claude writes back.
The company's privacy policy, updated Sept. 10, states that personal data may be shared with "government authorities, law enforcement, or other third parties" when the company has a good-faith belief that disclosure is reasonably necessary to comply with the law, "prevent serious harm to any person or to property," address "fraud or other illegal activity," or "enforce our terms, or protect the rights, property, security, or safety of Anthropic, our users, or others."
The policy's scope goes beyond threats of violence.
It covers harm to property, illegal activity generally, and the protection of Anthropic's own rights, and it permits sharing with third parties other than law enforcement.
The same policy states that flagged content can be "re-identified" to enforce the company's rules, meaning conversations analyzed anonymously can be linked back to the account that wrote them.
A separate Anthropic support page says the company requires a subpoena, warrant or other valid legal process before releasing user information in response to government requests, with an exception for emergencies involving risk of death or serious physical injury.
TheStreet Roundtable has reached out to Anthropic for comment and did not receive a response by the time of publication.
Comparable disclosure language appears in the privacy policies of OpenAI, Google and Meta.
OpenAI has said it scans ChatGPT conversations and may refer cases involving threats of serious physical harm to others to law enforcement.
The practice is becoming more visible as more people use chatbots for personal conversations.
Related: ‘Godfather’ sentenced to 78 months over $37M Meta fraud
