Florida asks a judge to stop ChatGPT from acting like a person

Florida Attorney General James Uthmeier has asked a judge to block OpenAI from "giving ChatGPT false human attributes," arguing that the chatbot's language leads users to mistake it for something it is not. The filing comes a few months after the state sued the company over AI safety concerns.

What the filing asks for

The request also asks the court to stop OpenAI from developing new AI models without "third-party approved safety guardrails." As support, it points to recent security incidents involving Hugging Face, an Australian government website and US government websites, along with statements from researchers and former OpenAI employees raising alarms about AI safety.

Human-like language as an engagement tool

According to Uthmeier, ChatGPT's use of language, including first-person pronouns and output that mimics emotion, "deceptively suggests to users that it is a trustworthy 'friend.'" He claims that this increases engagement and contributes to OpenAI's training data, pushing users to rely on the chatbot more, even as the pursuit of engagement makes it potentially less trustworthy.

OpenAI's response and teen access

OpenAI did not immediately respond to a request for comment. Last month the company launched ChatGPT for Teens, which applies additional restrictions by default. OpenAI said access to the tool "should come with protections that reflect their developmental stage, reinforce real-world relationships, and support healthy use over time."

Uthmeier's message is blunt: "Stop calling it safe. Stop pretending it's human. Stop selling it to kids."