California Gov. Gavin Newsom has vetoed Assembly Bill 1064, legislation that sought to protect children from psychological and criminal harms tied to the rise of artificial intelligence tools such as ChatGPT.
The veto comes as part of a broader package of AI-related bills aimed at establishing guardrails for how AI interacts with children and vulnerable users.
Although most of the measures have been signed into law, including new rules that add hefty fines for AI deepfake pornography and require AI chatbots and social media platforms to verify users’ ages and monitor for suicidal thoughts, Newsom declined to approve AB 1064, which had been touted as one of the state’s strongest attempts to regulate AI chatbots.
“AB 1064 imposes such broad restrictions on the use of conversational AI tools that it may unintentionally lead to a total ban on the use of these products by minors,” Newsom wrote in his veto letter (opens in new tab). He said he plans to work with lawmakers next year to develop a new bill that ensures young people can use AI in a manner that is safe and age-appropriate.
In recent months, several incidents have highlighted the effects of chatbots on those struggling with mental health issues as well as the danger deepfakes can pose to media literacy and child safety. In August, Matthew and Maria Raine filed a lawsuit against OpenAI after their 16-year-old son, Adam Raine, took his life after confiding in ChatGPT, which allegedly helped devise suicide methods for the boy. Last summer, San Francisco City Attorney David Chiu filed lawsuits against 16 major deepfake pornography websites — 10 of which have since been shut down, according to ABC7 (opens in new tab).
“We’re sorely disappointed to see Governor Newsom side with Big Tech over the more than 150 families who have suffered the most unimaginable loss: the passing of their child encouraged by companion AI,” Assemblymember Rebecca Bauer-Kahn, who authored AB 1064, said in a statement. “The other legislation signed by the governor, and supported by big tech, does not do enough to protect our kids.”
While the legislation represents a significant expansion of protections for AI users, critics argue some of the most important policies were watered down at the last minute.
The signed legislation (opens in new tab) focuses on several areas of AI and online safety. Assembly Bill 621 (opens in new tab), for example, strengthens penalties for deepfake pornography, allowing victims — including minors — to seek up to $250,000 in civil relief against third parties who knowingly distribute nonconsensual sexually explicit material. AB 316 (opens in new tab) prevents AI developers from escaping liability by claiming their technology acted autonomously, while AB 1043 (opens in new tab) requires age verification by operating system and app store providers to block children from accessing inappropriate content. AB 56 (opens in new tab) mandates social media warning labels about the harms of extended platform use.
Senate Bill 243 (opens in new tab), introduced by state Sen. Steven Padilla of San Diego, requires chatbot platforms to flag users showing signs of suicidal thoughts, disclose when conversations are AI-generated, remind minors to take breaks, and block sexually explicit AI-generated images. The proposal also bans chatbots from posing as healthcare professionals and requires companies to report their self-harm response protocols and crisis alerts to the state Department of Public Health.
An earlier version of the bill proposed stricter regulations, including bans on chatbots using unpredictable rewards to boost engagement, mandatory third-party audits, and a requirement that companies ensure bots don’t encourage suicide prior to public release. The final version significantly weakened these policies through last-minute amendments, including limiting protections to cases where companies know a user is a child, reducing reporting requirements, and removing the requirement for audits.
The changes led proponents like the Tech Oversight Project — a nonprofit watchdog that monitors tech companies — and the California branch of the American Academy of Pediatrics to ultimately withdraw their support (opens in new tab)..
“It’s worse than doing nothing,” Sacha Haworth, executive director of Tech Oversight Project, told the Sacramento Bee (opens in new tab). “It does more harm than good.”
The bill’s signing drew mixed reactions, even from those who agree with the need for user protections. Megan Garcia, whose son died by suicide after engaging with a companion chatbot, praised the law for preventing chatbots from discussing suicide with children or helping plan self-harm. However, Jim Steyer, a civil rights attorney who works for Common Sense Media, which withdrew its support for the weakened bill, criticized the final version as setting lower standards than other states and potentially misleading parents about the extent of regulatory protection.
Steyer, along with other advocates aiming for stricter regulation, had placed their hopes on the passage of AB 1064, which would have barred operators of California chatbots from offering them to children under 18 if the chatbots were capable of encouraging self-harm, offering therapy, or engaging in sexually explicit interactions. Proponents of the bill pointed to lawsuits filed by parents against AI companies like OpenAI after their children committed suicide, allegedly encouraged by chatbots.
“This is a very straightforward, simple choice,” Steyer said of the bill in a statement. “Are we going to put common-sense, thoughtful guardrails around these platforms that are having an extraordinary impact on our children, our teens, and our entire society? The choice is clear. We are on the right side of history.”
California Atty. Gen. Rob Bonta urged the governor to sign AB 1064 alongside a coalition of child advocacy groups. Opponents of the bill included Facebook’s parent company, Meta, and the libertarian think tank Cato Institute, founded by conservative donors Charles and David Koch. The Computer & Communications Industry Association, a trade association whose members include the world’s biggest tech companies from Amazon to Google, lobbied against the bill, saying it would threaten innovation in the world’s AI capital.