California Legislature Passes Landmark AI Safety Bill Targeting Powerful Models
California lawmakers approve a bill forcing developers of large AI systems to test for critical societal harms before deployment. The legislation now awaits a decision from Governor Gavin Newsom, who faces intense pressure from both tech giants and AI safety advocates.
California lawmakers approve a controversial bill that requires companies developing or modifying powerful artificial intelligence systems to test their models for severe safety risks. Under Senate Bill 1047, businesses that spend over $100 million to train an AI model or $10 million to modify one must evaluate the technology for its potential to enable cyberattacks, infrastructure threats, or the creation of chemical, biological, radioactive, and nuclear weapons.
The legislation creates a major rift within the technology sector, drawing fierce opposition from major companies like Google, Meta, and OpenAI. These opponents argue that the compliance costs will damage the industry, burden startups, and discourage the release of open-source AI tools due to fears of legal liability. Conversely, the bill receives strong backing from prominent figures like Elon Musk, AI researcher Yoshua Bengio, and former OpenAI whistleblowers who insist that federal regulators are failing to address significant AI threats.
The future of the landmark proposal now rests entirely in the hands of Governor Gavin Newsom. Eight members of California's congressional delegation recently urge Newsom to veto the bill, leaving his final decision uncertain as he balances the need for safety regulations against the risk of overregulation in a state that hosts the world's leading AI companies.