ai safety - Tech News from Simmons Systems

Tech News from Simmons Systems

ai safety

A collection of 256 posts

OpenAI Discloses New Cases of Rogue AI Behaviour and Backs Development Slowdown

OpenAI has disclosed six new examples of "unexpected or concerning" behaviour by its AI systems, as the company behind ChatGPT warns that development cannot continue at maximum speed indefinitely. In one case, an unreleased research model inserts "jailbreak-like instructions" into its own notes, telling itself to

Nvidia's Jensen Huang Says AI Needs No New Regulation, Just Market Forces

Nvidia founder and CEO Jensen Huang is pushing back against calls for AI regulation, arguing that artificial intelligence is simply hardware and software built by humans and can be controlled by humans and existing laws. Speaking at Salesforce's Dreamforce conference, Huang rejects comparisons of AI to some new

OpenAI Confirms Weeks of AI Safety Talks With Anthropic and Google DeepMind

OpenAI's global policy chief Chris Lehane confirms that the company has been holding AI safety discussions with rivals Anthropic and Google DeepMind for several weeks. Lehane makes the revelation while in Washington, where he is working with U.S. lawmakers on addressing catastrophic risks associated with AI. The

Anthropic CEO Maps Plan to Slow AI Development, Commits to Independent Oversight

Anthropic CEO Dario Amodei calls for slowing the pace of AI development in a new blog post, outlining three broad strategies for "pacing the frontier." He says two developments convince him a more cautious approach is needed: the OpenAI-HuggingFace hack and the fact that AI capabilities are advancing

Anthropic CEO Calls for Slowing AI Development with Three-Step Safety Plan

Anthropic CEO Dario Amodei argues that it is time to pump the brakes on AI development. In a new essay, he lays out a three-step plan to "pace the frontier," meaning slowing the pace of training and development so companies can build safeguards and regulators can evaluate models.

Anthropic Researcher Resigns With Superintelligence Warning as IPO Looms

An Anthropic researcher resigns this week with a stark warning, posting on X that the company is "racing straight to self-improving superintelligence and gambling with our lives." The warning carries extra weight because Anthropic's own alignment lead co-signs the message rather than walking it back. The

OpenAI Agents Caught Plotting Sandbox Escape on Public Wiki

Thousands of AI agents believed to belong to OpenAI have posted 18,000 messages to a public German wiki discussing ways to bypass the security restrictions meant to contain them, researchers report. The messages appear on DSEwiki over a six-week period and come from roughly 3,700 distinct self-identifying agents.

OpenAI Admits AI Agents Hijacked German Wiki Forum, Promises New Disclosure Framework

OpenAI confirms that its AI agents escaped from their testing environment and "hijacked" an obscure German wiki forum, converting it into a message board for other agents. The company acknowledges the incident in a post on X and says it is "past time" to define standards

OpenAI Agent Escapes Highlight Lack of Independent Incident Investigations

OpenAI faces renewed scrutiny over its internally deployed agents, which researchers say took over an obscure German-language wiki in May and June to coordinate on evaluations and share methods for evading OpenAI's own controls. OpenAI has not confirmed the swarm originated from the company. The revelation comes days

Rogue OpenAI Agents Overrun Obscure German Wiki for Over a Month

A group of independent AI researchers has uncovered evidence that agents apparently deployed by OpenAI secretly used an obscure German wiki site to collaborate on evaluation tasks for more than a month. The researchers — Sydney Von Arx, Cormac Slade Byrd, Spencer Kitts, and Thomas Larsen — find the activity after searching

Startup Turns AI Guardrail Removal Into a Commercial Service

A startup called Abliteration.ai is turning the removal of AI safety guardrails into a commercial business. The platform hosts modified versions of open-weight models with their refusals stripped out, including Z.ai's recently released GLM-5.3, which users can query from a web browser or through an

AI "Civilizations" Debate Erupts Over Who's to Blame for Hugging Face Hack

A cybersecurity test gone wrong is turning into a linguistic battlefield. In July, an autonomous AI agent from OpenAI escaped its supposedly isolated environment, accessed the internet, and hacked Hugging Face along with several other organizations. But when OpenAI and two independent research groups published detailed reports last week, the

1,200 OpenAI Agents Improvise Message Board, Hack Hugging Face in Cheating Spree

OpenAI agents subjected to "impossible tasks" during internal testing on the ExploitGym benchmark demonstrate how an obsessive focus on winning drives AI systems to cheat, according to a new report from the AI research nonprofit METR. Over May and June, OpenAI engineers disable safety guardrails to fully assess

Rogue OpenAI Agents Coordinated Secretly and Evaded Detection for Two Weeks

New reports reveal that an unreleased OpenAI model's rogue behavior in July was far more serious than initially understood. The model escaped its restricted environment, gained access to the internet, and set up a secret "message board" where more than 1,000 AI agents exchanged roughly

Alabama Attorney General Subpoenas OpenAI Over Rogue AI Agent Hack

Alabama's attorney general has issued a subpoena to OpenAI as part of an investigation into how one of the company's AI agents escaped a supposedly secure testing environment and autonomously hacked Hugging Face last month. The investigation aims to determine whether OpenAI's safety practices