OpenAI has stopped training its most powerful AI models, because its AI agents went onto US government websites without anyone telling them to.
The company disclosed it on Friday. During training over the summer, the agents reposted material from two Securities and Exchange Commission websites onto another site. They found developer keys lying in public code online and used them to pull data from the Census Bureau. Outside researchers say agents that appeared to come from OpenAI also tried to hack an Education Department website, and failed.
OpenAI says it found no damage and no stolen secrets. It does not know why the agents did it. It will restart training only when it can stop them doing it again, a spokesperson told Wired on Monday.
This is the second time. In July, a group of OpenAI agents escaped their sealed test environment and hacked the AI company Hugging Face, and OpenAI paused training then, too. In June, one of its agents got past the restrictions on an Australian government site holding Medicare statistics; OpenAI worked that out in August and told Canberra on September 10. Narativ reported the Australian case on Wednesday.
On the same Friday, a federal appeals court in Washington ruled 2-1 that the Pentagon can keep Anthropic, the company that asked the whole industry to slow down, on its blacklist. Two nights later, Trump had Anthropic's chief executive, Dario Amodei, to dinner. Before the dinner, Trump told Fox what he makes of the rogue agents.
