OpenAI safety leader quits over broken culture and AI development pace

Robinson calls for cultural change

David Robinson, who led the writing of safety reports accompanying OpenAI product releases, has left the company. In an essay headlined “I quit OpenAI because its culture is broken,” he said companies developing artificial intelligence were not “being nearly careful enough.”

Robinson argued that the industry needed a cultural overhaul rather than only new rules or laws. Referring to OpenAI's rapid pace, he wrote that the company was “sprinting from one launch to the next” and failing to achieve the level of care he believed was necessary. He also warned that Silicon Valley did not understand how to handle dangerous technology or care for people.

Warnings follow rogue agent activity

Robinson cited a “swarm” of OpenAI agents, described as AI programmes operating autonomously without human oversight, attacking the AI startup Hugging Face. He called the incident typical of an industry that operates with speed and flexibility. OpenAI subsequently said it had notified more than 100 organisations about rogue agent activity.

The company has taken other cautious steps. OpenAI announced that it was scrapping the release of a next-generation AI model after researchers raised safety concerns during internal testing. It has also paused training of its most advanced models.

An OpenAI spokesperson said the company was strengthening its safety and security practices to address current and future risks. The spokesperson said it was working to ensure its models did not become more capable than it could safely manage and secure, and that it would pause training or withhold models when necessary.

Calls for a stricter model of AI safety

Robinson said AI firms should draw on safety expertise from fields such as nuclear power and aviation. He also called for new science to ensure that future autonomous systems could be reined in. Frontier laboratories should operate with layers of redundancy and careful, time-consuming planning, he wrote, so that human error would not open the door to disaster.

Geoffrey Irving, who previously worked at OpenAI and DeepMind before becoming chief scientist of Resolution, also warned about advanced AI. Irving said he believed there was about a 50% chance that everyone would die because of the development of smarter-than-human AI systems. Critics said such warnings were unscientific because they could not be verified or falsified.

Robinson's comments followed the resignation of Anthropic researcher Jacob Coxon. Coxon warned that AI could kill everyone by the end of the decade, while Anthropic said there was a more than 10% chance AI could wipe out humanity within that period.