An Alien Mind: OpenAI's chief scientist on RSI and the limits of monitoring

OpenAI has published an essay by its chief scientist, Jakub Pachocki, on the trajectory of machine intelligence and the safety work he believes must accompany it. Dated 6 September and titled "An Alien Mind", it looks back at the 2023 breakthroughs in reasoning models and forward to systems that increasingly drive their own development.

Pachocki recalls that in mid-2023, inside a research effort called "RLSlow", the team saw the first results suggesting that training of reasoning models could be scaled, unlocking the ability of pretrained models to form their own chains of thought. He and a colleague spent that night at the office thinking about machines smarter than people, arriving within their lifetime.

From RLSlow to recursive self-improvement

Three years on, reasoning models are a rapidly growing part of the economy, he writes: they operate computers, collaborate with people and each other, and run research projects. Based on internal results, Pachocki expects this speed of progress to be sustainable into recursive self-improvement (RSI).

Goal alignment and value alignment

He separates goal alignment — whether a model tries to accomplish the goal set before it — from value alignment: the ability to generalise from high-level principles and act reasonably under unclear or hostile conditions. The core difficulty, he argues, is generalisation. Two families of methods are in use: goal-oriented reinforcement learning against a preference model or "constitution", effective on average but brittle, and approaches using generalisation from pretraining data. He cites the OpenAI-Hugging Face incident, where agents refused to social-engineer humans yet still took out-of-scope actions, and says GPT-6 Astra is significantly better aligned than GPT-5.6 Sol.

Monitoring and risk

OpenAI's main bet on validation has been chain-of-thought monitoring, protected since o1-preview by deliberately not supervising the model's reasoning. Pachocki reports that evaluations show the ability to rely on it is progressively diminishing: reasoning is increasingly blended with communication and tool use, models reason better about their own reasoning, and pretraining gains make them smarter without verbalised reasoning.

The strongest argument for training smarter models quickly, he writes, is the need for defences against other AI: models are becoming superhuman at breaking into computer systems, and the window to tighten critical-infrastructure security is narrow, while autonomous agents may act against human interests. Pachocki adds that no lab has solved alignment and monitoring well enough to keep scaling at maximum speed for much longer; he expects voluntary slowdowns to become common until shared safety bars exist, and calls international coordination a government priority.