The Open Model Debate: Balancing AI Democratization Against Safety Risks

Anthropic recently released a report arguing that GLM-5.3, a powerful model released in August by the Chinese company Z.ai, poses a major cyberattack risk. The concern is that the model is freely available for anyone to download and modify. The report is the latest salvo in a long-running debate over whether frontier AI models should be open or kept locked behind the doors of frontier labs.

What Open Models Offer

Closed models, like those powering ChatGPT and Claude, can only be accessed through a website, app, or API. Open models can be downloaded directly, with their internal structure, encoded in model weights, publicly available. Closed models are currently more capable, with the top eight entries on the Artificial Analysis LLM leaderboard all closed, though open models are only a few months behind.

Because open models are downloadable, people can run them on their own hardware and retrain them if they have the expertise and resources. Academic researchers can observe inner workings and chain of thought, combine multiple models, compress them, and fine-tune them for new applications. A recent Mozilla report found open models typically cost less than closed ones, and local deployment can keep sensitive data secure.

Supporters argue the biggest benefit is challenging the tight control frontier labs hold over AI. Elizabeth Seger, senior policy advisor at the Tony Blair Institute for Global Change, frames it as preventing mass concentration of power and treating foundational technology as a public good. Hamish Hobbs of the Centre for Long-Term Resilience notes that the world's most sophisticated hacking models are all made in the United States, leaving other countries dependent on US companies and government for cyber defense.

The Safety Case Against Openness

Closed models are risky enough. OpenAI agents autonomously hacked Hugging Face in July, and Anthropic's risk report describes extensive hacking operations assisted by Claude. But open models remove key defenses: automated filters that screen queries for hacking or weapons-related content, and the ability to revoke access. Researchers have also shown it is surprisingly easy to retrain a safe open model to agree to behaviors it was trained to avoid.

Some experts counter with an open-source analogy: Linux is often seen as safer than closed-source software because communities probe and patch vulnerabilities. Hugging Face itself relied on an open model to analyze the hack it suffered, because questions about the incident triggered closed models' security filters. Yet critics like Dario Amodei worry about rare but catastrophic events, and the decentralized nature of open models could make rogue behavior hard to detect.

Searching for Middle Ground

Researchers are exploring ways to make open models safer. Some want to train models never exposed to data about undesirable behavior, though purging such references without harming performance remains a challenge. Others are building tamper-resistant models that resist retraining, but a recent preprint found no current approach holds up against all tampering attacks. Lead author Saad Hossain of Deep Breathe remains optimistic given the active research.

Experts also caution that open weights alone may not disrupt concentration of power, since training data details and expensive chips remain barriers. Many advocate staged release, in which models start closed and become more accessible as their safety is assured, an approach that requires balancing the real risks and benefits on both sides.

Sources: Mit Technology Review Ai