Microsoft Declares Step Toward Medical Superintelligence With Diagnostic Orchestrator

Microsoft's Diagnostic Orchestrator scores higher than doctors in tests

Microsoft claims to have developed an AI-powered tool it says can diagnose conditions up to four times more accurately than human doctors. The company says the tool achieved about 83 percent accuracy on cases drawn from hundreds of reports published in The New England Journal of Medicine, compared with roughly 36 percent for generalist physicians using references and 22 percent for those working without them. On emergency department cases from a large database of patient admissions at Beth Israel Deaconess Medical Center in Boston, the system scored 80 percent accuracy, compared with an average of 47 percent for human physicians. The AI system also tended to reach diagnoses at a lower estimated cost than human counterparts. Microsoft compiled the cases into its own benchmark, SDBench, which it plans to open-source.

How the multi-agent system works

Designed to function like a panel of medical experts, the Diagnostic Orchestrator uses multiple AI agents to reason through symptoms and medical histories, decide what tests to order, and arrive at a diagnosis. Microsoft says this mimics the process doctors use to investigate cases. The system is powered by OpenAI's GPT-5 and also outperformed frontier AI models operating on their own, though the peer-reviewed paper does not include newer models such as GPT-6 Astra, Meta's Muse Wonder Full, Google's Gemini 4 Argon, and Anthropic's Opus 5.5. The Diagnostic Orchestrator was first developed when GPT-4.1 was the most advanced model available, and the researchers say subsequent testing suggests that improvements in model capability do not necessarily diminish the benefits of having multiple agents work together.

Validation and rollout remain ahead

Microsoft acknowledges a long road ahead before the technology can be widely deployed. The system has only been tested on past medical cases, and it remains unclear how well it would perform in real clinical settings. Microsoft AI CEO Mustafa Suleyman said it needs clinical validation. The company plans to begin testing the system in hospital settings with Beth Israel Deaconess Medical Center to see how it performs in real-world clinical environments. Neither AI health lead Dr. Dominic King nor Suleyman would commit to a firm timeline, though King said it will likely get underway in the next couple of months. Commercialization could also be on the horizon, though Microsoft has yet to say when or how the technology might become available. King emphasized the need for a responsible rollout, noting the high regulatory and compliance requirements in medicine.

Part of a broader medical superintelligence ambition

The research is part of a broader ambition at Microsoft to develop what the company calls medical superintelligence, meaning AI that can outperform the best human-led healthcare systems. Diagnosis is just part of Microsoft's health vision, King said. In a blog post published Thursday and a forthcoming article in the BMJ Digital Health & AI medical journal, the company outlines five other areas it believes AI will need to master, including coordinating clinical care, making treatment decisions, and handling the human side of medicine. Suleyman said next year will bring extremely impressive deployments of intelligent systems in production and that this is just going to change the way care is delivered in a big way. Even as he predicts something indistinguishable from human performance on medical superintelligence in the next couple of years, he insists that human clinicians should and will always have a central role in patient care. He said his hope is that AI will take some of the drudgery out of medicine, freeing up clinicians to spend more time with patients.

Sources: The Verge · Wired · Microsoft-Ai