Microsoft’s Multi-Agent AI Outperforms Generalists in Complex Diagnosis Trials

Microsoft’s artificial intelligence diagnostic platform has demonstrated a significant advantage over generalist physicians in handling complex medical scenarios, according to a study published in *Science* on Thursday. The system, known as MAI-DxO, correctly identified diagnoses in difficult cases more than twice as frequently as individual primary care doctors and general internists. Mustafa Suleyman, the CEO of Microsoft AI, described this achievement as a critical step toward achieving what the company terms “medical superintelligence,” a state where AI surpasses human-led health systems in reasoning and coordination.

The architecture of MAI-DxO mirrors a human medical team rather than a single diagnostic engine. It coordinates multiple AI agents that function like a panel of specialists, engaging in a process where they question the patient, order necessary tests, and debate potential diagnoses. This collaborative approach was tested using complex cases drawn from the *New England Journal of Medicine*. While the AI system achieved an accuracy rate of 83%, the individual physicians, working with reference materials, reached 36%. It is important to note that these highly complex cases are typically managed by specialist teams in real-world settings, rather than by generalists working alone.

Microsoft’s entry into this specific arena places it alongside competitors like Google and OpenAI, which have similarly claimed that their AI capabilities can match or exceed human medical expertise. However, the internal data from the study suggests that the bulk of the performance improvement stems from the use of more powerful underlying AI models, with the multi-agent orchestration providing additional, though smaller, gains. Microsoft tested the system using models from various providers, including OpenAI, Google, Anthropic, Meta, and DeepSeek. The highest performance was observed when the system utilized OpenAI’s GPT-5 and GPT-5.5 models.

Cost efficiency and accuracy were both measured in the study. When using GPT-5 with a basic prompt, the system achieved 78.9% accuracy on complex cases at an estimated cost of $8,400 per case. The MAI-DxO orchestration reduced this cost by approximately $3,000 while increasing accuracy by only about four points. In emergency department scenarios, the addition of the orchestration layer yielded minimal improvements in either cost or accuracy compared to using GPT-5 alone. With the newer GPT-5.5 model, released in April 2026, the orchestration layer proved more effective, boosting accuracy on complex cases from 72% to nearly 87% at a slightly lower cost, although emergency department results remained largely unchanged at around 85%.

The system’s performance is most pronounced with less capable models and complex cases. For instance, Google’s Gemini 2.5 Flash scored 46.4% independently but rose to nearly 69% when assisted by MAI-DxO. Despite this boost, researchers noted that the gap between weaker and stronger base models remained larger than the improvement provided by the orchestration system. A Microsoft spokesperson explained that as models approach their theoretical maximums, accuracy gains naturally diminish, but the orchestration helps optimize the trade-off between cost and precision.

Suleyman, who leads Microsoft’s frontier AI efforts, stated that the company is collaborating with the Mayo Clinic to train a specialized foundation model for healthcare, with a release expected next year. He anticipates that this model will allow for significant leaps in medical AI. Suleyman predicts that by 2027, there will be a “coding style breakthrough” in medical records, enabling the streaming of highly accurate predictions about patient outcomes. He described this foresight as “gold dust” for clinicians. Suleyman, a former co-founder of DeepMind and Hark, joined Microsoft in 2024 to lead its AI unit, bringing a long history of integrating AI with clinical applications.

To validate these findings in real-world conditions, Microsoft has partnered with Beth Israel Deaconess Medical Center in Boston. This collaboration will evaluate the clinical reasoning capabilities of MAI-DxO across diverse hospital settings, including emergency and urgent care environments. Patients will receive standard care while consenting to have their cases processed in parallel by the AI system. The results will then be compared to those of human physicians. Dominic King, vice president of Health at Microsoft AI, called this the “biggest study of its kind,” noting that it will provide compelling evidence for the potential large-scale deployment of such systems.

This study coincides with a broader manifesto from Microsoft outlining the requirements for medical superintelligence, moving beyond simple benchmarks like answering medical exams to integrating with clinicians for holistic care. Suleyman believes this focus on comprehensive clinical decision support could differentiate Microsoft from other frontier labs. The company already sells dictation and radiology tools to healthcare providers and views MAI-DxO as a natural expansion of that portfolio. Suleyman asserted that the company is committed to maintaining the “absolute number one state-of-the-art” position in medical AI over the coming years, driven by a flywheel effect of real-world usage and iterative model improvements.