What happened
Microsoft published a study in Science demonstrating that its AI diagnostic system, MAI-DxO, outperformed generalist physicians on complex medical cases. The system, which coordinates multiple AI agents to simulate a panel of doctors, achieved an 83% accuracy rate on cases from the New England Journal of Medicine, while primary care doctors and general internists achieved 36%. Microsoft plans to evaluate the system in emergency and urgent care settings at Beth Israel Deaconess Medical Center.
According to a study published in Science, Microsoft’s AI diagnostic system, MAI-DxO, correctly diagnosed complex medical cases more than twice as often as generalist physicians in a simulation. The system coordinates multiple AI agents that function like a panel of doctors, questioning patients, ordering tests, and debating diagnoses. Using complex cases from the New England Journal of Medicine, the system was correct 83% of the time, compared with 36% for primary care doctors and general internists working individually with reference materials. Specialists and physician teams, who typically handle such cases, were not included in the comparison.
The study’s data suggests much of the improvement comes from stronger underlying models, with the DxO architecture adding further gains. Microsoft tested models from OpenAI, Google, Anthropic, Meta, and DeepSeek, achieving its best results with OpenAI’s GPT-5 and GPT-5.5. On complex cases, the system lifted GPT-5.5 from 72% to nearly 87% at slightly lower cost, while GPT-5 gained about four points and roughly $3,000 in savings per case. Neither model saw meaningful gains on emergency department cases. The system helped weaker models most, raising Gemini 2.5 Flash from 46.4% to nearly 69%.
Dominic King, vice president of Health at Microsoft AI, stated that the biggest challenge in healthcare is the growing gap between demand for and supply of medical expertise. Through a new partnership with Beth Israel Deaconess Medical Center in Boston, Microsoft will evaluate the system in emergency and urgent care settings, comparing its recommendations against those of doctors treating consenting patients. King said he expects it to be the largest study of its kind. Google recently published a similar Beth Israel study of its AMIE diagnostic AI in The Lancet.
Source details: theaiinsider.tech ↗
Why it matters
The study suggests that multi-agent AI architectures can significantly enhance diagnostic accuracy for complex cases, potentially addressing shortages in medical expertise. By leveraging advanced models like GPT-5.5, the system demonstrates practical improvements in cost and accuracy, offering a pathway for AI to support clinical decision-making in high-stakes environments where specialist availability is limited.
The results indicate that AI systems can provide substantial diagnostic support for complex cases, potentially alleviating the burden on generalist physicians who may lack specialized training for rare or intricate conditions. The significant accuracy gap between the AI system and generalist doctors highlights the potential for AI to serve as a decision-support tool in settings where specialist access is limited.
The study also reveals that the performance of AI diagnostic systems is heavily dependent on the underlying foundation models. The fact that the system improved the performance of weaker models like Gemini 2.5 Flash suggests that architectural innovations can extend the utility of less capable models, making advanced diagnostic support more accessible and cost-effective.
Interactive Mechanism: How It Actually Works
Explore the underlying technology behind this development interactively.
crm_get_transaction(id='4092').An agent must create a draft calendar event for Tuesday at 2 p.m. Which evidence would establish the requested result?
What to watch next
The upcoming evaluation of MAI-DxO at Beth Israel Deaconess Medical Center will provide real-world data on its performance in emergency and urgent care settings. Additionally, Microsoft's planned release of a healthcare trained with the Mayo Clinic next year may further advance clinical AI capabilities.
The real-world evaluation at Beth Israel Deaconess Medical Center will be critical in determining whether the simulated performance translates to clinical utility in emergency and urgent care environments. This study will compare AI recommendations against actual doctor decisions for consenting patients, providing a more rigorous test of the system's reliability.
Microsoft’s collaboration with the Mayo Clinic to train a healthcare , planned for release next year, could lead to more specialized and effective AI tools for clinical decision support. This development may further solidify Microsoft’s position in the healthcare AI market.