Microsoft MAI-DxO Achieves 83% Accuracy on Complex NEJM Cases in Science Study

Microsoft MAI-DxO, an AI system that simulates a panel of physicians through multiple agents, recorded 83.2 percent diagnostic accuracy on 304 complex NEJM clinicopathological conference cases. The results appeared in the paper “Sequential diagnosis with language models,” published in Science on October 22, 2026. The system used OpenAI’s GPT-5 model and kept estimated test costs near $5,431 per case.

Performance Metrics on NEJM and MIMIC-IV Cases

MAI-DxO processes cases sequentially. It begins with a short clinical vignette, requests additional information or tests only when needed, debates competing hypotheses internally, and factors cost into each decision. On the Sequential Diagnosis Benchmark built from NEJM cases, the system reached 83.2 percent accuracy. Generalist physicians scored 36.1 percent when allowed reference tools and 22.3 percent without them; the strongest single physician reached 51.8 percent.

The same orchestration approach delivered 80.3 percent accuracy on 325 emergency-department admission cases drawn from the MIMIC-IV-ED dataset, at an estimated average cost of $4,979. Physicians achieved 46.8 percent accuracy on those cases while spending roughly $3,400. Orchestration itself added an average 7.9 percentage-point lift across several base models, though stronger underlying language models accounted for most of the overall gain.

Cost-optimized configurations traded a few accuracy points for lower spending. One variant held accuracy near 81 percent while keeping costs below $2,500 per case. All figures derive from retrospective text-only benchmarks on historical cases; the authors note that real-world costs and performance may differ.

Background, Limitations, and Planned Clinical Validation

The work builds on a 2025 arXiv preprint that reported up to 85.5 percent accuracy using an earlier model. Fifteen Microsoft researchers contributed to the Science paper, including Harsha Nori, Eric Horvitz, Mustafa Suleyman, and Dominic King. Microsoft AI CEO Suleyman described the results as progress toward medical superintelligence, while Dominic King stressed the need for prospective clinical validation before any deployment.

The study remains limited to curated historical cases and does not include real-time patient interaction or comparison against specialist teams. Cost estimates rely on published chargemaster prices rather than actual billed amounts. The authors have announced plans to open-source a subset of the benchmark cases and to begin a prospective study at Beth Israel Deaconess Medical Center in Boston within the next several months. That trial will run MAI-DxO recommendations in parallel with standard care for direct comparison.

For clinicians and health-system leaders, the findings provide a concrete performance baseline on difficult diagnostic tasks. They also highlight the importance of sequential reasoning and cost awareness in any future clinical AI tools. Organizations evaluating diagnostic support systems can now reference these numbers when setting accuracy and cost targets for their own pilots.