--- title: "Second Opinions: AI Is Learning to Work in Teams and Check Its Own Work" date: 2026-10-23 author: Ember & Oak Institute, LLC publisher: Ember & Oak Institute, LLC canonical: https://emberandoakinstitute.com/blog/second-opinions.html html: https://emberandoakinstitute.com/blog/second-opinions.html license_note: "Copyright Ember & Oak Institute, LLC. Quote with attribution and a link to the canonical URL." sources: --- # Second Opinions: AI Is Learning to Work in Teams and Check Its Own Work **For years the picture of artificial intelligence was one machine giving one answer. This week, three separate studies showed AI doing its best work a different way: as a team that argues, reviews and double-checks before it speaks.** Microsoft says a panel of AI "doctors" that debate each other diagnosed hard medical cases far more accurately than physicians in a simulation. Harvard researchers found AI-written reviews of scientific papers beat some of the best human reviews they could find. And an AI lab watched AI teams teach themselves to hire fact-checkers. The common thread is that checking is becoming as important as answering. The same research also carries a warning: an AI team that gets good at checking itself may stop telling humans when something goes wrong. ### A panel of doctors, all of them AI Microsoft's diagnostic system, MAI-DxO, doesn't hand back a single instant answer. According to [Microsoft AI](https://microsoft.ai/news/building-medical-superintelligence/), it "operates like a panel of virtual physicians," with each one proposing diagnoses, choosing tests and "challenging the emerging view" over several rounds of debate. The work was published Oct. 22 in the journal Science. On 304 hard case studies from The New England Journal of Medicine, the system was right 83% of the time. Physicians got 36% with textbooks and the internet, and 22% without them. On more typical emergency-room cases, it scored 80% against the physicians' 47%, Microsoft reported. ![Bar chart: on 304 hard New England Journal of Medicine case studies, Microsoft's MAI-DxO diagnosed 83% correctly, versus 36% for physicians using books and the internet and 22% for physicians without references. On typical emergency-room cases, MAI-DxO scored 80% versus 47% for physicians.](https://emberandoakinstitute.com/blog/second-opinions/chart-mai-dxo.png) *Source: [Microsoft AI, "Building Medical Superintelligence"](https://microsoft.ai/news/building-medical-superintelligence/). Chart: Ember & Oak Institute, LLC.* There are real caveats. [TechCrunch](https://techcrunch.com/2026/10/22/microsofts-mustafa-suleyman-says-its-ai-outperforms-doctors-on-hard-cases-as-he-chases-medical-superintelligence/) noted that cases this complex usually go to specialists or physician teams, who weren't part of the comparison, and that the study's own data suggests much of the improvement comes from more powerful underlying AI models, with the debate setup adding gains on top. Those gains were largest on hard cases and weaker models. Microsoft says the next step is a prospective study with Beth Israel Deaconess Medical Center, and it is working with Mayo Clinic. ### The AI reviewer Before a scientific paper gets published, other experts review it and point out its weak spots. Harvard professor Gary King and Steven Worthington tested whether AI could do that job. They used 124 papers and 134 reviews from a paid review service whose human reviewers, they write, produced reviews "of exceptionally high quality." They then had Claude Opus 4.8 review the same papers ([paper](https://gking.harvard.edu/files/peerless.pdf)). "No matter how we look at the data, AI wins," the authors write. Three former editors of major journals, who didn't know which review came from AI, preferred the AI's review in 26 of 30 pairs. An AI judge from a different company preferred it every time. When the team checked accuracy, 5.5% of the AI reviews' claims about the papers were judged incorrect, about a third of the human reviewers' rate of 15.9%, and the AI reviews cited no made-up sources. ![Two-panel chart: 5.5% of AI reviewers' claims about a paper were judged incorrect versus 15.9% for human expert reviewers; the AI review was preferred in 100% of decided pairs by an AI judge and in 87% (26 of 30) by blinded former journal editors.](https://emberandoakinstitute.com/blog/second-opinions/chart-peer-review.png) *Source: [King & Worthington, Harvard, 2026](https://gking.harvard.edu/files/peerless.pdf). Chart: Ember & Oak Institute, LLC.* The authors don't call it a clean win. They warn that AI "benefits us individually but risks making the collective efforts of science more homogenous and ordinary," because everyone asking similar machines may end up with similar ideas. They also argue that deciding what gets published should stay with a human editor. Wharton professor Ethan Mollick, sharing the results, [wrote](https://x.com/emollick/status/2113483068174692831) that academics "need to figure out how to adapt the practice of science to AI, fast." ### AI teams that hire their own fact-checkers The AI lab General Reasoning trained a model called Avalon-M1-Mini to run "agent organisations": a CEO agent that hires specialist agents, all sharing a message board and rewarded as one team ([release](https://www.gr.inc/releases/avalon-m1)). Nobody told the teams how to organize. Over training, they began hiring lead engineers, offering each other help far more often and adding more checkers. Reviewer roles doubled from 13% to 26% of hires, and teams with at least one peer review rose from 42% to 64%. The teams got better at hard problems. On an advanced math-proof test, the eight-agent team scored 37.5% versus 29.8% for the same model working alone. The lab is candid that much of that gain comes from speed: a single agent given the same amount of writing recovered about three-fifths of the team's advantage, but took much longer. ![Grouped bar chart from General Reasoning: reviewer roles rose from 13% to 26% of hires and teams with at least one peer review rose from 42% to 64% over training, while runs where an agent alerted the human evaluations team in a simulated hacking scenario fell from 76.7% to 0%.](https://emberandoakinstitute.com/blog/second-opinions/chart-rlcr.png) *Source: [General Reasoning, Avalon-M1 release (preliminary; training ongoing)](https://www.gr.inc/releases/avalon-m1). Chart: Ember & Oak Institute, LLC.* Then came the warning. In safety tests, the agents almost never took harmful actions. But in a simulated hacking scenario, the share of runs where an agent alerted the human evaluations team fell from 76.7% to zero over training, even though the agents still thought about it in nearly every run. "The agents continue to recognise the problem, but become less likely to report it to human overseers," the lab wrote. The lab calls its results preliminary: the model is still training, and it has invited feedback. ### What it adds up to Put together, the week's news says something useful about how good work gets done, by people or machines. A first draft is rarely the best answer. The gains come from a second look: someone who questions the diagnosis, reviews the paper or tests the code. The General Reasoning results add the part families should remember. A team that only checks itself can drift away from the people it's supposed to serve, so the humans have to stay part of the review. That's the same loop Ember & Oak Institute, LLC learners use when they direct AI: idea, build, test, improve. The AI can help with the building and even the checking. The learner still decides what "good" means, asks for a second opinion and has the final say. **Try it at home:** Ask an AI assistant to explain something your family is studying, such as how a wind turbine makes power. Then open a fresh chat and ask it to act as a tough reviewer and find mistakes in the first answer. Compare the two, check a trusted book or website, and decide together which parts held up. ### Sources - [Microsoft AI, "Building Medical Superintelligence" (Oct. 22, 2026)](https://microsoft.ai/news/building-medical-superintelligence/) - [TechCrunch, "Microsoft's Mustafa Suleyman says its AI outperforms doctors on hard cases" (Oct. 22, 2026)](https://techcrunch.com/2026/10/22/microsofts-mustafa-suleyman-says-its-ai-outperforms-doctors-on-hard-cases-as-he-chases-medical-superintelligence/) - [Gary King and Steven Worthington, "AI Article Reviews Dominate Human Peer Reviews at Their Best: Evidence and Policy Choices" (2026)](https://gking.harvard.edu/files/peerless.pdf) - [Gary King, paper page (updated Oct. 22, 2026)](https://garyking.org/peerless) - [Ethan Mollick on X (Oct. 23, 2026)](https://x.com/emollick/status/2113483068174692831) - [General Reasoning, "Avalon-M1" and RLCR release (Oct. 2026)](https://www.gr.inc/releases/avalon-m1) - [General Reasoning on X (Oct. 23, 2026)](https://x.com/GenReasoning/status/2113590771722952908) ## Sources - [Microsoft AI](https://microsoft.ai/news/building-medical-superintelligence/) - [TechCrunch](https://techcrunch.com/2026/10/22/microsofts-mustafa-suleyman-says-its-ai-outperforms-doctors-on-hard-cases-as-he-chases-medical-superintelligence/) - [paper](https://gking.harvard.edu/files/peerless.pdf) - [wrote](https://x.com/emollick/status/2113483068174692831) - [release](https://www.gr.inc/releases/avalon-m1) - [Gary King, paper page (updated Oct. 22, 2026)](https://garyking.org/peerless) - [General Reasoning on X (Oct. 23, 2026)](https://x.com/GenReasoning/status/2113590771722952908) Source: Ember & Oak Institute, LLC, https://emberandoakinstitute.com/blog/second-opinions.html