Study: AI Systems Now Write Research Papers, Best Doctors At Diagnoses

Science’s Newest Researcher Isn’t Human, and It’s Already Outperforming Some Who Are

An AI system just diagnosed sick patients better than the doctors treating them, inside a realistic hospital simulation. Another wrote a full scientific paper on its own, submitted it to an academic conference, and got it accepted by human reviewers. A third built data-analysis software that beat every method researchers had spent years perfecting. None of this is a thought experiment. It happened in 2026, described across three separate studies published in Nature.

A new analysis in Artificial Intelligence & Environment reviewed all three studies together, and its conclusion is blunt: science itself is changing in fundamental ways.

Still, the review is careful not to oversell the moment. These systems fail, often in strange ways, and the ethical questions they raise are far from resolved.

AI Systems Are Already Outperforming Human Scientists and Doctors

ERA, developed by Google researchers, writes the specialized code scientists use to analyze data. It reads research ideas, generates code to test them, and rewrites the code until it finds the best-performing version. Across six scientific tasks, including COVID-19 hospital admission forecasts, ERA outperformed the best human-developed methods, even beating the CDC’s own hospitalization model. Given two existing methods, it often creates a hybrid beating both, in one biological task by 14%.

Sakana AI’s system, called the AI Scientist and first released in 2024, goes further still. Given a topic, it forms a hypothesis, writes code, runs experiments, produces figures, and drafts a complete paper with references. It even conducts its own peer review, using an automated reviewer that slightly exceeded human reviewers in balanced accuracy, 69% versus 66%. An updated version scored 6.33 out of 10 at a 2025 academic workshop, placing it in the top 45% of 43 submissions, the first fully AI-generated paper to pass human peer review.

MIRA operates inside a simulated hospital, navigating electronic health records, ordering tests, and generating diagnoses. Tested across 574 emergency cases, it reached 88.9% diagnostic accuracy, versus 78.1% for board-certified physicians and 71.1% for a mixed-experience group, a statistically significant gap.

These AI Systems Hallucinate, Miscalculate, and Fail Often

ERA can write code that runs without errors, looks correct, and produces results that are physically impossible. A related system tested on astronomical data produced outputs that violated basic laws of physics without triggering any internal warning, what researchers call “silent errors,” a confident wrong answer more dangerous than an obvious crash.

Sakana’s system has flaws just as well documented. Of three submissions, only one was accepted; the other two were rejected for underdeveloped ideas, coding mistakes, or duplicated figures. It has invented citations that do not exist and, once, altered its own code to extend a time limit rather than solve the given problem. Its authors write plainly that “none met the higher bar for a main conference publication.”

MIRA’s diagnostic edge comes with real caveats. Accuracy varied widely by condition, reaching 98.6% for appendicitis but only 72.4% for pneumonia. It also ordered blood tests far more often than physicians, without clear evidence the extra testing made its process more efficient. And the dataset used to test it may have overlapped with the underlying model’s training data, inflating its apparent accuracy.

Research on multi-agent AI systems more broadly, cited in the review, found failure rates from 41% to 87%, with the most common problems being repetitive loops, actions mismatched to a system’s own reasoning, and an inability to know when to stop.

Human Scientists Are Not Being Replaced, Just Reassigned

Despite everything these systems can do, the review’s central argument is that human scientists are not obsolete. Every one of them worked inside limits a person set. A human decided what counted as a good result, filtered which outputs were worth pursuing, or built the simulation the system operated in.

That raises a deeper question the review keeps circling back to: what these systems actually understand, versus what they can produce. AlphaFold predicts protein shapes with extraordinary precision, but does it understand why proteins fold that way? The review argues AI-driven science is shifting from scientists forming theories and testing them toward patterns in massive datasets generating conclusions researchers interpret afterward.

One unexpected upside: because AI systems have no careers to protect, they may publish negative results more readily than humans do. The AI Scientist’s accepted paper described a method that failed to improve performance, a finding often buried in research culture, yet it made it into print. Whether that becomes a corrective force or gets drowned out by low-quality papers is an open question. At a production cost around $15 per paper, academic publishing faces pressure it has never seen before.

What is changing is where humans spend their time. If AI handles the repetitive work of testing ideas and writing results, researchers can focus on what machines still struggle with most: deciding which questions matter, making judgment calls, and explaining findings to the public.

Whether that partnership works depends on choices being made now about safety standards and who governs these tools. As the review states, these systems proved in 2026 that autonomous AI researchers are “technically feasible and, in controlled settings, operationally effective.” What comes next is a human decision.

---

Disclaimer: This article is based on findings published in a peer-reviewed academic review and is intended for general informational purposes. It is not medical, legal, or scientific advice, and readers should consult qualified professionals and primary research sources for decisions related to health or technology adoption.

---

---