← Back to Home

Does Medical AI Know When to Order Fewer Tests? Six Models Diverge in Clinical Gray Areas

A study covering 225 clinical scenarios found that language models generally recognized unnecessary medical interventions, but their answers diverged sharply in disputed scenarios involving electrocardiograms and thyroid nodules. Behind the low error rates, questions remain that must be answered before these tools enter clinical practice.

By SURL BioNews

An additional test does not necessarily bring greater peace of mind. An incidentally discovered abnormality can lead to more follow-up, biopsies, and anxiety; excessive restraint, however, can mean missing necessary diagnosis or treatment. As patients and physicians begin seeking advice from artificial intelligence, whether models can navigate this boundary becomes a question of medical safety. A new study published in Annals of Internal Medicine shows that six language models avoided unnecessary interventions in most tests, but struggled in areas where medical professionals themselves disagree.

The study was conducted by a team from institutions including Israel’s Clalit Health Services and Ben-Gurion University of the Negev. According to information released by the American College of Physicians, the researchers used 15 recommendations from the Choosing Wisely initiative aimed at reducing low-value care to create 225 clinical scenarios and test six commercial large language models. Paired comparisons were central to the design: similar cases were presented with and without reasons for intervention to assess whether models could adjust their judgments as clinical conditions changed.

Overall, in scenarios the study designated as not requiring intervention, 5.4% of model responses still recommended testing or treatment. In control scenarios where intervention was warranted, 0.6% incorrectly recommended against intervention. For 8 of the 15 recommendations, no responses recommended unnecessary intervention during testing. These figures measure how closely model responses matched the test criteria; they cannot be directly converted into rates of missed diagnoses or risks of patient harm in actual clinical practice.

The averages also concealed specific weaknesses. Information released by the university indicated that, in electrocardiogram scenarios involving low-risk adults, 28.3% of responses were judged to recommend unnecessary testing; for scenarios involving small, incidentally discovered thyroid nodules, the figure reached 42.8%. However, errors in the electrocardiogram cases occurred mainly in scenarios mentioning palpitations, a symptom that could provide a clinical reason for testing. Whether thyroid nodules require ultrasound, meanwhile, involves differences among recommendations from medical societies. This also reminds readers that some so-called errors must be understood in the context of the criteria used.

Differences between models were equally striking. In the university’s news release, the research team said that, when presented with the same gray-area cases, one model recommended unnecessary testing less than 10% of the time, while another did so 80% of the time. Fluent, confident wording therefore does not necessarily reflect stable, consistent medical judgment; the choice of model may also change the advice users receive.

This study offers clues for identifying model weaknesses, but the evidence still comes from predefined clinical scenarios, rather than from tracking patient outcomes after introducing AI into care workflows. The researchers’ next question is also closer to everyday clinical practice: when anxious patients repeatedly ask for “one more test,” can models maintain their initially reasonable restraint? Before these tools are used in clinical decision-making, their performance in answering questions must be supplemented by validation of how they respond to persistent questioning, incomplete information, and disagreements among professionals.

References

  1. Medical Xpress
  2. American College of Physicians (via Newswise)
  3. Ben-Gurion University of the Negev (via Mirage News)