How Strictly Should Medical AI Be Regulated? Nature Health Argues It Depends on How Much Decision-Making Authority It Holds

The risks of medical AI depend not only on whether a model gives incorrect answers, but also on whether people act on those answers. As chatbots, wearable devices, and health applications increasingly enter everyday life, algorithms with medical influence are no longer confined to hospitals. If a recommendation can change whether a patient seeks medical care, takes medication, or undergoes testing, its consequences may extend far beyond those of a general information tool.

In a commentary published in Nature Health, Yu Gu and Eric J. Topol propose that health AI should be assessed and regulated according to the extent of its influence over individuals’ clinical and health decisions. This perspective shifts the focus away from product names, settings of use, or a single technical metric toward a more direct question: To what extent can a system shape a person’s health trajectory?

Under this logic, a tool that merely helps organize medical records and whose output must be reviewed by a professional should not be subject to exactly the same requirements as a system that directly provides diagnostic, triage, or treatment recommendations. A symptom assessment application or health assistant used independently by patients should not be considered low risk merely because it is labeled “for informational purposes” if it could lead users to delay seeking care, adjust medications, or take other health-related actions, even if it is not deployed within a traditional healthcare institution.

“Decision-making authority” also brings factors beyond model accuracy into the regulatory field of view. If the same model is used only as a reference in one setting but can automatically take action or serve as the user’s sole available source of advice in another, the actual risks are not equivalent. Whether humans can override the output, whether errors are reversible, how recommendations are presented, and whether the system declines to answer when faced with uncertain information can all alter the influence AI actually holds.

To implement this approach, regulators would still need to translate the abstract “degree of influence” into inspectable standards, including who holds final decision-making authority, whether AI outputs trigger subsequent actions, whether users understand the system’s limitations, and whether product updates alter the original boundaries of its authority. Tools used outside healthcare institutions are particularly challenging because their contexts of use are dispersed, and monitoring errors and tracking harm are generally more difficult than within hospitals.

This article is a policy commentary, not a clinical study validating a new system. The publicly available abstract does not present classification thresholds, datasets, or performance data, nor can it demonstrate that classifying systems by decision-making authority can directly reduce harm. What it offers is a regulatory principle: medical AI should not be labeled solely according to its technical form. Instead, requirements for evidence, human oversight, and continuous monitoring should be proportionate to how much scope for action it is given in the real world.