← Back to Home

When Heart Sound AI Starts Making Errors, Having Physicians Interpret Independently First Still Fails to Preserve Accuracy

A randomized experiment involving 48 participants found that as the reliability of simulated AI declined, the accuracy of combined human–AI interpretations also fell. The study brings clinical oversight into sharper focus: is human review alone enough to ensure safety?

By SURL BioNews

When medical AI makes errors, can the people providing the final check detect them in time? A heart sound interpretation experiment published in *npj Digital Medicine* on November 13 found that requiring participants to make their own judgments before viewing AI suggestions still failed to prevent final accuracy from declining as the system became less accurate.

The study randomly assigned 48 participants to two groups. Each participant interpreted 40 digital heart sound recordings, distinguishing normal sounds from abnormal sounds, including pathological murmurs. One group viewed AI suggestions first; the other answered independently before deciding whether to revise their answers. The paper collectively describes the participants as physicians, but its methods section specifies that 18 were final-year medical students, a distinction that must be retained when interpreting the findings.

The heart sounds came from the public CirCor dataset. The research team manually manipulated the AI suggestions so that accuracy declined across four stages to 90%, 60%, 50%, and 0%, respectively. This was therefore a behavioral experiment deliberately simulating a malfunction, rather than a record of an actual failure of a clinical product.

The average accuracy of combined human–AI interpretations fell from approximately 81% to 58%. During the low-reliability stages, participants corrected some AI errors, but their performance remained below the baseline for independent interpretation. Having participants answer independently first also failed to improve overall results. This suggests that forming one's own answer may not be enough to withstand the influence of subsequent incorrect suggestions.

After the experiment, only 12.5% of participants spontaneously mentioned the decline in system reliability. The study also found that, in the group that interpreted independently first, confidence in their own judgments showed a significant decline before trust in the AI did. The authors speculate that participants may have doubted themselves first, but this psychological mechanism remains to be verified.

These results cannot be directly extrapolated to risks in real clinical settings. The experiment lacked complete medical histories and physician–patient interaction, while the malfunction rapidly worsened in a fixed sequence. The final stage also introduced cues in which sounds and images were inconsistent, making it difficult to disentangle the effects of individual factors. Patient health outcomes were not measured.

The practical question raised by the study is how to define “effective oversight”: is it enough to perform better than malfunctioning AI, or must performance at least match physicians' independent interpretations? The authors argue that, alongside review by clinical staff, hospitals and developers should also participate in system-level monitoring. Which oversight arrangements can actually preserve diagnostic quality still needs to be tested in settings that closely resemble clinical practice.

References

  1. npj Digital Medicine