Medical Imaging and Artificial Intelligence · global
How can mammography AI help? Retrospective study shows that integrated human–AI interpretation can improve detection performance
A multicenter study focused on image interpretations that physicians find uncertain. Integrating AI and physician assessments improved detection performance, but the cost of recall examinations, inconsistencies in some data, and effectiveness in real-world screening still need clarification.
The challenges of interpreting mammograms often lie in shadows and tiny calcifications that defy an immediate conclusion: missing a lesion may delay diagnosis, while excessive warnings can lead to additional examinations. A study published on October 13 in *Insights into Imaging* shows that whether AI can improve this trade-off is closely related to how it participates in physicians’ decision-making.
According to its abstract, this retrospective multicenter study included 130,838 mammographic images from 35,709 patients, with data drawn from multiple medical institutions in Iran. Beyond this large database, the actual reader experiment used 600 cases comprising 2,202 images from an independent test set, with four radiologists who had two to five years of mammography experience participating. The size of the entire database therefore cannot be treated as the number of patients used for clinical validation of human–AI collaboration.
The system used different neural networks to identify calcifications, delineate larger lesions, and then estimate malignancy risk, and was validated using the external CBIS-DDSM dataset. The original study and the abstract republished by JoVE Visualize listed five scenarios: physicians interpreting images alone, physicians revising their assessments after consulting AI, AI handling uncertain cases, integrated physician and AI judgments, and AI interpreting images independently. The JoVE content came from the same study and was not a separate independent validation.
When reader performance was pooled, the area under the curve (AUC) in the integrated judgment scenario increased from 0.8304 without AI to 0.8840, a statistically significant difference (p < 0.001). AUC reflects the overall ability to distinguish positive from negative cases; it cannot be read directly as “88.4% diagnostic accuracy.” When physicians simply reviewed AI results and revised their assessments themselves, the AUC was 0.8396, indicating that different integration methods did not yield the same benefits.
However, some metrics require cautious interpretation. The abstract attributed both an increase in sensitivity from 54.8% to 83.9% and an increase in specificity from 83.6% to 91.1% to the integrated scenario. The discussion in the main text, however, attributed the highest specificity of 91.1% to another strategy that used AI only when the assessment was uncertain. This inconsistency remains to be clarified, and these findings should not be used to assert that a single workflow can simultaneously achieve all the best results.
The study’s discussion also presented the cost behind the performance gains: in the integrated scenario, the simulated recall rate increased from 18.89% to 25.84%, while the positive predictive value declined. In other words, improved lesion detection may be accompanied by more people being asked to undergo further examinations. The improvements described in the abstract still need to be understood in the context of the specific decision rules and evaluation metrics.
External data validation provided a test of the model across data sources, but there is still some distance to go before routine screening. The study involved few interpreting physicians, used a retrospective design, and did not adequately analyze differences in breast density, age, or equipment. The selected cases may also not represent the general screening population. Prospective studies are needed next to confirm whether this collaborative approach can reduce missed diagnoses in real-world workflows and to assess the additional examinations and clinical workload.