← Back to Home

Combined AI identifies 63 of 65 lung cancers, but high sensitivity comes with many false positives

A retrospective study covering 5,110 low-dose CT scans shows the potential of integrating different AI approaches to image interpretation; however, with a positive predictive value of just 2.7%, whether it can improve screening care remains to be validated.

By SURL BioNews

Lung cancer screening aims to detect tumors early while also avoiding unnecessary tests for large numbers of people who do not have cancer. A study published in *Radiology: Cardiothoracic Imaging* combined two artificial intelligence models with nodule interpretation rules, identifying 63 of 65 lung cancers diagnosed within one year. However, only 2.7% of the system’s positive signals corresponded to lung cancers diagnosed during that period, highlighting the trade-off between missing fewer cancers and controlling false positives.

The study retrospectively analyzed baseline low-dose CT scans from two lung cancer screening cohorts: 2,343 scans from the U.S. National Lung Screening Trial (NLST) and 2,767 scans from the Pamplona International Early Lung Cancer Action Program (P-IELCAP). According to the study description published by HeartLung.AI, the endpoint was histologically confirmed lung cancer within one year after the scan, rather than whether suspicious nodules appeared on imaging.

The combination was designed to allow different interpretation approaches to complement one another. Sybil analyzes the entire scan, while another model based on MONAI focuses on lung nodules. The study also added a rule that flagged nodules detected by MONAI with a diameter greater than 6 millimeters and a detection confidence score above 0.95. This score reflects the model’s confidence in detecting a nodule and cannot be interpreted as a greater than 95% probability of cancer.

The combined model achieved a sensitivity of 96.9%, still missing two cancers. Its specificity was 55.0%, meaning that approximately 45% of the noncancer scans included in this calculation were labeled positive. The abstract reported that only 63 of 2,297 positive predictions corresponded to lung cancers diagnosed within one year. In other words, only about three of every 100 positive signals met the study’s defined cancer endpoint. If these alerts enter clinical practice, how review and follow-up are organized will be key to their practical usefulness.

The two screening cohorts also had different underlying risk profiles: lung cancer incidence was 17.7 cases per 1,000 person-years in NLST and 8.7 in P-IELCAP. Sybil’s area under the receiver operating characteristic curve (AUC) was 0.93 in both cohorts, while the MONAI model’s AUC was 0.90 and 0.79, respectively. AUC describes a model’s ability to distinguish cancer from noncancer; it does not equate to the accuracy of positive predictions at a particular alert threshold. The latter is also affected by the proportion of people with cancer in the screened population.

Some aspects of the publicly available information still need clarification. The abstract listed a total of 5,110 scans, but the 4,965 noncancer scans used to calculate specificity, together with the 65 cancers, total 5,030. The abstract did not explain why the remaining 80 scans were absent from this denominator. In addition, the HeartLung.AI page still marked the manuscript as submitted, while the PubMed record reproduced by Life Science Network already listed the journal volume, issue, and article number, making the two publication status labels inconsistent.

These findings support further evaluation of the combined model but do not yet demonstrate that it can improve patient outcomes. Retrospectively identifying cancers in existing images provides a different level of evidence from actually helping physicians diagnose cancer earlier and reduce unnecessary procedures. HeartLung.AI also noted that prospective studies are still needed to confirm performance in routine practice and clinical benefit. The next question is whether high sensitivity can translate into better screening care with a manageable follow-up burden.

References

  1. OpenRounds, referencing Radiology: Cardiothoracic Imaging
  2. HeartLung.AI
  3. Life Science Network