Medical Artificial Intelligence · us
Does Clinical AI Become More Accurate as It Reads More? Study Finds Benefits of Data Integration Depend on the Task
Combining medical records, radiology reports, and chest X-rays helps predict discharge diagnoses and medication use; for mortality risk and length of stay, structured medical records alone can deliver similar or even better performance.
After a patient is admitted, vital signs, laboratory results, images, and physician reports gradually accumulate. For artificial intelligence supporting clinical judgment, does reading all of this information lead to more accurate predictions? A University of Florida team found that the answer depends on the question: integrating data can improve predictions of diagnoses and medication use, but it does not necessarily make risk assessments more reliable, and performance is worse on some tasks.
The university described the study on October 26. The related paper was published in the Journal of the American Medical Informatics Association (JAMIA). The researchers combined data from MIMIC-IV, MIMIC-IV-Note, and MIMIC-CXR-JPG to establish an analysis cohort covering 22,254 hospital admissions. They compared general-purpose and medically adapted language and vision-language models using one, two, or three types of data.
According to the university, the models used only information from the first 24 hours of hospitalization to perform four tasks: predicting in-hospital mortality, predicting length of stay, identifying discharge diagnoses, and predicting which medications patients would receive. The last task assessed whether the models could predict actual medication use; it cannot be taken as direct evidence that they had learned to select the most appropriate treatment.
For diagnosis and medication tasks, the best models combining all three data types achieved F1 scores of 0.589 and 0.405, respectively, representing relative improvements of 21.4% and 18.4% over the best models using a single data type. F1 combines precision and recall; these gains do not mean that accuracy increased by the same number of percentage points, nor do they indicate improved treatment outcomes for patients.
Risk prediction showed a different pattern. Using structured medical records alone, the best AUROC values for mortality and length-of-stay predictions were 0.849 and 0.868, respectively, achieving the best or comparable performance. The model using all three data types listed by the university had a length-of-stay AUROC of 0.861, slightly below that achieved with medical records alone. AUROC reflects a model’s ability to distinguish between different outcomes; it cannot be directly interpreted as the proportion of patients whose outcomes were predicted correctly.
How images were incorporated also affected the results. The paper’s abstract states that, as supplementary inputs, text-based radiology reports consistently provided more benefit than single-view chest X-rays; using multiple image views performed better than a single view across all tasks. This suggests that the format and coverage of data also need to be evaluated according to their intended use, rather than simply counting how much information a model receives.
For now, this study provides a basis for designing clinical AI. The university noted that the analysis was limited to a single-hospital dataset and information from the early stages of hospitalization, and that performance on diagnosis and medication tasks remained insufficient to support bedside use. Before these models can enter clinical care, their reliability in other hospitals and real-world workflows still needs to be established, as does whether their predictions can actually improve medical decision-making.