biomedicine · us
AI Trained on Hospital Imaging Archives Brings Neuroimaging Diagnosis Closer to Real Clinical Practice
NeuroVFM shifts the focus of model training away from public internet data and back to the medical setting; its results suggest that MRI and CT scans accumulated in hospitals’ daily operations may be closer to the core problems of radiologic diagnosis than larger internet-scale models.
When AI enters medical imaging, the real difficulty is often not seeing a clear demonstration image, but understanding highly variable, uneven-quality scans in busy hospitals, where lesions can sometimes be extremely subtle. The NeuroVFM study, newly published in Nature Medicine, pushes this issue to the center: if a model learns directly from MRI and CT data accumulated over years by a health system, can it understand neuroimaging better than models that rely on public internet images or radiology-report text supervision?
NeuroVFM, proposed by the research team, is a general-purpose neuroimaging visual foundation model. According to the paper and official model materials, it was trained on data generated through more than 20 years of routine care at Michigan Medicine, at a scale of about 566,915 examinations and 5.24 million three-dimensional MRI and CT imaging volumes, covering examinations of the brain, head and neck, face, and orbits. These data were not carefully curated for competitions or textbooks, but came from everyday clinical practice, making them closer to the noise, workflows, and long tail of disease that radiology departments actually face.
Technically, NeuroVFM uses a volumetric joint-embedding predictive architecture called Vol-JEPA, allowing the model to learn representations of neuroanatomy and lesions in three-dimensional images, rather than only pairing images with text reports. The study states that the model can embed CT and MRI into a shared neuroanatomical latent space and connect different task heads for downstream diagnostic tasks; the official code repository also lists research components including a 3D Vision Transformer encoder, MRI and CT diagnostic heads, and a language-model workflow for generating imaging findings.
For validation, the study used temporally held-out clinical examination data and multiple public benchmarks, covering diagnostic classification for CT and MRI, report generation, urgency triage, and expert preference. The paper reports that NeuroVFM achieved an overall AUROC of about 92 or above across 156 diagnostic tasks and outperformed several comparison baselines, including models also using health-system data but trained with report supervision or reconstruction objectives, as well as models trained on internet-scale data or biomedical vision-language data. These comparisons do not mean the model can already diagnose patients independently, but they indicate that the clinical source of training data and the learning objective for three-dimensional imaging may both affect what a model can learn.
Report generation is a part of the workflow that is closer to clinical practice and also more prone to error. The research team used NeuroVFM as an image-aware module, paired it with open-source language models to generate radiologic imaging findings, and then used language models and clinical experts to evaluate urgency, correctness, and preference. The paper says reports generated by NeuroVFM had fewer fabricated findings and major errors, and experts also preferred its outputs; the University of Michigan MLiNS laboratory introduction similarly emphasizes its performance in visual localization, triage, and reducing hallucinations.
The biomedical significance of this study lies in making concrete the common statement that “medical AI needs clinical data.” Neuroimaging often contains identifiable facial structures, public datasets are limited in scale, and even very large internet models may not have seen enough variation in MRI and CT scans from real hospitals. NeuroVFM shows that if internal health-system data can be used under compliance and privacy protections, medical foundation models may not need to fully copy the path by which general AI expands through reliance on public internet data.
The limitations are equally clear. The official Hugging Face model card states that the NeuroVFM encoder is positioned as a frozen feature extractor for research use, is not a diagnostic device, has not been approved by the FDA, and requires outputs to be confirmed by clinical personnel. Its primary training data come from a single large academic health system. Although the study includes external and public benchmark testing, its stability across different countries, hospital equipment, population structures, and clinical workflows still requires prospective, multicenter, and regulatory validation.
### Background Context
Recent discussions of biomedical AI often stop at drug design or whether large language models can shorten the research and development process; NeuroVFM instead brings the focus back to image interpretation, one of the most intensive areas in the medical setting and one that most needs clear boundaries of responsibility. It is not a declaration that radiologists should be excluded from the workflow, but a more pragmatic question: when a model learns from a hospital’s own imaging experience, can it become a more reliable second reader, triage tool, or report-drafting system, and how should those uses be reviewed, deployed, and monitored?