Medical Technology · us
Interpreting Incomplete Questionnaires: Quest2Dx Explores Parkinson’s Screening Through Cross-Dataset Validation
Finding disease signals in hundreds of health questions depends not only on detection performance, but also on whether results remain reliable with a different questionnaire. A new model demonstrates the ability to interpret responses across questionnaires, but further validation is needed to establish whether it can identify patients earlier in clinical practice.
A health questionnaire may reveal clues that warrant further examination, but lengthy questionnaires and incomplete answers can also leave information buried. A research team at Rensselaer Polytechnic Institute (RPI) in the United States has developed an artificial intelligence model, Quest2Dx, that attempts to identify signals associated with Parkinson’s disease in self-reported questionnaires, with the aim of providing a more accessible initial screening approach for primary care.
The study, published on November 4 in *Nature Aging*, focuses on two practical barriers to questionnaire analysis: respondents skip questions, and different surveys use different wording to ask about similar things. Quest2Dx uses language modeling to encode the meaning of questions and answers, enabling the model to handle missing responses and transfer information learned from one questionnaire to another.
The study used two datasets: the Parkinson’s Progression Markers Initiative (PPMI Online) and Fox Insight. An RPI announcement and a report on the same study published by Medical Xpress stated that the two questionnaires contained 187 and more than 300 questions, respectively. The team trained the model on PPMI data and used Fox Insight for external validation. Compared with splitting data from a single source into training and test sets, this approach provides a stronger test of the model’s ability to adapt to different questionnaires.
The paper’s abstract further distinguishes between two types of evaluation: the model’s area under the receiver operating characteristic curve (AUROC) within each dataset was 0.977 and 0.974, respectively; when transferred from PPMI Online to Fox Insight, it was 0.920, and in the reverse direction, it was 0.952. AUROC measures a model’s ability to distinguish between two groups of people. It cannot be interpreted directly as the proportion of correct answers, nor does it mean the model has been shown to predict disease onset before symptoms appear.
Whether the questionnaire can be shortened also affects its usability in clinical practice. Based on the informational value identified by the model, the research team selected 30 questions to create a shortened version, which performed nearly as well as the full questionnaire. The model can also highlight combinations of questions and answers that are more discriminative for individual respondents, potentially allowing the basis for its assessments to be examined, rather than simply producing a score.
These clues include difficulty with daily activities such as getting out of bed and eating, as well as information about sleep, anxiety, and exposure to industrial solvents. However, related features may also occur in other diseases or life circumstances. The model’s finding that an answer helps with classification does not mean that answer can independently establish a diagnosis, much less demonstrate a causal relationship. How multiple pieces of information combine to form a risk signal is the question tools of this kind need to address.
The study showed that Quest2Dx outperformed several comparison methods, including an approach based on Llama 2, but it still produced false positives and false negatives. The current evidence supports its screening potential within the research data. Before it can be used in general outpatient care, the next step is to determine whether it can appropriately guide further evaluation for patients who have not yet been diagnosed and present with a mix of symptoms, and whether the additional referrals and examinations deliver practical benefits.