Validation of a machine-learning clinical decision aid for the differential diagnosis of transient loss of consciousness

Background and Objectives

The aim of this study was to develop and validate a machine-learning classifier based on patient and witness questionnaires to support differential diagnosis of transient loss of consciousness (TLOC) at first presentation.

Methods

We prospectively recruited patients newly presenting with TLOC to an emergency department, an acute medical unit, and a first seizure or syncope clinic. We invited participants to complete an online questionnaire, either at home or at time of initial assessment. Two expert raters determined the cause of participants' TLOC after 6-month follow-up. We used independent development and validation samples to train a random forest classifier to predict diagnosis from participants' questionnaire responses and validate classifier performance. We compared classifier performance against penalized linear regression and referrer diagnosis.

Results

We included 178 participants in the final analysis, of whom 46 identified a witness able to complete an additional witness questionnaire. Given low witness recruitment, we developed a classifier based on patient answers only. A classifier trained on 9 items correctly identified 63 of 78 diagnoses (80.8%) (95% CI 70.0–88.5), an increase over the accuracy of initial assessing clinicians who were only able to diagnose 70.5% correctly. Within this, 96% (87.0%–99.4%) of those expertly rated as having syncope were correctly classified by the classifier (classifier sensitivity); 40% (20%–63.6%) of those expertly rated after follow-up as having either epilepsy or functional/dissociative seizures were similarly classified as being nonsyncope (classifier specificity).

Discussion

A machine-learning classifier for differential diagnosis of TLOC has comparable performance in differentiating between 3 main causes of primary TLOC as the current standard of care but is insufficiently accurate in its current form to warrant incorporation into routine care. A system including information from witnesses might improve classification performance.