Identifying factors associated with user retention and outcomes of a digital intervention for substance use disorder: A retrospective analysis of real-world data

Objective.

Successful delivery of digital health interventions is affected by multiple real-world factors. These factors may be identified in routinely collected, ecologically valid data from these interventions. We propose ideas for exploring these data, focussing on interventions targeting complex, comorbid conditions.

Materials and Methods.

This study retrospectively explores pre-post data collected between 2016 and 2019 from users of digital cognitive behavioural therapy (CBT) - containing psychoeducation and practical exercises - for substance use disorder (SUD) at UK addiction services. To identify factors associated with heterogenous user responses to the technology, we employed multivariable and multivariate regressions and random forest models of user reported questionnaire data.

Results.

The dataset contained information from 14,078 individuals of which 12,529 reported complete data at baseline and 2,925 did so again after engagement with the CBT. 93% screened positive for dependence on one of 43 substances at baseline, and 73% screened positive for anxiety or depression. Despite pre-post improvements independent of user sociodemographics, women reported more frequent and persistent symptoms of SUD, anxiety and depression. Retention - minimum two use events recorded - was associated more with deployment environment than user characteristics. Prediction accuracy of post-engagement outcomes was acceptable (AUC: 0.74-0.79), depending non-trivially on user characteristics.

Discussion.

Traditionally, performance of digital health interventions is determined in controlled trials. Our analysis showcases multivariate models with which real-world data from these interventions can be explored and sources of user heterogeneity in retention and symptom reduction uncovered.

Conclusion.

Real-world data from digital health interventions contain information on natural user-technology interactions which could enrich results from controlled trials.