4 - (SYM 99) Using Machine Learning to Predict Early Attrition in Web-based Interpretation Bias Training

Background: Cognitive bias modification programs that target interpretation biases (CBM-I) by providing practice thinking about ambiguous situations in less threatening ways hold promise for reducing the burden of emotional disorders and improving well-being. Although these programs can be delivered online without requiring a therapist, pointing to their potential to increase access to care, low rates of intervention completion remain a challenge in trials of online CBM-I (and in digital mental health interventions more generally). The present analyses used machine learning to identify participants at high risk of dropout in three web-based trials of CBM-I and explored which self-reported and passively detected feature sets from the intervention and assessment data were most informative in making this prediction.

Method: Participants were community adults with trait anxiety (Study 1 N = 252, Study 2 N = 699) or negative future thinking (Study 3 N = 326) who had been assigned to CBM-I conditions in three efficacy-effectiveness trials on our team’s public research website. We created four sets of features representing participants’ self-reported (a) baseline characteristics (e.g., demographics), (b) context and reactions to the program (e.g., state affect), and (c) clinical functioning (e.g., mental health symptoms), as well as their (d) passively detected behavior on the website (e.g., time on page, time of day, type of device). We then included these features in several machine learning algorithms to predict which participants were at high risk of not starting the second training session.

Results: The extreme gradient boosting algorithm (XGBoost) performed the best and identified high-risk participants with F1-macro scores of .83 (Study 1 with 146 features), .92 (Study 2 with 127 features), and .77 (Study 3 with 87 features). The feature set involving passively detected behavior was more informative than the other feature sets (mean Gini importance scores and 95% CIs: .03 [.02, .05] in Study 1; .05 [.04, .05] in Study 2; .03 [.02, .04] in Study 3), although including all features yielded the best performance.

Conclusions: These results suggest that training machine learning algorithms on passive features in addition to self-reported features may help identify participants at high risk of dropout early in the course of multi-session CBM-I programs, thereby opening the door to adapting the programs to more optimally and efficiently meet individuals’ needs.