(PS13-C70) Using a Novel Method to Measure Collaboration in Usual Care Between Veterans with SMI and Their Clinicians

Collaborative Decision Skills Training (CDST) is a skill-based intervention designed to increase patient-provider collaboration in treatment decision-making (Treichler et al., 2018). While skills-based intervention studies often find improvements in self-report measures, findings do not consistently generalize to community contexts (Bowie et al., 2020). Evaluating the real-world effectiveness of CDST requires novel measurement strategies. Nine Veterans with serious mental illness at the San Diego VA participated in an eight week open trial of CDST, in which we tested a novel method to measure collaboration during usual care. We aim to assess the feasibility of this data collection method and examine preliminary outcomes of the open trial prior to the RCT.

The primary outcome of the study is Veteran-clinician collaboration in the usual care setting. We audio-recorded up to eight mental health appointments for the nine study participants across multiple time points. We used two validated coding systems to measure collaboration, which we adapted with the developer: Shared Decision-Making Coding Scale (SDM-CS) and the Consumer-Created Opportunities for Active Involvement Coding System (CCOAICS) (Salyers et al., 2012; Bonfils et al., 2017). The SDM-
CS measures elements of decision-making for each decision in an appointment. The CCOAICS looks at consumer-initiated behavior throughout the entire appointment and is not limited to decision-making. Two trained research staff members coded each appointment, and coders discussed discrepancies until consensus for each appointment was reached.

We recorded 28 appointments throughout the study (avg. three per participant, max six). Of these appointments, 11% were individual therapy and 89% were recovery coaching across eight providers at the clinic. Barriers to audio recording included appointment cancellations, limitations on the types of providers at the clinic, and variability in appointment frequency for Veterans. Preliminary effectiveness outcomes of CDST were positive and are presented as Cohen’s d effect sizes due to the small sample size. The percent of appointments where Veterans and clinicians collaboratively agreed on a decision increased from 33% at baseline to 80% at 3-month follow-up. Veteran-initiated collaboration during decision making increased by a large effect size from baseline to post (Cohen’s d = 1.03) and baseline to 3-month follow-up (Cohen’s d = 1.22). In non-decision making contexts, Veteran-initiated collaboration increased by a large effect size from baseline to post (Cohen’s d = 1.31) and a small effect size from baseline to 3-month follow-up (Cohen’s d = 0.34).

We used a novel coding strategy to measure Veteran-clinician collaboration in the CDST open trial. We found increases in collaboration following participation in the intervention. While audio recording was feasible, there were notable challenges; to optimize feasibility for the RCT, we will record mental health appointments outside of the clinic where the research is conducted and record throughout the study as opposed to at fixed time points. Despite the complexity of audio-recording and coding usual care appointments, we strongly recommend this innovative, ecologically valid approach.