Skip to main content
ABCT 2023 Annual Convention Main banner
Final Program


Program Flipbook
Icon Legend
This session is not in your schedule.
This session is in your schedule. Click again to remove it.
Presentation Icons
Ticketed Session
Ticketed Session
Recorded Session
Recorded Session
Back
ViewAttendees 6

Symposia

Dissemination & Implementation Science

Session: The Joys of Cultivating CBT Competency and EBP Fidelity

5 - (SYM 41) Development and Initial Validation of a Single Item Observer Rated CBT Competency Score Measure

Friday, November 17, 2023
2:30 PM – 4:00 PM PST
Location: Sol Duc (708), Level 7



Background: No brief, validated exist to capture clinician cognitive behavioral therapy (CBT) competency. Many tools contain multiple items and have substantial rater burden. We developed and validated a single item competency score (SIC) for independent observers to rate on a 7-point Likert Scale (1= Very Poor to 7 = Excellent). Coders were trained to consider both how skillfully a clinician delivered CBT and how responsive and flexible the clinician was to the client’s clinical presentation.



Method: Coders first rated 288 audio-recorded therapy sessions using the SIC. These recordings were collected from 126 community mental health clinicians as part of a randomized trial comparing methods of CBT fidelity assessment; 49% of sessions were double coded. We calculated interrater reliability using intraclass correlation coefficients (ICCs) and examined SIC range and variability. For initial criterion validation, we examined how the SIC correlated with the Cognitive Therapy Rating Scale (CTRS; Young & Beck, 1980), a validated 11-item observer-coded scale that yields an overall composite score and 11 individual item scores, in a subsample of sessions. This subsample was selected via stratified random sampling to ensure sessions represented the full range of SIC scores, with one session per therapist (n = 42). CTRS codes were assigned by two independent coders trained to reliability and masked to SIC scores.



Results: The ICC for the SIC was .79, suggesting good interrater reliability. The SIC also showed good variability, with scores representing the full range (M = 3.09, SD = 1.45). Within the subsample of recordings coded with both the SIC and the CTRS, the SIC and CTRS composite score were moderately correlated (r = .57, p < 0.001). Nearly all CTRS individual item scores also significantly correlated with the SIC (rs range from .21 - .60), except for ‘homework’ and ‘interpersonal effectiveness’.


Conclusion: Results suggest that a single item competency score can reliably capture CBT delivery competence. Implications for future measurement of CBT competency will be discussed.