4 - (SYM 150) a Longitudinal Examination of an Electronic Case Management System Implementation Determinants in a Publicly-funded Mental Health System

The State of Hawai'i Department of Health Child and Adolescent Mental Health Division (CAMHD) provides services and intensive mental health support to approximately 2300 youth and families with serious emotional and behavioral challenges every year through an integrated public-private partnership with community providers. In 2019, CAMHD implemented an electronic case management system (MAX) in Family Guidance Centers (FGCs) and provider agencies across the state as part of a larger effort to infuse data-driven decision-making in the CAMHD system. The present research is a longitudinal examination of implementation determinants such as system usability and practicality, and the differential impact based on FGCs and role. CAMHD staff attitudes, perceptions, and system usability were evaluated. 
The study utilized a mixed methods (QUANT-- >QUAL) design to evaluate MAX perspectives using multilevel models, and inductive coding techniques. The System Usability Scale, user-centered questions (e.g., “What if...”), and demographic information were assessed at three time points at pre-implementation (February 2019), implementation (April 2019), and post-implementation (February 2020).  Preliminary ANOVAs showed initial FGC variation in system usability at pre-implementation (F(8,115)= 2.067, p =.045) and implementation (F(8,90)= 3.442, p =.002). Multilevel model results indicated a significant effect for time (β = -13.308, p < .001), FGC unit (β = .769, p = .048), and time by role interaction (β = 1.410, p = .040) on the total System Usability Scale score. These findings converged with qualitative themes of MAX system feedback, MAX training, organizational characteristics, and barriers (i.e., attitudes/beliefs, technology, and time) at the individual (e.g., “being tech challenged”, “oppositionality”, “typing speed”) and organizational levels (e.g., “lack of tools”, “resistance to change”, “infrastructure”). Two independent raters also identified subthemes of access, attitudes/beliefs, content, delivery, efficiency, features, resources, reliability, technology, time, and trainers. These findings elucidate important multicontextual and longitudinal determinants of electronic case management system implementation. It is crucial to understand end-user perspectives as public mental health digitizes and disseminates health information to inform their rapid improvement.