(PS5-D94) Are We Speaking the Same Language? Outcomes of a 20-year Journey to Develop a Shared Practice Language Across Children’s Public Mental Health Services Statewide

The development of shared ontologies in the behavioral sciences is critical to bridging research and practice (National Academy of Sciences, Engineering, & Medicine, 2022). Hawaii has been building a shared ontological framework for mental health treatment spanning both its children’s public mental health and education systems. This use of a common language around treatment has facilitated the promotion of evidence-based practices and investigations on local, practice-based evidence (e.g., Daleiden & Chorpita, 2005).

This poster reports on the 20-year evolution of the state’s practice element (PE; a discrete intervention strategy) framework within its mental health system. Originally distilled from the research literature by the state’s Evidence-Based Service Committee, a list of 55 PEs was initially used to record treatment encounters starting in 2003 (Chorpita et al., 2005; Orimoto et al., 2012). Two major revisions of the framework have occurred with changes based on developments in the research literature and the frequency of locally reported practices. The most recent revision occurred in tandem with the institution of a new Electronic Health Record system along with substantial changes to the “Progress Note” record in which PEs are reported by therapists, for every clinical encounter with a youth (Child and Adolescent Mental Health Division (CAMHD), Hawai‛i Department of Health, 2019).

By analyzing entries where the therapist selected an “Other,” or unlisted PE, this study aimed to assess how well the current PE list captures treatment providers’ practices, and potential gaps in the ontology system. Support staff assisted with a pull of PE data from the latest version of the Progress Note system (February 2020 to December 2022). Preliminary examination of PE data (n=251,745) found that 0.8% (n=2,023) of PEs were reported as “Other” across all clinical encounters. Of this list of “Other” PEs a majority had a description authored by providing therapists (n=1,628). Entries included a wide variety of descriptions of practices and encounters with youth and their families. One limitation was that the data did not include info on the number of unique authors.  

With the consultation of an expert in research and clinical practice, the senior investigator and the research team developed a deductive coding scheme to content analyze the written entries listed as “Other” in the progress note. The research team then concurrently reviewed the coding scheme and data in four rounds of revisions to develop the final coding scheme. The team then independently coded the whole of the dataset using this final version. An initial reliability analysis found high percent agreement between coders with agreement ranging from 72.6 to 81.6% between all pairings of coders.

 
Of the “Other PEs” with written descriptions, 37.0% could have been reported using existing PEs, 2.6% described potential unlisted PEs, and 58.6% described something other than PEs such as unclear information and references to general treatment packages. Overall, findings suggest that the list of PEs provides a fairly comprehensive nomenclature and demonstrates success of the state’s ontological framework, and also highlights areas for further improvement in PE reporting and data collection.