Skip to main content
ABCT 2023 Annual Convention Main banner
Final Program


Program Flipbook
Icon Legend
This session is not in your schedule.
This session is in your schedule. Click again to remove it.
Presentation Icons
Ticketed Session
Ticketed Session
Recorded Session
Recorded Session
Back
Evaluation ViewAttendees 23

Spotlight Research Presentations

Assessment

Spotlight Research 2 - A Paradigm Shift in Measurement Validation When Assessing Mental Health

Friday, November 17, 2023
3:30 PM – 5:00 PM PST
Location: Cispus (403), Level 4
Earn 1.5 Credit
Keywords: Assessment, Measurement, Psychometrics
Recommended Readings: De Los Reyes, A., Wang, M., Lerner, M.D., Makol, B.A., Fitzpatrick, O., & Weisz, J.R. (2023). The Operations Triad Model and youth mental health assessments: Catalyzing a paradigm shift in measurement validation. Journal of Clinical Child and Adolescent Psychology, 52(1), 19-54. https://doi.org/10.1080/15374416.2022.2111684,   Balling, C.E., Lane, S.P., & Samuel, D.B. (2022). Ratings of dimensional traits in clinical practice: Comparing therapist and client perspectives. Personality Disorders: Theory, Research, and Treatment, 13(3), 254-265. https://doi.org/10.1037/per0000509,   Nock, M.K., Millner, A.J., Ross, E.L., Kennedy, C.J., AlSuwaidi, M., Barak-Corren, Y., Castro, V.M., CastroRamirez, F., Lauricella, T., Murman, N., Petukhova, M., Bird, S.A., Reis, B., Smoller, J.W., & Kessler, R.C. (2022). Prediction of suicide attempts using clinician assessment, patient self-report, and electronic health records. JAMA Network Open, 5(1), e2144373. https://doi.org/10.1001/jamanetworkopen.2021.44373,   ,  
Mental health researchers approach measurement validation the way many other social scientists do. They construct instruments (e.g., surveys, interviews, behavioral tasks), administer them to participants, and test the properties of scores taken from administering these instruments using the classic Multitrait-Multimethod Matrix (MTMM) paradigm (Campbell & Fiske, 1959). The MTMM paradigm rests on the idea that two or more measurements of a particular domain (e.g., anxiety, ADHD, cognitions, social support) are valid if their data converge; MTMM emphasizes converging data produced by instruments assessing the same domain but using “plausibly independent methods” (see p. 393 of Fiske & Campbell, 1992). In the MTMM paradigm, discrepant results produced by such measures are deemed measurement confounds (e.g., random error, rater biases)―threats to measurement validity (see Watts et al., 2022). One can trace this assumption back to the seminal Converging Operations concept developed by Garner and colleagues (1956), and before that to the Theory of Errors that Edgeworth (1888) repurposed from Physics to devise schemes for measuring mental states. But what if this central assumption―that discrepant results constitute error, and not meaningful information about mental health domains―doesn’t “fit” the data that mental health researchers collect? In this Spotlight Research Presentation, we dispute that assumption. We describe a recently published validation paradigm that we designed to detect domain-relevant information in both converging results and discrepant results, including cases in which reports from youths, caregivers, and other informants disagree (De Los Reyes et al., 2023). Because the long history of discrepant results in youth mental health assessments precedes the development of the MTMM (i.e., Lapouse & Monk, 1958), we illustrate our paradigm using both historical and recent examples. Our illustrations traverse domains (e.g., autism, anxiety, disruptive behavior, parenting), study designs (experimental, observational, longitudinal), and clinical populations (e.g., acute care, outpatient services), underscoring the range of scenarios in which discrepant results produce meaningful, domain-relevant information.

Learning Objectives:

At the end of the session, the learner will be able to:
  • Discuss the typical patterns of converging and discrepant results in assessments of mental health.
  • List theories underlying why mental health assessments might produce discrepant results.
  • List innovative and novel approaches for integrating discrepant assessment outcomes.
  • Discuss a validation paradigm designed to detect meaningful mental health assessment data in both converging results and discrepant results.
  • Recite the empirical work supporting the validation paradigm described in this presentation.