454 - Development and Validation of a Trauma-Informed Care Communication Skills Assessment Tool for Pediatric Residents

Friday, April 28, 2023

5:15 PM – 7:15 PM ET

Poster Number: 454
Publication Number: 454.122

Christine K. Thang, University of California, Los Angeles David Geffen School of Medicine, Los Angeles, CA, United States; Samantha Kucaj, UCLA Mattel Childrens Hospital, Los Angeles, CA, United States; James H. Lee, UCLA Mattel Childrens Hospital, Los Angeles, CA, United States; Cambria Garell, UCLA, Los Angeles, CA, United States; Aaron W. Calhoun, University of Louisville School of Medicine, Louisville, KY, United States; Ken Lay, University of California, Los Angeles David Geffen School of Medicine, Los Angeles, CA, United States; Moira Szilagyi, UCLA, Los Angeles, CA, United States

Presenting Author(s)

Christine K. Thang, MD, FAAP (she/her/hers)

Assistant Clinical Professor
University of California, Los Angeles David Geffen School of Medicine
University of California Los Angeles
Glendale, California, United States

Background:

Our pediatric residency program implemented trauma-informed care (TIC) training using the “Pediatric Approach to Trauma, Treatment, and Resilience” (PATTeR) course. Finding no published assessment tool validated to assess learner TIC communication skills, we developed a tool based on the previously validated Gap-Kalamazoo Communication Skills Assessment Form (GKCSAF).

Objective:

To develop a standardized scoring tool to measure the TIC communication skills of pediatric residents and to assess its validity for the evaluation of residents undergoing TIC training using Messick’s unified framework.

Design/Methods:

The GKCSAF-Trauma Informed Care (GKCSAF-TIC) was developed using an expert panel of educators, clinicians, and mental health experts from academic centers, community clinics, the American Academy of Pediatrics, and the National Child Traumatic Stress Network. Participants adapted the original GKCSAF content using TIC guidelines and scripted TIC-oriented standardized patient (SP) encounters. The TIC domains included relationship building, engagement, trauma inquiry, understanding perspectives, response to distress, collaboration, empowerment, and closure, scored along a 5-point Likert scale.
The tool was piloted with 15 pediatric residents. Each trainee participated in a 15-minute SP encounter (pre-training), received the PATTeR training, and then participated in a 15-minute SP follow-up encounter (post). Both encounters were video-taped and scored by 2 trained faculty reviewers using the GKCSAF-TIC. Rater training described the tool contents and scoring system in a one-hour meeting.
Cronbach’s alpha was used to measure internal consistency and intra-class correlation (two-way random effects, absolute value, average measure) for inter-rater reliability.

Results:

Content and response process were supported by the development and rater training process. Cronbach’s alpha (internal consistency) ranged from 0.94-0.95 across the 2 faculty raters, and intra-class correlations (inter-rater reliability) ranged between 0.82 (pre-data) and 0.67 (post-data).

Conclusion(s):

The GKCSAF-TIC demonstrates preliminary validity evidence as an assessment tool for TIC communication skills across Messick’s domains of content and response process, as evidenced by the development process. Initial internal structure is supported by relatively high internal consistency statistics and moderate to high inter-rater reliability, but further refinement is needed. We offer this tool to the educational community as a preliminary means of assessing the effectiveness of TIC training.