478 - To DO or Not to DO: Exploring Medical Student Experiences with Direct Observation (DO) During the Pediatric Clerkship

Friday, April 28, 2023

5:15 PM – 7:15 PM ET

Poster Number: 478
Publication Number: 478.123

Capri Alex, University of Chicago Division of the Biological Sciences The Pritzker School of Medicine, Chicago, IL, United States; H. Barrett Fromme, University of Chicago Division of the Biological Sciences The Pritzker School of Medicine, Chicago, IL, United States; Michael Ryan, University of Virginia, Charlottesville, VA, United States; Larrie Greenberg, Children's National Medical Center, Washington, DC, Potomac, MD, United States; Sarah Gustafson, The Lundquist Institute at Harbor-UCLA, David Geffen School of Medicine at UCLA, Charles R Drew University of Medicine and Science, Torrance, CA, United States; Maya Neeley, Vanderbilt Children’s Hospital, Nashville, TN, United States; Shaughn Nunez, Texas Tech University Health Sciences Center School of Medicine, Lubbock, TX, United States; Molly Rideout, The University of Vermont Children's Hospital, Burlington, VT, United States; Jessica VanNostrand, The University of Vermont Children's Hospital, Burlington, VT, United States; Nicola Orlov, University of Chicago Division of the Biological Sciences The Pritzker School of Medicine, Chicago, IL, United States

Presenting Author(s)

Capri Alex (she/her/hers)

Medical Student
University of Chicago Division of the Biological Sciences The Pritzker School of Medicine
Chicago, Illinois, United States

Background: Direct observation (DO) is a critical tool to support competency-based medical education. DOs assess learners in their performance of vital competencies such as interpersonal and communication skills - valuable markers of trainee development. Despite the LCME requirement that medical students obtain at least one DO on each clerkship, the medical student experience of DO assessments is unreported and unclear.

Objective: To 1) describe the medical student experience of factors that encouraged or inhibited high-quality DO assessments and 2) assess the frequency, type, and quality of direct observations and feedback in the pediatrics clerkship.

Design/Methods:

We performed a mixed methods convergent study across 6 diverse U.S. medical schools in 2022. We invited medical students to a semi-structured focus group at the end of their pediatrics clerkship. The qualitative arm used a constructivist grounded theory and constant comparison process; the quantitative arm utilized an internally developed 34-item survey.

Results: 73 medical students participated across 20 focus groups. Seven themes emerged with thematic sufficiency. Participants emphasized the value of creating a safe learning environment with intentional expectations, formative assessments over summative, and a desire for more dedicated DO opportunities with actionable longitudinal feedback.

97% (71/73) of participants completed the survey. Resident observation occurred the most (n=62, 89%); few students ranked attendings as their primary observer (n=6, 9%). For 31% (n=21) of students, attending feedback rarely occurred. Over half reported that an attending observed their skills outside the patient encounter “daily”, and while over half reported that an attending observed a history-physical at least once, a third said it never occurred at all (n = 23, 32%).

Conclusion(s):

The qualitative arm revealed medical students often experience unplanned, informal DOs that lack the actionable feedback they desire. The quantitative arm elucidated a significant majority of observations occurred by residents, who are not trained in assessment or feedback. Attending observation of patient encounters are limited, and even less are followed with subsequent feedback. These findings validate the concern that LCME DO requirements are not always being met, but more importantly that students are not receiving competency-based feedback. We contextualized our findings to recommend that residents and attendings receive training in competency-based assessment and feedback to substantially improve the medical student experience, competency training, and future patient care.