486 - Usability and Utility of Virtual and In-Person Observation of Pediatric Residents' Patient Encounters

Poster Number: 486
Publication Number: 486.421

Aaron J. Samide, Johns Hopkins All Children's Hospital, St Petersburg, FL, United States; John M. Morrison, Johns Hopkins All Children's Hospital, Saint Petersburg, FL, United States; Taryn A. Hill, Johns Hopkins All Children's Hospital, Saint Petersburg, FL, United States; Matthew W.. Thomas, Johns Hopkins All Children's Hospital, St. Petersburg, FL, United States; Kimberly Collins, Johns Hopkins All Children's Hospital, Saint Petersburg, FL, United States

Background: Direct observation of trainees’ clinical skills and subsequent feedback delivery is fundamental to medical education, but time constraints, stress, and impacts on performance have been described as barriers to this practice. Virtual observation has been described for inpatient rounds and could be implemented as an alternative to in-person observation to potentially address some of these barriers, but little has been described regarding virtual observation for feedback purposes.

Objective: Compare virtual to in-person observation of pediatric residents in terms of feasibility, adequacy of observation, overall satisfaction, family comfort, and resident comfort.

Design/Methods:

We conducted a randomized crossover interventional study of first-year pediatric residents and faculty observers where residents were observed during 2 inpatient initial encounters: 1 observed in-person and 1 observed virtually via tablet. Resident-observer dyads were randomized with regards to first observation type. Resident, observer, and caregiver were surveyed after each encounter using 5-point Likert-type questions regarding impacts of the method of observation on the encounter, and resident and observer were surveyed again after completing both to determine overall preference.

Results: After 36 encounters, no significant differences between virtual and in-person observation were found in terms of observer time and effort, trainee comfort or perceived performance, or overall satisfaction (Table 1). Median rankings were higher for in-person observation for ability to adequately observe interviewing, exam, and counseling skills; organization/efficiency; and overall competence (Table 2, Fig. 1). Observers’ satisfaction with virtual observation was lower when ≥1 technical issue was reported compared to none (median 3 [IQR 2-4] vs. 5 [IQR 4.5-5] on 5-point scale; p< .01). For overall preference, using scale ranging from “strongly prefer in-person” (1) to “strongly prefer virtual” (7), median response was 2.5 (IQR 2-5) for observers and 2 (IQR 2-4) for residents, without significant difference between groups (p=.6) (Fig. 2). Caregivers indicated comfort regardless of observation method (Table 1).

Conclusion(s): Early results of our study suggest that observers, residents, and families report high satisfaction with both in-person and virtual observation. Though in-person was rated higher for adequacy of observation of key encounter components and overall preference, virtual may be an acceptable alternative when in-person is not feasible.