459 - In or Out of the Room? Describing Alignment Between Senior Pediatric Residents and Preceptors When Making Ad-Hoc Procedural Entrustment Decisions

In the Pediatric Emergency Department, newly formed preceptor / trainee dyads make “ad-hoc” procedural entrustment decisions balancing learning with the outcome and safety of the patient.

Objective: The aims of this work were to describe alignment between senior pediatric residents and their preceptors on ad-hoc procedural entrustment before and after residents completed a simulated formative assessment (FA).  

Design/Methods: The FA consisted of obtaining informed consent from trained actors and performing simulated procedures: toddler forehead laceration (LAC) and infant lumbar puncture (LP). Residents recorded a self-assessment of procedural entrustment before and after the FA on an 8-point entrustment scale (Table 1). A representative Pediatric Emergency Medicine Panel (PEM Panel) reviewed recordings and scored resident performance. Residents and PEM Panelists also assigned entrustment scores to a hypothetical case (HC) for a future patient requiring a core pediatric procedure. T-tests compared the mean PEM Panel’s entrustment score to the senior residents’ scores. Composite two-by-two tables generated kappa statistics and the percentage of dyads in agreement with the dichotomized procedural entrustment decision of “in versus out” of the room. 

Results: Twenty-four representative senior pediatric residents completed the FA and 9 PEM Panelists reviewed their performance. PEM Panel Inter-rater reliability on the 8-point entrustment scale and the dichotomized outcome for LAC was 0.51, 0.70; LP 0.49, 0.77; and HC 0.55, 0.76. Before the FA, entrustment alignment between residents and the PEM Panel on the 8-point scale were as follows: (LAC – 4.08 vs. 4.97, p< 0.001; LP – 4.75 vs. 5.31, p=0.15; HC – 3.96 vs. 5.01, p=0.001). After the FA, entrustment alignments were: (LAC – 5.21 vs. 4.97, p=0.32; LP – 5.54 vs. 5.31, p=0.52; HC – 4.96 vs. 5.01, p=0.85) -Table 2. Before the FA, the dichotomized analysis revealed the following kappas (k) and percentage of dyads in agreement with “in versus out” of the room procedural entrustment: (LAC – k=0.03, 58%; LP – k=(-0.03), 58%; HC – k=(-0.07), 50%). After the FA: (LAC – k=0.46, 92%; LP – k=0.24, 83%; HC – k=0.24, 83%) -Table 3.

Conclusion(s): Through multiple analyses, our findings raise concern that prior to an ad-hoc procedural entrustment decision, senior pediatric residents do not align with their preceptors, generally desiring less entrustment. However, after participating in the FA, alignment improved. The next steps in this work include multi-center enrollment and study of which elements of the FA facilitate ad-hoc procedural entrustment alignment.