529 - What is Entrustment? Development of an Objective Approach to Assessing an Entrustable Professional Activity

Poster Number: 529
Publication Number: 529.424

Rebecca Callahan, Baylor College of Medicine, San Antonio, TX, United States; Beth Byrne, Baylor College of Medicine, San Antonio, TX, United States; Sasha Wee, Baylor College of Medicine- Children's Hospital of San Antonio, San Antonio, TX, United States

Background: Entrustable Professional Activities (EPAs) are observable clinical activities developed from competencies and milestones from the American College of Graduate Medical Education (ACGME). EPAs provide a framework by which faculty can theoretically assess competency and recommend levels of supervision for a learner. While EPAs focus on competency-based medical education and provide big-picture goals, how these EPAs are judged, carried out, and monitored is variable and ill-defined. Limited data on implementation or assessment of EPAs, absence of current gold standards for evaluating EPAs, and lack of objective data and guidance for the assignment of entrustment levels for EPAs make it difficult for learners and evaluators to identify specific, actionable feedback to guide future improvement.

Objective: Develop a paired simulated phone call scenario and checklist to objectively assess learners on the American Board of Pediatrics (ABP) EPA 4: Manage patients with acute, common diagnoses in an ambulatory, emergency, or inpatient setting.

Design/Methods: A simulated phone call scenario involving patient triage, management, and escalation of care was created. An initial checklist of possible interventions performed during the simulated phone call was developed to align with the curricular components of ABP EPA 4. The initial checklist underwent an iterative review by physicians with expertise in medical education, pediatric hospital medicine, complex care, and pediatric emergency medicine. Content validity was supported by mapping EPA curricular components to the objective checklist items, incorporating feedback from content experts, and piloting the scenario with an attending and resident.

Results: We created a final objective checklist and paired simulated phone call scenario with content validity which can theoretically be used to provide a more objective evaluation of learners for ABP EPA 4.

Conclusion(s): A paired simulation and checklist can be created to more objectively evaluate entrustment levels for an EPA through mapping of curricular components, use of an iterative review process, and incorporation of feedback from pilot participants. Limitations include sparse literature regarding objective evaluations of learners’ level of entrustment for EPAs and the lack of validated methods or tools for evaluating EPAs. Next steps include implementation of the simulated phone call scenario and objective checklist with current learners to determine if further validity evidence can be provided for the use of our checklist and simulated scenario in assessing a pediatric resident’s level of entrustment for ABP EPA 4.