500 - Systematic program evaluation of a novel clinical coaching program using the CIPP model

Poster Number: 500
Publication Number: 500.422

Kristin Maletsky, Children's Hospital of Philadelphia, Philadelphia, PA, United States; Jay Mehta, Children's Hospital of Philadelphia, Philadelphia, PA, United States; Tashveen Kaur, Childrens Hospital of Philadelphia, Penn Valley, PA, United States; Marciel Gonzalez, Childrens Hospital of Philadelphia, Philadelphia, PA, United States; Anna K.. Weiss, Children's Hospital of Philadelphia, Philadelphia, PA, United States; Donald Boyer, Children's Hospital of Philadelphia, Philadelphia, PA, United States; Daniel C. West, Children's Hospital of Philadelphia/U Penn, Philadelphia, PA, United States; Dorene F. Balmer, Perelman School of Medicine at the University of Pennsylvania, Philadelphia, PA, United States

Background: Performance coaching has great potential to support learning across the continuum of medical education, yet knowledge of how to best design such programs to achieve this potential is limited. To date, most coaching programs have focused on resident learning. We sought to create a coaching program to enhance the clinical performance of both trainees and faculty. Here we report our early experience, primarily with residents, but also with fellow and faculty coachees.

Objective: To evaluate the early phase of a novel performance-based coaching program (see Figure 1) using the Context, Input, Process, and Product (CIPP) Evaluation Model. 

Design/Methods: Our program focuses on growth mindset-oriented, non-evaluative coaching partnerships designed to execute iterative learning cycles that consist of value diving, goal setting, reflections, learning activities, and debriefing. We used the CIPP Model to evaluate the program, examining Context (needs assessment, problems, and opportunities), Input (identification of potential approaches), Process (periodic assessment of successes and opportunities for improvement), and Product (intended and unintended outcomes) See Table 1 for data collection tools and exemplar findings.

Results: CONTEXT: Stakeholders felt coaching should occur in a longitudinal, trusting, non-evaluative relationship characterized by psychological safety. They emphasized the need for clarity around a coaching definition and expectations, and adequate time. A learning environment survey identified potential challenges in the context. INPUT: Regular coaching team meetings informed program development, including coaching skills workshops. Retrospective pre-post questionnaires completed by workshop attendees indicated grasp of performance coaching skills. PROCESS: Monthly coach meetings facilitated mid-course corrections. Observation of workshops demonstrated alignment with learning objectives and identified areas for improvement (e.g., more practice). PRODUCT: The p</span>rogram has enrolled 21 coaches representing 9 divisions/departments and 25 coachees (13 residents, 2 fellows and 10 faculty members). Based on p</span>reliminary data, coachees express positive regard for feedback, reflection, and safety within coaching relationships.

Conclusion(s): Our evaluation data suggest the program is on target to achieve its goal of supporting the clinical performance of trainee and faculty coachees. Our use of the CIPP Model facilitated systematic evaluation of this new coaching program at our institution, providing valuable information for future expansion of this program and a model of evaluation for other programs.