553 - Relationship between Comprehensive Osteopathic Medical Licensing Examination (COMLEX) Scores and Performance on the American Board of Pediatrics (ABP) Certifying Examination

Over the last 2 decades, osteopathic graduate matches have increased 4-fold, now representing about 20% of all pediatric interns as compared to only 5% in 2006.  There is currently no evidence relating COMLEX performance to ABP Certification as there is relating USMLE and ABP.

Objective:

To investigate the relationship between COMLEX scores and performance on the ABP certifying examination. 

Design/Methods:

A retrospective, cross-sectional study was conducted to investigate the relationship between performance on the COMLEX and ABP Certifying Examination. This study was completed with the aid of the Association of Pediatric Program Directors’ Longitudinal Educational Assessment Research Network (APPD LEARN). Program Directors were queried for participation via the APPD Listserv and eligible if the program contained osteopathic graduates who had taken the ABP exam during the study period. Data containing COMLEX and ABP scores was collected Mar-Dec 2021. Data represented osteopathic graduates taking the ABP examination between the years 2015-2020. ROC curves were constructed to evaluate how well each COMLEX exam most accurately predicted passing the ABP exam. Sensitivities and specificities of each prediction were also assessed.

Results: Six programs across the United States provided data on 64 osteopathic graduates. COMLEX-1 score > 424 had 98% sensitivity (SEN) and 67% specificity (SPEC) for passing ABP (PPV 96%, NPV 80%, AUC 0.82). Among residents with reported COMLEX-1 score, 5 were below the 424-score threshold (4 of 5 failed ABP) and 55 were above (2 of 55 failed ABP). COMLEX-2 score > 513 had 76% SEN and 84% SPEC for passing ABP (PPV 98%, NPV 28%, AUC 0.85). Among residents with COMLEX-2 score, 18 residents were below the 513-score threshold (5 of 18 failed ABP) and 53 were above (1 of 43 failed ABP). COMLEX-3 score > 431.5 was a perfect predictor among those who took COMLEX-3; three below the threshold all failed ABP and 41 above all passed ABP.

Conclusion(s):

The COMLEX-2 score best predicted ultimate ABP passage as a substantial number of residents below the score threshold ultimately failed the ABP exam while only 2% (n=1) scoring above the threshold also failed. COMLEX-1 score below the score threshold is also a sensitive red flag, but a COMLEX-1 score higher than the threshold is less informative. While the COMLEX-3 score was a perfect predictor, sample size was relatively small. These findings can aid residency program directors in identifying high- and low-risk candidates entering pediatric residency programs allowing targeted individualized interventions with the goal of ABP certification.