486 - Preparing Emergency Medical Services Radio Call Recordings for Natural Language Processing

Sunday, April 30, 2023

3:30 PM – 6:00 PM ET

Poster Number: 486
Publication Number: 486.31

Shawn A.. Haupt, Georgetown University School of Medicine, Washington, DC, United States; Desiree Edemba, Children's National Health System, Arlington, VA, United States; Nakul Gopalan, Arizona State University, Chandler, AZ, United States; Alyssa Abo, Children’s National Hospital, Washington, DC, United States; Emily J. Dorosz, Children's National Hospital, Washington, DC, United States; Matthew Gombolay, Georgia Institute of Technology, Atlanta, GA, United States; Kenneth McKinley, George Washington University School of Medicine and Health Sciences, Washington, DC, United States

Presenting Author(s)

Shawn A. Haupt, MD, MS (he/him/his)

PGY-3 Pediatrics Resident
MedStar Georgetown University Hospital
Washington, District of Columbia, United States

Co-Author(s)

Kenneth McKinley, MD (he/him/his)

Assistant Professor of Pediatrics and of Emergency Medicine
George Washington University School of Medicine and Health Sciences
Washington, District of Columbia, United States

Background: Emergency Medical Services (EMS) clinicians regularly consult with hospitals for online medical control regarding destination decisions, using unencrypted radio channels. These calls are meant to ensure an appropriate destination based on the patient’s clinical presentation and other factors, to notify Emergency Department (ED) staff of the incoming patient, and to allow for necessary preparations to take place prior to the patient’s arrival. Natural language processing (NLP) may offer opportunities to predict the clinical needs of pediatric EMS patients prior to arrival in a destination ED.

Objective: To determine the acceptability of EMS recordings for future NLP analyses by (a) screening for the presence of protected health information (PHI) and (b) assessing the audio quality of recordings.

Design/Methods: Two independent reviewers (SH and DE) listened to audio recordings between EMS and a pediatric destination hospital, from 1/2018-12/2021. Reviewers screened all calls for PHI, including names and dates of birth, which should not be discussed over radio channels. Reviewers documented the duration and number of patients discussed in each call, patient age, sex, and chief complaint, and assessed the audio quality of the recording, choosing one of four levels between 0 (“nothing intelligible”) and 3 (“high quality”). Only one reviewer listened to the majority of recordings; to assess inter-rater reliability, reviewers independently listened to 100 of the same recordings and we calculated the Cohen kappa score for patient age, sex, and audio quality.

Results: None of the recordings reviewed contained PHI. 156 recordings (28.3%) contained only radio checks or other noise, 387 (70.2%) contained information about one patient, and 8 (1.5%) contained information about multiple patients. Calls containing no useful patient information were associated with shorter duration (Figure 1). Cohen kappa scores were 0.98 for age, 1.00 for sex, and 0.74 for reviewer assessment of audio quality. 91.9% (363/395) of calls containing patient information were assessed as one of the highest 2 quality levels.

Conclusion(s): EMS clinicians for this single children’s hospital destination avoided the use of names and other PHI during online medical control via radio. There was excellent agreement between independent reviewers regarding objective data extracted from audio recordings, and recording quality was assessed favorably. These findings support the acceptability of applying NLP analyses to EMS audio recordings, but more work is necessary to determine if the findings are generalizable to other hospitals and EMS systems.