May 26, 2023 from Medscape
A new study demonstrates that an artificial intelligence (AI) technology outperforms human call takers in identifying stroke victims from calls to the emergency services.
The AI model had a greater positive predictive value—a measure of the percentage of anticipated positive cases that are actually positive—than the human phone operators and accurately identified more patients who were actually having a stroke.
The model performed better on both counts. According to study co-author Jonathan Wenstrup, MD, of Copenhagen University Hospital-Herlev and Gentofte & Copenhagen Emergency Medical Services, Denmark, “It flagged fewer patients in total with a suspected stroke but correctly identified more patients who were actually having a stroke than the human dispatcher” as reported to theheart.org | Medscape Cardiology.
At the European Stroke Organization Conference (ESOC) 2023 in Munich, Germany, on May 24, Wenstrup presented the study.
The emergency call registry, which has recordings of all phone calls to the medical helpline at Copenhagen emergency services, and the Danish stroke registry, which contains information on every patient admitted to hospital with a stroke, including the time of onset of symptoms, were linked for the study. The calls were divided among those made by people who later proved to be experiencing a stroke and those who were later found not to be.
The AI model was trained to transcribe the recordings of emergency calls and search for variations between stroke- and non-stroke-related calls.
The algorithm was trained using information from 1.5 million emergency service contacts between 2015 and 2020, 7370 of which were confirmed to be stroke victims. Then it was evaluated with 2021 data from 344,000 calls, 750 of which were stroke cases.
The AI model performed better than human emergency call dispatchers, who correctly identified only 52.7% of stroke cases. The results showed that the AI model accurately identified 63% of patients who were having a stroke.
Additionally, the model exhibited a higher positive predictive value (24.9%) compared to the human dispatcher’s (17.1%). When the two metrics are added together, the AI model has a total F1 score (a global indicator of a test’s correctness) of 35.7 compared to the human call handlers’ 25.8.
“The AI model recognized stroke better and has a lower rate of false positives than the actual emergency services dispatchers,” Wenstrup said. He clarified that it was challenging to diagnose a stroke from calls to the emergency services. Many cases go undiagnosed at this point, which causes treatment delays that could have fatal outcomes for patients.
This study’s use of a retrospective design is a drawback. A live test of the model has not yet been conducted. Wenstrup stated that research was necessary to determine how well it worked in practice and that he thinks the AI tool might be used to help emergency telephone operators identify stroke patients.
Wenstrup said that this model might be running in the background while they talk to a variety of different callers, alerting them to a patient who has a high risk of suffering a stroke and should be given priority for urgent care. “If the model performs similarly well in a real-life setting,” he continued, “then it could improve stroke recognition by the emergency call handlers, enabling more stroke patients to get the rapid advanced treatment that improves outcomes.”
Wenstrup remarked that the model’s capabilities might be increased with additional advancements. He stated that “In future, it may be possible to train the framework directly from the call audio, bypassing the transcription step, as well as incorporating nonword audio — such as a slurred voice — into the training data. However, given the promising results of this study, it is already clear that technologies like this have the capability to completely transform stroke diagnosis and care”.
=================================================================
Considering an IME or document review to resolve an insurance claim, legal file, or workplace health and safety issue?
Our specialists provide evidence-based opinions, so get in touch with Western Medical today to learn more about our services.