By Haidy Nasief, PhD, MS, Medical College of Wisconsin
Incident learning platforms are powerful tools that can help us ensure efficient and safe delivery of treatment through lessons learned. However, querying data from different incidents into an efficient learning platform can be very challenging. Artificial intelligence (AI) and large language models have the potential to perform these tasks and present users with valuable information if designed properly.
Abbas Johar Jinia, et al. presented at the ASTRO Annual Meeting an AI powered re-irradiation Incident analysis and Learning System (AI-ILS) that analyzes radiotherapy incidents using the Human Factors Analysis and Classification System framework (HFACS) to identify opportunities to improve workflow, safety and efficiency.
Under an IRB-approved protocol, the authors reviewed all reported incidents from January 1, 2019 to September 30, 2024. Re-irradiation-related incidents were identified using a large language model in combination with keyword-based filtering. Incident severity and reporter roles were assessed. Incidents were analyzed using AI-ILS to generate aggregated HFACS classifications. They also identified the workflow stages at which incidents occurred and were detected.
Building this multilayer platform shown in the figure below, the investigators found that out of 9,919 reported incidents, 267 incidents (<3%) involved re-irradiation. Of these, 76% were near-misses and 24% reached the patient without resulting in harm. Incidents were most frequently reported by plan check physicists (58%), followed by planner physicists or dosimetrists (25%), radiation therapists (11%), and physicians (0.75%). HFACS analysis identified the most common contributing factors as Personnel Factors (79%; e.g., miscommunication), Organizational Processes (65%; e.g., inadequate rules and procedures), and Errors (57%; e.g., missing contours of prior targets). 123 incidents (46%) occurred during pre-planning documentation (e.g., placement of consultation note and simulation order). 43 incidents (16%) occurred during planner review of the pre-planning documentation. 69 incidents (26%) occurred during treatment planning (e.g., incorrect cumulative EQD2 estimation, misinterpretation of prior dose metrics). 10 incidents (4%) occurred during post-planning and pre-treatment review of the plan and corresponding documentation (e.g., special physics consultation document was not signed). Regarding incident detection, 65 incidents (24%) were identified during planner document review, 29 (11%) during planning, and 134 (50%) during physicist plan check.
The authors of the study showed that workflow inefficiencies and system vulnerabilities in documentation, cumulative dose assessment, and interdisciplinary communication can be detected. Such early detection of inefficiencies can minimize additional work at multiple stages and thereby avoid significant time loss. Targeted mitigation strategies, including automated re-irradiation status flags that are tailored to user roles, and enhanced software tools for cumulative Equivalent Dose (EQD2) analysis can also be implemented.
Employing AI and large language models can provide clinicians with insights that not only strengthen safety and standardize processes but also improve efficiency in a complex re-irradiation workflow early on.
Abstract 1126, Re-Irradiation Incident Analysis with AI: Identifying Opportunities for Workflow Optimization and Improvement, was presented during QP 21: Innovating with Software to Drive Patient Safety & Quality, at ASTRO's 68th Annual Meeting.
