The Use of AI in Predicting Patient Outcomes and Deterioration in the Emergency Department
Amal Akeel, Ayed Aljohani, Osama Alnasser, Abdulmalik Alwabel, Ibrahim Alsaif, Atyaf Thabet, Abdulmalik Bakhsh, Yasir Albuhayri, Ahmed Alsalman, Razan Aljohani, Talal Abu Suliman
Keywords: Artificial intelligence, Emergency department, Patient outcomes, Clinical deterioration, Prediction models
The emergency department is an ever-changing environment where patients arrive with a range of conditions and different levels of urgency. One of the struggles for ED healthcare professionals is to assess and manage the risk of clinical deterioration, as it can significantly impact patient outcomes, resource allocation, and decision-making processes. Unfortunately, existing approaches to identifying and responding to patients’ worsening conditions often rely on judgments, lack consistency, or experience delays. Hence, there is a need for objective, dependable, and timely tools to assist ED triage and patient care. Artificial intelligence (AI) is a modern field within computer science that aims to develop systems of performing tasks that typically require intelligence, like learning from data, reasoning logically, and making informed decisions. AI has gained usage in healthcare, encompassing aspects such as diagnosing illnesses, predicting outcomes, providing treatments, and monitoring patients. One area where AI has shown promise is in predicting outcomes and identifying signs of deterioration in the emergency department. This review provides an insight into the advancements in this field, discussing the advantages and challenges associated with utilizing AI to predict patient outcomes and detect deterioration in the ED. It discusses how AI can enhance ED triage and care's accuracy, efficiency, and interpretability by providing objective, timely predictions and explanations. It also tackles the concerns surrounding the quality and accessibility of data, potential biases in algorithms, ethical and legal considerations, human-computer interaction, and the integration of AI into clinical practices. The review concludes by highlighting AI's future directions and implications for patient safety, quality of care, and resource utilization in the ED.
The emergency department is a setting where individuals arrive with a variety of conditions and differing degrees of seriousness. A significant challenge for ED healthcare professionals is to assess and manage the risk of deterioration, as it can have an impact on outcomes, resource allocation, and decision-making (1-3). However, current methods used to identify and respond to deteriorating patients often lack objectivity, consistency, or timeliness. This highlights the need for objective and timely tools that can assist in triage and care within the ED setting. Artificial intelligence is a branch of computer science that focuses on creating systems for performing tasks that necessitate intelligence. These tasks encompass learning, reasoning, and decision-making processes. AI has increasingly been utilized across domains, including diagnosis, prognosis assessment, treatment planning, and patient monitoring. Within the realm of emergency medicine, AI has demonstrated potential in predicting patient outcomes and identifying deterioration by leveraging diverse data sources such as clinical variables, vital signs measurements, laboratory test results, and imaging techniques, particularly chest X-ray imaging, which serves as an important initial tool, for triaging patients presenting with respiratory illnesses like COVID-19 pneumonia or pulmonary edema (4-6). However, it can be quite difficult and time-consuming for clinicians to interpret chest X-ray images, making them susceptible to errors. To address these challenges, AI can employ networks (DNNs). DNNs are algorithms utilized in machine learning that have the capability to analyze patterns within datasets. Specifically, in the context of chest X-ray images, DNNs have the capability to extract information automatically (7). This information can then be used to predict the likelihood of deterioration or adverse events for patients in emergency departments (ED). For instance, a recent study presented an AI system that utilized a DNN trained on chest X-ray images along with a gradient-boosting model that learned from variables to predict the deterioration of COVID-19 patients in the ED (8-10). The system achieved a performance with an area under the operating characteristic curve (AUC) of 0.786 when predicting deterioration within 96 hours. Additionally, the DNN identified areas in chest X-ray images and provided insights for clinicians while performing comparably to two radiologists in a reader study. During the wave of the pandemic, this system was successfully deployed at New York University Langone Health. Provided real-time accurate predictions. Another research project focused on developing an AI system for predicting mortality risk among ED patients experiencing acute dyspnea. Similar to before, they employed a DNN trained on chest X-ray images and clinical variables. The achieved performance was remarkable, with an AUC of 0.88 when predicting mortality within 30 days. Moreover, this DNN also highlighted regions of interest in chest X-ray images linked to increased mortality risk while explaining its predictions. It's worth mentioning that apart from chest X-ray imaging, there are data sources that can be utilized for predicting outcomes and identifying ED deterioration. For example, in the emergency department, it is a practice to measure and document vital signs of patients. These measurements can offer insights into their condition. However, it is worth noting that vital signs in the emergency department are frequently subject to noise, incompleteness, or even absence. AI can tackle these challenges by utilizing machine learning techniques, such as imputation, normalization, aggregation, or feature engineering, to preprocess and analyze data on signs. A particular research study examined 22 scoring systems that aggregate vital signs data to forecast deterioration in patients visiting the emergency department (ED). The study discovered that the National Early Warning Score (NEWS) exhibited accuracy among these systems. It achieved an AUC of 0.81 in predicting ICU admission within 48 hours. The NEWS assigns points to six signs (heart rate, oxygen saturation, temperature, systolic blood pressure, heart rate, and level of consciousness) and calculates a cumulative score that indicates the risk of deterioration. Another study focused on developing an AI system for forecasting arrest or ICU transfer in ED patients. This system utilized a machine learning model trained on both vital signs data and electronic health records (EHRs). It achieved an AUC of 0.84 in predicting arrest or ICU transfer within 24 hours. Moreover, this system provided risk scores and alerts to assist ED clinicians in taking interventions. The emergence of AI has brought about promising advancements in predicting outcomes and identifying deterioration in the ED using sources of data (11). By offering timely predictions as well as explanations, AI can greatly enhance the accuracy, efficiency, and interpretability of triage and care processes in the ED. Nevertheless, there are obstacles and constraints linked to the utilization of AI in the emergency department (ED) environment. These include factors like the quality and accessibility of data, ethical and legal concerns, and interaction between humans and computers as integration into clinical practices. Hence, it is important for future studies to tackle these challenges and assess how AI affects safety, the quality of care, and the efficient use of resources in emergency departments. Therefore, this research aims to examine how AI is utilized in predicting outcomes and identifying deterioration in the emergency department.
This review study is based on already existing literature, and it was conducted on October 22, 2023, The databases used for searching the literature are from Medline and PubMed databases. The search involved using medical subject headings and a combination of keywords according to the database specifications. The search terms included “artificial intelligence,” “emergency department,” “patient outcomes,” “clinical deterioration,” and “prediction models.” The search was done within the studies involving humans and published in English. Studies older than 2008 were excluded.
Integrating intelligence (AI) into emergency department (ED) clinical management brings possibilities but poses significant challenges. In the realm of AI research, it is crucial to evaluate bias to ensure the dependability and excellence of models. Bias can emerge from sources such as data quality, algorithmic biases, as well as ethical and legal considerations. The quality and availability of data take precedence since AI relies on extensive and pertinent information. Inadequate or erroneous data can result in misclassifications and other related problems.
Bias Assessment and Complications
Assessing the bias is a fundamental step in the field of artificial intelligence (AI) research and the creation of predictive models. Its main purpose is to evaluate the reliability and quality of these models by identifying any sources of bias that could impact the study's outcomes and applicability. In AI research, there are factors that contribute to bias, which require careful scrutiny (12, 13). AI data must meet standards of being accurate, comprehensive, representative, and relevant to the population and context being studied. Poor data quality or lack thereof can lead to issues like overfitting, under fitting, or misclassifying both outcomes and predictors (14). Dealing with incomplete or missing data is often common in healthcare settings, like emergency departments (ED). Inaccurate vital signs data can negatively impact the performance of AI models used for predicting deterioration. Therefore, assessing data quality and availability is crucial while transparently reporting them in AI studies and models (15, 16). Another important aspect that requires attention in AI research is bias. Developing and implementing AI models require algorithms prioritizing transparency, robustness, and fairness. Algorithmic bias occurs when these algorithms produce results that are either inaccurate, unfair, or discriminatory towards groups or individuals due to their inherent traits, like age, gender, race, or ethnicity. Consider the scenario where AI models are utilized to forecast outcomes in emergency departments (ED) based on chest X-ray images (17). It is important to acknowledge that these images quality, resolution, and angles can differ significantly depending on the hospital or type of scanner used, potentially leading to outcomes. Consequently, it becomes imperative to focus on detecting and mitigating bias during the development of AI studies and models. Moreover, legal considerations play a vital role when implementing AI models for predicting outcomes and deterioration in the ED (18). The utilization of AI models must respect both patient's and clinician’s rights, values, and preferences. Several ethical and legal dilemmas may arise in this context, particularly concerning data privacy, informed consent, accountability, and liability issues as decision-making authority. Electronic health records (EHRs), which are frequently employed in AI models for predicting outcomes in the ED setting, pose risks regarding data privacy and security. EHRs contain personal information; thus, unauthorized access or improper sharing without adequate authorization or protection raises ethical concerns along with legal implications. Therefore, it is crucial to give consideration to legal aspects while also developing protocols that address these concerns responsibly while complying with relevant laws. In addition to the concern of bias, it's crucial to take into account the challenges that may arise when utilizing AI models to forecast outcomes in the emergency department. These intricacies hold the power to influence both care and the overall efficiency of operations greatly. One major obstacle is the occurrence of predictions by AI models resulting in positives and false negatives. False positives happen when AI models wrongly predict a risk of deterioration or adverse events for individuals who do not actually experience such outcomes. Conversely, false negatives occur when AI models inaccurately predict a risk of deterioration for individuals who do experience adverse events. So, both false positives as well as false negatives can have consequences for care. False positives may lead to tests, treatments, referrals, hospital admissions, delays, errors, harm to patients, increased costs, and missed opportunities for improving care. On the other hand, false negatives can result in delayed or missed interventions that can adversely affect outcomes. It is crucial to minimize the occurrence of both positives and false negatives while actively monitoring their frequency when utilizing AI models. Another important consideration is human-computer interaction – how effectively individuals interact with AI systems and computers. This encompasses factors including usability, acceptability, trustworthiness, feedback mechanisms, communication channels, collaboration methods, and education. Poor human-computer interaction can give rise to issues such as user frustration, confusion, dissatisfaction resistance, and misuse. In order to make sure that AI models are effectively integrated, it is crucial to enhance the interaction between humans and computers. Additionally, the smooth integration of AI models into the workflow, known as integration, plays a vital role in their successful implementation (19). Clinical integration covers aspects such as feasibility, scalability, adaptability, compatibility, and sustainability. When implementing AI models in healthcare settings, challenges include issues, operational complexities, organizational barriers, and resistance to change. Successfully integrating AI into practices is crucial for its adoption. While using AI to predict outcomes and emergency department deterioration brings benefits, it also presents challenges. Additionally, considering complications, like positives and false negatives, and improving human-computer interaction can facilitate seamless clinical integration. By addressing these aspects within the healthcare community framework, we can harness the potential of AI to enhance care and outcomes while managing risks effectively.
The incorporation of intelligence (AI) into emergency department (ED) management is a development in healthcare. The paced nature of EDs, where critical decisions are made for patients with high acuity cases, involves crucial functions like patient assessment, diagnosis, treatment, monitoring, and disposition (20). AI has the potential to revolutionize each of these aspects and greatly improve management. Patient assessment is the foundation of ED care, and AI can play a role in optimizing this crucial step. By utilizing AI-powered algorithms that swiftly analyze signs, medical history, and symptoms, patients can be triaged efficiently based on the severity of their conditions. Automating this process using AI enables the identification and prioritization of life-threatening cases to ensure attention for patients. Alongside this triage system, AI tools utilize natural language processing (NLP) to aid in assessment by extracting necessary information from electronic health records (EHRs) and clinical documents (21, 22). This valuable data includes details such as allergies and medications taken by the patient diagnosed with conditions. It equips clinicians with an understanding that's necessary for making quick and accurate decisions. The diagnostic process and decision support also benefit significantly from the effectiveness demonstrated by AI. Advanced AI algorithms have the capability to accurately analyze images, like X-rays, CT, and MRI. This leads to the accurate identification of conditions such as fractures, pulmonary embolisms, or intracranial hemorrhages. These findings greatly assist doctors in making precise diagnoses. When faced with unclear challenges, AI can be a valuable tool in supporting healthcare professionals. By analyzing data, AI can offer diagnoses, expanding the range of possibilities for clinicians and helping them determine the most suitable diagnostic tests or consultations with specialists. Additionally, AI algorithms play a role in providing treatment recommendations. They consider factors such as a patient's history of existing health conditions and current clinical parameters to create tailored treatment plans. For example, when dealing with suspected infections, AI can suggest an antibiotic regimen while considering local resistance patterns to minimize the risk of medication-related complications. In terms of monitoring for those in critical condition, AI-powered systems enable real-time analysis that quickly alerts clinicians to subtle changes in patient's conditions. Through algorithms, deviations in signs or signs of worsening respiratory function can be promptly identified and addressed through early intervention. Moreover, AI's capabilities extend to analytics for deterioration. Machine learning models analyze patient data to identify risk factors associated with outcomes. This predictive approach empowers clinicians to allocate resources effectively and intensify monitoring for high-risk patients, ultimately preventing events and optimizing patient outcomes. The management of patients in the emergency department mainly focuses on determining what should be done with them—whether they should be admitted, discharged, or transferred to facilities. Artificial intelligence (AI) can play an important role in this decision-making process by offering evidence-based suggestions. For example, AI algorithms have the ability to predict the chances of readmission or complications, which can provide healthcare professionals with insights to help them make informed decisions regarding patient care. Moreover, AI can play a role in optimizing the allocation of resources by predicting admissions and ED occupancy. By offering insights into the expected flow of patients, AI enables the management of staff and resources, ensuring that the ED can efficiently handle the influx of patients and improve throughput. The integration of AI into management brings notable benefits. Firstly, it improves the accuracy and speed of processes. AI's quick analysis of data leads to more precise diagnoses and treatment recommendations, which is particularly important in time-sensitive situations. Secondly, AI streamlines resource allocation, reducing waiting times, increasing throughput, and enhancing patient satisfaction. Thirdly, AI-powered predictive analytics enable intervention in cases where patient's conditions are deteriorating, thereby saving lives and mitigating the severity of complications. Lastly, decision support systems based on AI ensure consistency in practice by reducing variability and maintaining standards of care. However, integrating AI into management also poses challenges. Ensuring the quality and privacy of data is crucial as AI relies heavily on high-quality information. Protecting patient privacy and complying with data regulations are considerations. Algorithmic bias presents a challenge as AI models can show bias if they are trained on datasets that don't properly capture the range of diversity among patients. It is crucial to make efforts in order to reduce bias and promote fairness in AI algorithms. It is crucial to have oversight in conjunction with AI, as AI functions as a tool that supports clinical decision-making rather than replacing it. The responsibility for patient care still lies with clinicians. Additionally, incorporating AI seamlessly into existing emergency department (ED) systems and workflows might pose challenges requiring process changes and staff training to ensure implementation. However, the integration of AI into the management of EDs holds the potential for transforming patient care through improvements in patient assessment, diagnosis, treatment recommendations, monitoring, and overall disposition. While addressing challenges like data quality, algorithmic bias, and human oversight is necessary, the benefits of integrating AI in the ED are substantial. As AI technology continues to advance, its successful integration into management will enhance patient outcomes, streamlining healthcare delivery processes and maintaining consistent, high-quality care in emergency settings.
AI has the potential to bring about a transformation in care within the emergency department (ED). It can enhance healthcare quality and efficiency by improving assessment, diagnosis, treatment recommendations, monitoring, and disposition. Although there are challenges like ensuring data quality, addressing algorithm bias, and maintaining oversight, the advantages of AI in the ED are substantial. As technology progresses, it becomes increasingly important to integrate AI into management to improve patient outcomes and overall effectiveness.
Conflict of interest
There is no conflict of interest.
Data that support the findings of this study are embedded within the manuscript.
All authors contributed to conceptualizing, data drafting, collection and final writing of the manuscript.