Email Alert | RSS

Journal of Peking University (Health Sciences) ›› 2021, Vol. 53 ›› Issue (6): 1163-1170. doi: 10.19723/j.issn.1671-167X.2021.06.026

Previous Articles     Next Articles

Predicting prolonged length of intensive care unit stay via machine learning

WU Jing-yi1,2,LIN Yu1,LIN Ke1,HU Yong-hua1,3,KONG Gui-lan2,4,△()   

  1. 1. Department of Epidemiology and Biostatistics, Peking University School of Public Health, Beijing 100191, China
    2. Advanced Institute of Information Technology, Peking University, Hangzhou 311200, China
    3. Peking University Medical Informatics Center, Beijing 100191, China
    4. National Institute of Health Data Science, Peking University, Beijing 100191, China
  • Received:2019-11-26 Online:2021-12-18 Published:2021-12-13
  • Contact: Gui-lan KONG E-mail:guilan.kong@hsc.pku.edu.cn
  • Supported by:
    National Natural Science Foundation of China(81771938);National Natural Science Foundation of China(91846101);Beijing Municipal Science & Technology Commission(7212201);Chinese Scientific and Technical Innovation Project 2030(2018AAA0102100);University of Michigan Health System-Peking University Health Science Center Joint Institute for Translational and Clinical Research(BMU2020JI011)

RICH HTML

 31   

PDF (PC)

152

Abstract:

Objective: To construct length of intensive care unit (ICU) stay (LOS-ICU) prediction models for ICU patients, based on three machine learning models: support vector machine (SVM), classification and regression tree (CART), and random forest (RF), and to compare the prediction perfor-mance of the three machine learning models with the customized simplified acute physiology score Ⅱ(SAPS-Ⅱ) model. Methods: We used medical information mart for intensive care (MIMIC)-Ⅲ database for model development and validation. The primary outcome was prolonged LOS-ICU(pLOS-ICU), defined as longer than the third quartile of patients’ LOS-ICU in the studied dataset. The recursive feature elimination method was used to do feature selection for three machine learning models. We utilized 5-fold cross validation to evaluate model prediction performance. The Brier value, area under the receiver operation characteristic curve (AUROC), and estimated calibration index (ECI) were used as perfor-mance measures. Performances of the four models were compared, and performance differences between the models were assessed using two-sided t test. The model with the best prediction performance was employed to generate variable importance ranking, and the identified top five important predictors were pre-sented. Results: The final cohort in our study consisted of 40 200 eligible ICU patients, of whom 23.7% were with pLOS-ICU. The proportion of the male patients was 57.6%, and the age of all the ICU patients was (61.9±16.5) years. Results showed that the three machine learning models outperformed the customized SAPS-Ⅱ model in terms of all the performance measures with statistical significance (P<0.01). Among the three machine learning models, the RF model achieved the best overall performance (Brier value, 0.145), discrimination (AUROC, 0.770) and calibration (ECI, 7.259). The calibration curve showed that the RF model slightly overestimated the risk of pLOS-ICU in high-risk ICU patients, but underestimated the risk of pLOS-ICU in low-risk ICU patients. Top five important predictors for pLOS-ICU identified by the RF model included age, heart rate, systolic blood pressure, body tempe-rature, and ratio of arterial oxygen tension to the fraction of inspired oxygen(PaO2/FiO2). Conclusion: The RF algorithm-based pLOS-ICU prediction model had a best prediction performance in this study. It lays a foundation for future application of the RF-based pLOS-ICU prediction model in ICU clinical practice.

Key words: Intensive care units, Length of stay, Machine learning, Random forest, Simplified acute physiology score

CLC Number: 

  • R181.2

Table 1

Characteristics of ICU patients in MIMIC-Ⅲ (n=40 200)"

Items Data
Age/years 61.9±16.5
Gender, n (%)
Male 23 145 (57.6)
Female 17 055 (42.4)
SAPS-Ⅱ score 32.5 ± 12.6
LOS-ICU / d, M (Q1-Q3) 2.1 (1.2-3.9)
pLOS-ICU, n (%) 9 514 (23.7)

Figure 1

The pLOS-ICU proportion pyramid stratified by age and gender pLOS-ICU, prolonged length of ICU stay."

Figure 2

The relationship curve that the prediction performance of RF model varies with the number of selected features AUROC, area under the receiver operation characteristic curve."

Table 2

Prediction performance of the four models"

Model Brier value AUROC ECI
Customized SAPS-Ⅱ 0.169 0.666 8.435
SVM 0.152* 0.738* 7.596*
CART 0.154* 0.730* 7.712*
RF 0.145* 0.770* 7.259*

Figure 3

Calibration plot of RF model RF, random forest."

[1] Bin D, Youzhong A, Yan K, et al. Characteristics of critically ill patients in ICUs in mainland China[J]. Crit Care Med, 2013, 41(1):84-92.
doi: 10.1097/CCM.0b013e31826a4082 pmid: 23222268
[2] Milbrandt EB, Kersten A, Rahim MT, et al. Growth of intensive care unit resource use and its estimated cost in Medicare[J]. Crit Care Med, 2008, 36(9):2504-2510.
doi: 10.1097/CCM.0b013e318183ef84 pmid: 18679127
[3] Tsai PF, Chen PC, Chen YY, et al. Length of hospital stay prediction at the admission stage for cardiology patients using artificial neural network[J]. J Healthc Eng, 2016, 2016:7035463.
[4] Nassar AP, Caruso P. ICU physicians are unable to accurately predict length of stay at admission: A prospective study[J]. Int J Qual Health Care, 2016, 28(1):99-103.
doi: 10.1093/intqhc/mzv112
[5] Le Gall JR, Lemeshow S, Saulnier F. A new simplified acute physiology ccore (SAPSⅡ) based on a European/North American multicenter study[J]. JAMA, 1993, 270(24):2957-2963.
doi: 10.1001/jama.1993.03510240069035
[6] Zimmerman JE, Kramer AA, Mcnair DS, et al. Acute physiology and chronic health evaluation (APACHE) Ⅳ: Hospital mortality assessment for today’s critically ill patients[J]. Crit Care Med, 2006, 34(5):1297-1310.
pmid: 16540951
[7] Vasilevskis EE, Kuzniewicz MW, Cason BA, et al. Mortality probability model Ⅲ and simplified acute physiology scoreⅡassessing their value in predicting length of stay and comparison to APACHE Ⅳ[J]. Chest, 2009, 136(1):89-101.
doi: S0012-3692(09)60409-8 pmid: 19363210
[8] Zimmerman JE, Kramer AA, Mcnair DS, et al. Intensive care unit length of stay: Benchmarking based on acute physiology and chronic health evaluation (APACHE) Ⅳ[J]. Crit Care Med, 2006, 34(10):2517-2529.
pmid: 16932234
[9] Lin K, Hu Y, Kong G. Predicting in-hospital mortality of patients with acute kidney injury in the ICU using random forest model[J]. Int J Med Inform, 2019, 125:55-61.
doi: S1386-5056(18)31087-6 pmid: 30914181
[10] Saria S, Rajani AK, Gould J, et al. Integration of early physiolo-gical responses predicts later illness severity in preterm infants[J]. Sci Transl Med, 2010, 2(48):48-65.
[11] Johnson AEW, Pollard TJ, Shen L, et al. MIMIC-Ⅲ, a freely accessible critical care database[J]. Sci Data, 2016, 3:160035.
doi: 10.1038/sdata.2016.35
[12] Agrawal S, Chen L, Tergas A I, et al. Characteristics associated with prolonged length of stay after hysterectomy for benign gynecologic conditions [J]. Am J Obstet Gynecol, 2018, 219(1): 89.e1-89.e15.
doi: 10.1016/j.ajog.2018.05.001
[13] Wolff J, Mccrone P, Patel A, et al. Predictors of length of stay in psychiatry: Analyses of electronic medical records[J]. BMC Psychiatry, 2015, 15:238.
doi: 10.1186/s12888-015-0623-6 pmid: 26446584
[14] Louw N, Steel SJ. Variable selection in kernel Fisher discriminant analysis by means of recursive feature elimination[J]. Comput Stat Data Anal, 2006, 51(3):2043-2055.
doi: 10.1016/j.csda.2005.12.018
[15] Boser BE, Guyon IM, Vapnik VN. A training algorithm for optimal margin classifiers[C]// Proceedings of the fifth annual workshop on computational learning theory. New York: Association for Computing Machinery, 1992: 144-152.
[16] Furey TS, Cristianini N, Duffy N, et al. Support vector machine classification and validation of cancer tissue samples using microarray expression data[J]. Bioinformatics, 2000, 16(10):906-914.
pmid: 11120680
[17] Breiman L, Friedman JH, Olshen RA, et al. Classification and regression trees[M]. Spain: Routledge, 2017.
[18] Austin PC. A comparison of regression trees, logistic regression, generalized additive models, and multivariate adaptive regression splines for predicting AMI mortality[J]. Stat Med, 2007, 26(15):2937-2957.
pmid: 17186501
[19] Breiman L. Random forests[J]. Machine Learning, 2001, 45(1):5-32.
doi: 10.1023/A:1010933404324
[20] Supatcha L, Chinae T, Chakarida N, et al. Heterogeneous ensemble approach with discriminative features and modified-SMOTE bagging for pre-miRNA classification[J]. Nucleic Acids Res, 2013, 41(1):e21.
doi: 10.1093/nar/gks878
[21] Wang SQ, Yang J, Chou KC. Using stacked generalization to predict membrane protein types based on pseudo-amino acid composition[J]. J Theor Biol, 2006, 242(4):941-946.
doi: 10.1016/j.jtbi.2006.05.006
[22] Touw WG, Bayjanov JR, Overmars L, et al. Data mining in the life sciences with random forest: A walk in the park or lost in the jungle?[J]. Brief Bioinformatics, 2013, 14(3):315-326.
doi: 10.1093/bib/bbs034
[23] Steyerberg EW, Vickers AJ, Cook NR, et al. Assessing the performance of prediction models a framework for traditional and novel measures[J]. Epidemiology, 2010, 21(1):128-138.
doi: 10.1097/EDE.0b013e3181c30fb2 pmid: 20010215
[24] van Hoorde K, van Huffel S, Timmerman D, et al. A spline-based tool to assess and visualize the calibration of multiclass risk predictions[J]. J Biomed Inform, 2015, 54:283-293.
doi: 10.1016/j.jbi.2014.12.016 pmid: 25579635
[25] Peigne V, Somme D, Guerot E, et al. Treatment intensity, age and outcome in medical ICU patients: Results of a French admi-nistrative database[J]. Ann Intensive Care, 2016, 6(1):7.
doi: 10.1186/s13613-016-0107-y
[26] Fallenius M, Skrifvars MB, Reinikainen M, et al. Common intensive care scoring systems do not outperform age and glasgow coma scale score in predicting mid-term mortality in patients with spontaneous intracerebral hemorrhage treated in the intensive care unit[J]. Scand J Trauma Resusc Emerg Med, 2017, 25(1):102.
doi: 10.1186/s13049-017-0448-z
[27] Szalados JE. Long-term mortality and quality of life after prolonged mechanical ventilation. Age and functional status as determinants of intensive care unit outcome: Sound basis for health policy or tip of the outcomes iceberg[J]. Crit Care Med, 2004, 32(1):291-293.
pmid: 14707597
[28] Brandberg C, Blomqvist H, Jirwe M. What is the importance of age on treatment of the elderly in the intensive care unit?[J]. Acta Anaesthesiol Scand, 2013, 57(6):698-703.
doi: 10.1111/aas.12073 pmid: 23373851
[29] Diringer MN, Reaven NL, Funk SE, et al. Elevated body temperature independently contributes to increased length of stay in neurologic intensive care unit patients[J]. Crit Care Med, 2004, 32(7):1489-1495.
pmid: 15241093
[30] Yien HW, Hseu SS, Lee LC, et al. Spectral analysis of systemic arterial pressure and heart rate signals as a prognostic tool for the prediction of patient outcome in the intensive care unit[J]. Crit Care Med, 1997, 25(2):258-266.
pmid: 9034261
[31] Grander W, Mullauer K, Koller B, et al. Heart rate before ICU discharge: a simple and readily available predictor of short- and long-term mortality from critical illness[J]. Clin Res Cardiol, 2013, 102(8):599-606.
doi: 10.1007/s00392-013-0571-4
[32] Esteve F, Lopez-Delgado JC, Javierre C, et al. Evaluation of the PaO2/FiO2 ratio after cardiac surgery as a predictor of outcome during hospital stay[J]. BMC Anesthesiol, 2014, 14:83.
doi: 10.1186/1471-2253-14-83
[33] Hu BJ, Tao LL, Rosenthal VD, et al. Device-associated infection rates, device use, length of stay, and mortality in intensive care units of 4 Chinese hospitals: International Nosocomial Control Consortium findings[J]. Am J Infect Control, 2013, 41(4):301-306.
doi: 10.1016/j.ajic.2012.03.037
[34] 徐文秀, 方理超, 刘励军. APACHE Ⅱ评分和 SAPSⅡ评分预测危重症患者病死率的应用价值分析[J]. 中国血液流变学杂志, 2010, (2):245-247.
[35] 汪洋, 陈上仲, 胡才宝, 等. 基于随机森林法的严重脓毒症/脓毒性休克预后评估模型对患者28 d死亡的预测价值[J]. 中华危重病急救医学, 2017, 29(12):1071-1076.
[1] LIN Yu,WU Jing-yi,LIN Ke,HU Yong-hua,KONG Gui-lan. Prediction of intensive care unit readmission for critically ill patients based on ensemble learning [J]. Journal of Peking University (Health Sciences), 2021, 53(3): 566-572.
[2] Xue-hua ZHU,Ming-yu YANG,Hai-zhui XIA,Wei HE,Zhi-ying ZHANG,Yu-qing LIU,Chun-lei XIAO,Lu-lin MA,Jian LU. Application of machine learning models in predicting early stone-free rate after flexible ureteroscopic lithotripsy for renal stones [J]. Journal of Peking University(Health Sciences), 2019, 51(4): 653-659.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
[1] Author. English Title Test[J]. Journal of Peking University(Health Sciences), 2010, 42(1): 1 -10 .
[2] . [J]. Journal of Peking University(Health Sciences), 2009, 41(2): 188 -191 .
[3] . [J]. Journal of Peking University(Health Sciences), 2009, 41(3): 376 -379 .
[4] . [J]. Journal of Peking University(Health Sciences), 2009, 41(4): 459 -462 .
[5] . [J]. Journal of Peking University(Health Sciences), 2009, 41(6): 682 -686 .
[6] . [J]. Journal of Peking University(Health Sciences), 2010, 42(1): 82 -84 .
[7] . [J]. Journal of Peking University(Health Sciences), 2007, 39(3): 319 -322 .
[8] . [J]. Journal of Peking University(Health Sciences), 2007, 39(3): 333 -336 .
[9] . [J]. Journal of Peking University(Health Sciences), 2007, 39(3): 337 -340 .
[10] . [J]. Journal of Peking University(Health Sciences), 2007, 39(3): 225 -328 .
Copyright © Journal of Peking University (Health Sciences), All Rights Reserved.
Powered by Beijing Magtech Co. Ltd