Development and Validation of a Predictive Model for In-Hospital Recurrence Risk in Ischemic Stroke Patients

doi:10.3969/j.issn.1673-5765.2023.12.009

Abstract

Abstract: Objective Develop a predictive model for in-hospital recurrence risk of ischemic stroke patients based on machine learning algorithms and externally validate it to provide insights for related research.
Methods The development cohort was the China Stroke Center Alliance (CSCA) research cohort, and ischemic stroke patients in this cohort were randomly divided into training and internal validation sets in an 8∶2 ratio. The validation cohort was the Third China National Stroke Registry (CNSR-Ⅲ) research cohort. A list of candidate predictive factors was determined based on guidelines, literature, and data, followed by selection using least absolute shrinkage and selection operator (LASSO) regression. A predictive model for the in-hospital recurrence risk of ischemic stroke patients was developed using logistic regression and machine learning algorithms[random forest model, eXtreme gradient boosting (XGBoost) model, light gradient boosting machine (LightGBM) model]. Model evaluation primarily focused on discrimination (C-statistic) and calibration (Brier score).
Results The CSCA research cohort included 1 587 779 cases of ischemic stroke patients, with 99 085 cases of in-hospital recurrence (6.2%). The CNSR-Ⅲ research cohort included 14 146 cases of ischemic stroke patients, with 623 cases of in-hospital recurrence (4.4%). LASSO feature selection revealed that age, gender, stroke history, hypertension, diabetes, lipid metabolism disorders, atrial fibrillation, heart failure, coronary artery heart disease, peripheral vascular disease, LDL-C, fasting blood glucose, serum creatinine and in-hospital anticoagulation therapy were important predictive factors for predicting in-hospital recurrence of ischemic stroke patients. In internal validation, the discrimination of each model was around 0.75, with XGBoost model slightly outperforming other models (AUC 0.765, 95%CI 0.759-0.770), and the Brier scores for all models were around 0.05. In external validation, the predictive performance of all models was relatively low (AUC<0.60), with Brier scores for all models less than 0.08.
Conclusions In the limited context of the number and dimensions of predictive factors, the efficacy of logistic models and machine learning algorithms in predicting the recurrence risk of stroke was relatively low. Future exploration should involve more investigation into predictive factors and algorithm models.

Key words: Ischemic stroke; In-hospital recurrence; Predictive model; Machine learning

摘要： 目的开发基于机器学习算法的缺血性卒中患者院内复发风险的预测模型，并进行外部验证，为相关研究提供借鉴。
方法开发队列为中国卒中联盟（China Stroke Center Alliance，CSCA）研究队列，将此队列中的缺血性卒中患者按照8∶2随机划分为训练集和内部验证集。验证队列为第3次中国国家卒中登记（the third China national stroke registry，CNSR-Ⅲ）研究队列。基于指南、文献回顾，确定备选预测因子，然后采用拉索（least absolute shrinkage and selection operator，LASSO）回归进行筛选。基于logistic回归模型以及机器学习算法[随机森林模型、极端梯度提升（extreme gradient boosting，XGBoost）、轻量级梯度提升机器学习（light gradient boosting machine，LightGBM）模型]开发缺血性卒中患者院内复发风险预测模型。评价模型区分度（C统计量）和校准度（Brier得分）两方面的指标。
结果 CSCA研究队列共纳入1 587 779例缺血性卒中患者，其中院内复发99 085例（6.2%）。CNSR-Ⅲ研究队列共纳入14 146例缺血性卒中患者，其中院内复发623例（4.4%）。LASSO回归选择出年龄、性别、卒中病史、高血压、糖尿病、脂质代谢紊乱、心房颤动、心力衰竭、冠心病、周围血管病、LDL-C、空腹血糖、血清肌酐以及院内抗栓治疗作为缺血性卒中院内复发的预测因子。内部验证中，各模型的区分度均在0.75左右，XGBoost模型的区分度（AUC 0.765，95%CI 0.759～0.770）略高于其他模型，各模型的Brier分数均在0.05左右。外部验证中，所有模型的预测效能均较低（AUC＜0.60），各模型的Brier分数均＜0.08。
结论在预测因子数量和维度有限的情况下，logistic回归模型和机器学习算法预测缺血性卒中院内复发风险的效能均较低。未来需从预测因子和算法模型上做更多探索。

关键词: 缺血性卒中; 院内复发; 预测模型; 机器学习

CHEN Siding, JIANG Yingyu, WANG Chunjuan, YANG Xin, LI Zixiao, JIANG Yong, WANG Yongjun, GU Hongqiu. Development and Validation of a Predictive Model for In-Hospital Recurrence Risk in Ischemic Stroke Patients[J]. Chinese Journal of Stroke, 2023, 18(12): 1397-1404.

陈思玎, 姜英玉, 王春娟, 杨昕, 李子孝, 姜勇, 王拥军, 谷鸿秋. 缺血性卒中患者院内复发风险预测模型开发与验证研究[J]. 中国卒中杂志, 2023, 18(12): 1397-1404.

References

[1] WANG Y J，LI Z X，WANG Y L，et al. Chinese Stroke Center Alliance：a national effort to improve healthcare quality for acute stroke and transient ischaemic attack：rationale，design and preliminary findings[J]. Stroke Vasc Neurol，2018，3（4）：256-262.
[2] GBD 2016 Lifetime Risk of Stroke Collaborators，FEIGIN V L，NGUYEN G，et al. Global，regional，and country-specific lifetime risks of stroke，1990 and 2016[J]. N Engl J Med，2018，379（25）：2429-2437.
[3] JOHNSTON S C，ELM J J，EASTON J D，et al. Time course for benefit and risk of clopidogrel and aspirin after acute transient ischemic attack and minor ischemic stroke：a secondary analysis from the POINT randomized trial[J]. Circulation，2019，140（8）：658-664.
[4] PAN Y S，MENG X，JIN A M，et al. Time course for benefit and risk with ticagrelor and aspirin in individuals with acute ischemic stroke or transient ischemic attack who carry CYP2C19 loss-of-function alleles：a secondary analysis of the CHANCE-2 randomized clinical trial[J]. JAMA Neurol，2022，79（8）：739-745.
[5] WANG Y J，JING J，MENG X，et al. The Third China National Stroke Registry（CNSR-Ⅲ）for patients with acute ischaemic stroke or transient ischaemic attack：design，rationale and baseline patient characteristics[J]. Stroke Vasc Neurol，2019，4（3）：158-164.
[6] GU H Q，YANG X，WANG C J，et al. Clinical characteristics，management，and in-hospital outcomes in patients with stroke or transient ischemic attack in China[J/OL]. JAMA Netw Open，2021，4（8）：e2120745[2023-11-01]. https://doi.org/10.1001/jamanetworkopen.2021.20745.
[7] POWERS W J，RABINSTEIN A A，ACKERSON T，et al. Guidelines for the early management of patients with acute ischemic stroke：2019 update to the 2018 guidelines for the early management of acute ischemic stroke：a guideline for healthcare professionals from the American Heart Association/American Stroke Association[J/OL]. Stroke，2019，50（12）：e344-e418[2023-11-01]. https://doi.org/10.1161/STR.0000000000000211.
[8] CHAUDHARY D，ABEDI V，LI J，et al. Clinical risk score for predicting recurrence following a cerebral ischemic event[J/OL]. Front Neurol，2019，10：1106[2023-11-01]. https://doi.org/10.3389/fneur.2019.01106.
[9] TANG G Y，QI L，SUN Z P，et al. Evaluation and analysis of incidence and risk factors of lower extremity venous thrombosis after urologic surgeries：a prospective two-center cohort study using LASSO-logistic regression[J/OL]. Int J Surg，2021，89：105948[2023-11-01]. https://doi.org/10.1016/j.ijsu.2021.105948.
[10] 朱华锋. Logistic 模型的参数估计及其实证研究分析[J/OL]. 科技信息，2011，1：169-170[2023-11-01]. https://doi.org/10.3969/j.issn.1001-9960.2011.01.118.
ZHU H F. The parameter estimation and the existent problems analysis of the logistic model[J/OL]. Science & Technology Information，2011，1：169-170[2023-11-01]. https://doi.org/10.3969/j.issn.1001-9960.2011.01.118.
[11] BREIMAN L. Random forests[J]. Machine learning，Springer，2001，45：5-32.
[12] CHEN T，GUESTRIN C. Xgboost：a scalable tree boosting system[A]. Proceedings of the 22nd acm sigkdd international conference on knowledge discovery and data mining[C]. 2016，San Francisco，USA：785-794.
[13] KE G，MENG Q，FINLEY T，et al. Lightgbm：a highly efficient gradient boosting decision tree[J]. Advances in neural information processing systems，2017，30.
[14] HOSTETLER Z S，HSU F C，YOGANANDAN N，et al. An improved method for developing injury risk curves using the brier metric score[J]. Ann Biomed Eng，2021，49（11）：3091-3098.
[15] 陈旭，刘鹏鹤，孙毓忠，等. 面向不均衡医学数据集的疾病预测模型研究[J]. 计算机学报，2019，42（3）：596-609.
CHEN X，LIU P H，SUN Y Z，et al. Research on Disease Prediction Models Based on Imbalanced Medical Data Sets[J]. Chinese Journal of Computers，2019，42（3）：596-609.
[16] ELFIKY A A，PANY M J，PARIKH R B，et al. Development and application of a machine learning approach to assess short-term mortality risk among patients with cancer starting chemotherapy[J/OL]. JAMA Netw Open，2018，1（3）：e180926[2023-11-01]. https://doi.org/10.1001/jamanetworkopen.2018.0926.
[17] 王孟，覃露，王春娟，等. 基于机器学习算法的脑出血相关肺炎预测模型研究[J]. 中国卒中杂志，2020，15（3）：243-249.
WANG M，QIN L，WANG C J，et al. Machine learning-based models for prediction of intracerebral hemorrhage associated pneumonia[J]. Chin J Stroke，2020，15（3）：243-249.
[18] 上官艺，王孟，王春娟，等. 基于机器学习的缺血性卒中功能预后预测模型研究[J]. 中国卒中杂志，2021，16（9）：895-900.
SHANGGUAN Y，WANG M，WANG C J，et al. Machine learning-based models for prediction of functional outcome of ischemic stroke[J]. Chin J Stroke，2021，16（9）：895-900.
[19] CHEN S D，YOU J，YANG X M，et al. Machine learning is an effective method to predict the 90-day prognosis of patients with transient ischemic attack and minor stroke[J/OL]. BMC Med Res Methodol，2022，22（1）：195[2023-11-01]. https://doi.org/10.1186/s12874-022-01672-z.
[20] ABEDI V，AVULA V，CHAUDHARY D，et al. Prediction of long-term stroke recurrence using machine learning models[J/OL]. J Clin Med，2021，10（6）：1286[2023-11-01]. https://doi.org/10.3390/jcm10061286.
[21] 王玥，侯晓雯，陈会生，等. 基于颅内斑块影像组学联合传统标志物预测缺血性脑卒中复发风险[J]. 磁共振成像，2023，14（8）：1-9.
WANG Y，HOU X W，CHEN H S，et al. Prediction of the risk of recurrent ischemic stroke based on intracranial plaque radiomics with traditional biomarkers[J]. Chin J Magn Reson Imag，2023，14（8）：1-9.
[22] JING J，LIU Z，GUAN H，et al. A deep learning system to predict recurrence and disability outcomes in patients with transient ischemic attack or ischemic stroke[J/OL]. Adv Intell Syst，2023，5（4）：2200240[2023-11-01]. https://doi.org/10.1002/aisy.202200240.