Research Progress on the Basic Medical Abilities of Large Language Models and Their Clinical Applications in Cerebrovascular Diseases

doi:10.3969/j.issn.1673-5765.2024.06.002

Abstract

Abstract: In recent years, large language models have demonstrated remarkable intelligence capabilities across various general domains and have been widely and effectively applied in multiple industries. However, due to the high level of specialization and complexity of scenarios in clinical medicine, there is no consensus on whether large language models can provide accurate, reliable, and efficient medical services. This study reviewed the progress of large language models in clinical medicine from the following aspects: evaluation of medical knowledge and basic medical capabilities; research progress on specific capability requirements in clinical scenarios; research progress on clinical diseases such as cerebrovascular diseases and related clinical applications.

Key words: Large language model; Medical large language model; Intelligent healthcare

摘要： 近年来，大语言模型在通用领域涌现出了惊人的智慧能力，并快速在各行业得到了广泛而有效的应用。然而，临床医学领域由于专业性强、场景复杂，大语言模型能否提供准确、可靠、高效的医疗服务，目前还没有形成一致性的结论。本研究从以下几个方面对大语言模型在临床医学中的研究进展进行综述：医学知识和基本医学能力评测情况；临床场景特定能力方面的研究进展；脑血管病等临床疾病及相关临床应用方面的研究进展。

关键词: 大语言模型; 医疗大语言模型; 智慧医疗

CLC Number:

LIU Xien, LIU Shaohui, ZHOU Kaiyin, YOU Xinxin, ZHOU Yuxuan, NING Chen, FU Xiangling, WU Ji. Research Progress on the Basic Medical Abilities of Large Language Models and Their Clinical Applications in Cerebrovascular Diseases[J]. Chinese Journal of Stroke, 2024, 19(6): 614-620.

刘喜恩, 刘少辉, 周开银, 尤心心, 周宇轩, 宁辰, 傅湘玲, 吴及. 大语言模型基本医学能力及其在脑血管病等临床应用上的研究进展[J]. 中国卒中杂志, 2024, 19(6): 614-620.

References

[1] ACHIAM J，ADLER S，AGARWAL S，et al. GPT-4 technical report[J/OL]. ArXiv Preprint，2023，arXiv：2303.08774[2024-05-10]. https://doi.org/10.48550/arXiv.2303.08774.
[2] TOUVRON H，MARTIN L，STONE K，et al. LLaMA 2：open foundation and fine-tuned chat models[J/OL]. ArXiv Preprint，2023，arXiv：2307.09288[2024-05-10]. https://doi.org/10.48550/arXiv.2307.09288.
[3] ANIL R，DAI A M，FIRAT O，et al. PaLM 2 technical report[J/OL]. ArXiv Preprint，2023，arXiv：2305.10403[2024-05-10]. https://doi.org/10.48550/arXiv.2305.10403.
[4] SINGHAL K，AZIZI S，TU T，et al. Large language models encode clinical knowledge[J]. Nature，2023，620（7972）：172-180.
[5] DU Z X，QIAN Y J，LIU X，et al. GLM：general language model pretraining with autoregressive blank infilling[C]//Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics（Volume 1：Long Papers）. Dublin：[s.n.]，2022：320-335.
[6] MENG Z，LIU F，SHAREGHI E，et al. Rewire-then-probe：a contrastive recipe for probing biomedical knowledge of pre-trained language models[C]//Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics（Volume 1：Long Papers）. Dublin：[s.n.]，2022：4798-4810.
[7] BODENREIDER O. The unified medical language system（UMLS）：integrating biomedical terminology[J/OL]. Nucleic Acids Res，2004，32（suppl_1）：D267-D270[2024-05-10]. https://doi.org/10.1093/nar/gkh061.
[8] CHUNG H W，HOU L，LONGPRE S，et al. Scaling instruction-finetuned language models[J]. J Mach Learn Res，2024，25（70）：1-53.
[9] ZHOU Y X，LIU X E，CHEN N，et al. MultifacetEval：multifaceted evaluation to probe LLMs in mastering medical knowledge[J/OL]. ArXiv Preprint，2024，arXiv：2406.02919[2024-05-10]. https://doi.org/10.48550/arXiv.2406.02919.
[10] LIU J L，ZHOU P L，HUA Y，et al. Benchmarking large language models on CMExam—a comprehensive Chinese medical exam dataset[J/OL]. Adv Neural Inf Process Syst，2023，36：52430-52452[2024-05-10]. https://openreview.net/pdf？id=HhcQ0zeqZp.
[11] CAI Y，WANG L L，WANG Y，et al. MedBench：a large-scale Chinese benchmark for evaluating medical large language models[J/OL]. ArXiv Preprint，2024，arXiv：2312.12806v1[2024-05-10]. https://doi.org/10.48550/arXiv.2312.12806v1.
[12] SANDMANN S，RIEPENHAUSEN S，PLAGWITZ L，et al. Systematic analysis of ChatGPT，Google search and LLaMA 2 for clinical decision support tasks[J/OL]. Nat Commun，2024，15（1）：2050[2024-05-10]. https://doi.org/10.1038/s41467-024-46411-8.
[13] HABIB S，BUTT H，GOLDENHOLZ S R，et al. Large language model performance on practice epilepsy board examinations[J]. JAMA Neurol，2024，81（6）：660-661.
[14] LIU F L，ZHOU H J，HUA Y，et al. Large language models in healthcare：a comprehensive benchmark[J/OL]. medRxiv Preprint，2024：24.24306315[2024-05-10]. https://doi.org/10.1101/2024.04.24.24306315.
[15] YANG X，CHEN A，POURNEJATIAN N，et al. A large language model for electronic health records[J/OL]. NPJ Digit Med，2022，5（1）：194[2024-05-10]. https://doi.org/10.1038/s41746-022-00742-2.
[16] SAVAGE T，NAYAK A，GALLO R，et al. Diagnostic reasoning prompts reveal the potential for large language model interpretability in medicine[J/OL]. NPJ Digit Med，2024，7（1）：20[2024-05-10]. https://doi.org/10.1038/s41746-024-01010-1.
[17] KWON T，ONG K T，KANG D，et al. Large language models are clinical reasoners：reasoning-aware diagnosis framework with prompt-generated rationales[C/OL]. Proceedings of the AAAI Conference on Artificial Intelligence，2024，38（16）：18417-18425[2024-04-10]. https://arxiv.org/abs/2312.07399.
[18] MENZ B D，KUDERER N M，BACCHI S，et al. Current safeguards，risk mitigation，and transparency measures of large language models against the generation of health disinformation：repeated cross sectional analysis[J/OL]. BMJ，2024，384：e078538[2024-05-10]. https://doi.org/10.1136/bmj-2023-078538.
[19] TEAM G，ANIL R，BORGEAUD S，et al. Gemini：a family of highly capable multimodal models[J/OL]. ArXiv Preprint，2023，arXiv：2312.11805[2024-05-10]. https://doi.org/10.48550/arXiv.2312.11805.
[20] AGARWAL M，GOSWAMI A，SHARMA P. Evaluating ChatGPT-3.5 and Claude-2 in answering and explaining conceptual medical physiology multiple-choice questions[J/OL]. Cureus，2023，15（9）：e46222[2024-05-10]. https://doi.org/10.7759/cureus.46222.
[21] EREMEEV M，VALMIANSKI I，AMATRIAIN X，et al. Injecting knowledge into language generation：a case study in auto-charting after-visit care instructions from medical dialogue[C]//Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics（Volume 1：Long Papers）. Toronto：[s.n.]，2023：2373-2390.
[22] SOUSA S，KERN R. How to keep text private？A systematic review of deep learning methods for privacy-preserving natural language processing[J]. Artificial Intelligence Review，2023，56（2）：1427-1492.
[23] ZHANG X L，LI S Y，YANG X J，et al. Enhancing small medical learners with privacy-preserving contextual prompting[J/OL]. ArXiv Preprint，2023，[24]arXiv：2305.12723[2024-05-10]. https://doi.org/10.48550/arXiv.2305.12723.
[24] SINGH C，ASKARI A，CARUANA R，et al. Augmenting interpretable models with large language models during training[J/OL]. Nat Commun，2023，14（1）：7913[2024-05-10]. https://doi.org/10.1038/s41467-023-43713-1.
[25] JIANG P C，XIAO C，CROSS A R，et al. GraphCare：enhancing healthcare predictions with personalized knowledge graphs[J/OL]. ArXiv Preprint，2023，arXiv：2305.12788[2024-05-10]. https://doi.org/10.48550/arXiv.2305.12788.
[26] ALTUNISIK E. Artificial intelligence and cerebrovascular diseases：ChatGPT model[J]. Cerebrovasc Dis，2024，53（3）：354-358.
[27] CHLOROGIANNIS D D，APOSTOLOS A，CHLOROGIANNIS A，et al. The role of ChatGPT in the advancement of diagnosis，management，and prognosis of cardiovascular and cerebrovascular disease[J/OL]. Healthcare（Basel），2023，11（21）：2906[2024-05-10]. https://doi.org/10.3390/healthcare11212906.
[28] BEAULIEU-JONES B K，VILLAMAR M F，SCORDIS P，et al. Predicting seizure recurrence after an initial seizure-like episode from routine clinical notes using large language models：a retrospective cohort study[J/OL]. Lancet Digit Health，2023，5（12）：e882-e894[2024-05-10]. https://doi.org/10.1016/s2589-7500（23）00179-6.
[29] FENG Y J，XU X Y，ZHUANG Y T，et al. Large language models improve Alzheimer’s disease diagnosis using multi-modality data[C]//2023 IEEE International Conference on Medical Artificial Intelligence（MedAI）. Haikou：[s.n.]，2023：61-66.
[30] BETZLER B K，CHEN H C，CHENG C Y，et al. Large language models and their impact in ophthalmology[J/OL]. Lancet Digit Health，2023，5（12）：e917-e924[2024-05-10]. https://doi.org/10.1016/s2589-7500（23）00201-7.
[31] YALAMANCHILI A，SENGUPTA B，SONG J，et al. Quality of large language model responses to radiation oncology patient care questions[J/OL]. JAMA Netw Open，2024，7（4）：e244630[2024-05-10]. https://doi.org/10.1001/jamanetworkopen.2024.4630.
[32] GRIEWING S，GREMKE N，WAGNER U，et al. Challenging ChatGPT 3.5 in senology—an assessment of concordance with breast cancer tumor board decision making[J/OL]. J Pers Med，2023，13（10）：1502[2024-05-10]. https://doi.org/10.3390/jpm13101502.
[33] SARRAJU A，BRUEMMER D，VAN ITERSON E，et al. Appropriateness of cardiovascular disease prevention recommendations obtained from a popular online chat-based artificial intelligence model[J]. JAMA，2023，329（10）：842-844.
[34] PAIS C，LIU J F，VOIGT R，et al. Large language models for preventing medication direction errors in online pharmacies[J/OL]. Nat Med，2024：1-9[2024-05-10]. https://doi.org/10.1038/s41591-024-02933-8.
[35] PASLı S，ŞAHIN A S，BEŞER M F，et al. Assessing the precision of artificial intelligence in emergency department triage decisions：insights from a study with ChatGPT[J/OL]. Am J Emerg Med，2024，78：170-175[2024-05-10]. https://doi.org/10.1016/j.ajem.2024.01.037.
[36] VAN VEEN D，VAN UDEN C，BLANKEMEIER L，et al. Adapted large language models can outperform medical experts in clinical text summarization[J]. Nat Med，2024，30（4）：1-9.
[37] WANG H Y，GAO C F，DANTONA C，et al. DRG-LLaMA：tuning LLaMA model to predict diagnosis-related group for hospitalized patients[J/OL]. NPJ Digit Med，2024，7（1）：16[2024-05-10]. https://doi.org/10.1038/s41746-023-00989-3.