Email Alert | RSS

Journal of Peking University (Health Sciences) ›› 2023, Vol. 55 ›› Issue (4): 670-675. doi: 10.19723/j.issn.1671-167X.2023.04.017

Previous Articles     Next Articles

Value of artificial intelligence in the improvement of diagnostic consistency of radiology residents

Xiang LIU1,Hui-hui XIE1,Yu-feng XU1,Xiao-dong ZHANG1,Xiao-feng TAO2,Lin LIU3,Xiao-ying WANG1,*()   

  1. 1. Department of Radiology, Peking University First Hospital, Beijing 100034, China
    2. Department of Radiology, Shanghai Ninth People's Hospital, Shanghai Jiaotong University School of Medicine, Shanghai 200011, China
    3. Department of Radiology, China-Japan Union Hospital of Jilin University, Changchun 130000, China
  • Received:2020-11-12 Online:2023-08-18 Published:2023-08-03
  • Contact: Xiao-ying WANG E-mail:wangxiaoying@bjmu.edu.cn

RICH HTML

   

PDF (PC)

Abstract:

Objective: To explore the value of artificial intelligence (AI) in improving the detection rate of traumatic rib fractures by radiologist residents and the consistency among different readers. Methods: Chest CT images of 393 patients with acute trauma from China-Japan Union Hospital of Jilin University (hospital 02) and Shanghai Ninth People' s Hospital (hospital 03) were collected in this research. The consensus achieved by three radiology experts was regarded as the reference standard. All the images assigned to three hospitals: Peking University First Hospital (hospital 01), hospital 02 and hospital 03, and were then randomly divided into two groups (group A and group B: group A included 197 patients, and group B included 196 patients). Each group was read by one radiologist resident from each hospital for rib fracture detection. Each case was read twice by the same radiologist, with and without the assistance of the AI ["radiologist-only" reading and "radiologist + AI" reading]. The detection rates of different types of rib fractures (displaced fractures and occult fractures) were compared between "radiologist-only" reading and "radiologist + AI" reading. The consistencies of different radiologists with different reading methods were evaluated. Results: The detection rates of displaced rib fractures and occult rib fractures by "radiologist + AI" reading were significantly higher than those read by "radiologist-only" reading (94.56% vs. 78.40%, 76.60% vs. 49.42%, P < 0.001). For "radiologist-only reading", the Kappa coefficients of the radiologists between hospital 01 and hospital 03 were slightly greater than 0.4 (indicating moderate consistency), the coefficients of the radiologists between hospital 01/hospital 02 and hospital 02/hospital 03 were less than 0.4 (indicating poor consistency). The Phi coefficients of the radiologists among different hospitals were all less than 0.6 (indicating moderate correlation). With "radiologist + AI" reading, the Kappa and Phi coefficient among the radiologists in dif-ferent hospitals were greater than or equal to 0.6 (indicating good consistency and correlation). Conclusion: AI software can be used to automatically detect suspected rib fracture lesions, which helps to improve the detection rate of fracture lesions and the consistency among different readers.

Key words: Rib fracture, Computed tomography imaging, Chest trauma, Artificial intelligence

CLC Number: 

  • R814.4

Table 1

The typical parameters of five CT scanning"

Parameters Brilliance 64 Brilliance 256 Lightspeed 16 NeuViz 128 Aquilion ONE
Scan length/mm 500 500 500 500 400-500
Tube voltage/kV 120 120 120 120 120
Tube current/mA 345 194-492 210 273-580 281-440
Slice thickness/mm 1 1 1.25 1 1
Image matrix 512×512 512×512 512×512 512×512 512×512
Kernal Bone Bone Bone Bone Bone

Table 2

Detection rates of different types of rib fractures"

Types of rib fractures Index Radiologist-only, n (%) Radiologist + AI, n (%) P value
Displaced fracture Detected 461(78.40) 556(94.56) < 0.001
Not detected 127(21.60) 32(5.44)
Total 588(100.00) 588(100.00)
Occult fracture Detected 642(49.42) 995(76.60) < 0.001
Not detected 657(50.58) 304(23.40)
Total 1 299(100.00) 1 299(100.00)
All fractures Detected 1 103(58.45) 1 551(82.19) < 0.001
Not detected 784(41.55) 336(17.81)
Total 1 887(100.00) 1 887(100.00)

Figure 1

A 65-year-old male patient with displaced fractures on the right 5th and 8th rib A, displaced fractures on the right 5th and 8th rib by reference standard reading (as shown in pink box); B, two rib fractures were detected by "radio-logist + AI" reading (as shown in yellow box); C, two rib fractures were detected by "radiologist-only" reading as well (as shown in orange box). AI, artificial intelligence."

Figure 2

A 60-year-old male patient with occult fracture on the left 6th rib A, occult fracture on the left 6th rib by reference standard reading (as shown in pink box); B, rib fracture was detected by "radiologist + AI" reading (as shown in yellow box); C, rib fracture was not detected by "radiologist-only" reading. AI, artificial intelligence."

Table 3

Consistency among different "radiologists-only" reading"

Items χ2 P value Phi coefficient Degree of correlation Kappa (95%CI) Degree of consistency
A1-A2 38.586 < 0.001 0.402 Moderate 0.371 (0.250, 0.493) Poor
A1-A3 61.843 < 0.001 0.509 Moderate 0.491 (0.366, 0.616) Moderate
A2-A3 23.119 < 0.001 0.311 Moderate 0.308 (0.182, 0.435) Poor
B1-B2 52.185 < 0.001 0.366 Moderate 0.365 (0.273, 0.457) Poor
B1-B3 67.168 < 0.001 0.415 Moderate 0.415 (0.325, 0.505) Moderate
B2-B3 58.210 < 0.001 0.386 Moderate 0.385 (0.294, 0.477) Poor

Table 4

The consistency among different "radiologists + AI" reading"

Items χ2 P value Phi coefficient Degree of correlation Kappa (95%CI) Degree of consistency
A1-A2 68.209 < 0.001 0.618 Good 0.601 (0.458, 0.744) Good
A1-A3 85.393 < 0.001 0.707 Good 0.698 (0.564, 0.832) Good
A2-A3 102.028 < 0.001 0.653 Good 0.652 (0.520, 0.783) Good
B1-B2 144.135 < 0.001 0.608 Good 0.596 (0.497, 0.695) Good
B1-B3 177.701 < 0.001 0.675 Good 0.668 (0.577, 0.759) Good
B2-B3 232.594 < 0.001 0.772 Good 0.771 (0.687, 0.855) Excellent
1 Hashmi ZG , Kaji AH , Nathens AB .Practical guide to surgical data sets: National trauma data bank (NTDB)[J].JAMA Surg,2018,153(9):852-853.
doi: 10.1001/jamasurg.2018.0483
2 Peek J , Beks RB , Hietbrink F , et al.Epidemiology and outcome of rib fractures: A nationwide study in the Netherlands[J].Eur J Trauma Emerg Surg,2022,48(1):265-271.
doi: 10.1007/s00068-020-01412-2
3 Weikert T , Noordtzij LA , Bremerich J , et al.Assessment of a deep learning algorithm for the detection of rib fractures on whole-body trauma computed tomography[J].Korean J Radiol,2020,21(7):891-899.
doi: 10.3348/kjr.2019.0653
4 Pishbin E , Ahmadi K , Foogardi M , et al.Comparison of ultrasonography and radiography in diagnosis of rib fractures[J].Chin J Traumatol,2017,20(4):226-228.
doi: 10.1016/j.cjtee.2016.04.010
5 Bizimungu R , Sergio A , Baumann BM , et al.Thoracic spine fracture in the panscan era[J].Ann Emerg Med,2020,76(2):143-148.
doi: 10.1016/j.annemergmed.2019.11.017
6 Dennis BM , Bellister SA , Guillamondegui OD .Thoracic trauma[J].Surg Clin North Am,2017,97(5):1047-1064.
doi: 10.1016/j.suc.2017.06.009
7 Cho SH , Sung YM , Kim MS .Missed rib fractures on evaluation of initial chest CT for trauma patients: Pattern analysis and diagnostic value of coronal multiplanar reconstruction images with multidetector row CT[J].Br J Radiol,2012,85(1018):e845-e850.
doi: 10.1259/bjr/28575455
8 Singleton JM , Bilello LA , Canham LS , et al.Chest computed tomography imaging utility for radiographically occult rib fractures in elderly fall-injured patients[J].J Trauma Acute Care Surg,2019,86(5):838-843.
doi: 10.1097/TA.0000000000002208
9 Talbot BS , Gange CP Jr , Chaturvedi A , et al.Traumatic rib injury: Patterns, imaging pitfalls, complications, and treatment[J].Radiographics,2017,37(2):628-651.
doi: 10.1148/rg.2017160100
10 Murphy CE , Raja AS , Baumann BM , et al.Rib fracture diagnosis in the panscan era[J].Ann Emerg Med,2017,70(6):904-909.
doi: 10.1016/j.annemergmed.2017.04.011
11 Langdorf MI , Medak AJ , Hendey GW , et al.Prevalence and clinical import of thoracic injury identified by chest computed tomography but not chest radiography in blunt trauma: Multicenter prospective cohort study[J].Ann Emerg Med,2015,66(6):589-600.
doi: 10.1016/j.annemergmed.2015.06.003
12 Kim J , Kim S , Kim YJ , et al.Quantitative measurement method for possible rib fractures in chest radiographs[J].Healthc Inform Res,2013,19(3):196-204.
doi: 10.4258/hir.2013.19.3.196
13 He J , Baxter SL , Xu J , et al.The practical implementation of artificial intelligence technologies in medicine[J].Nat Med,2019,25(1):30-36.
doi: 10.1038/s41591-018-0307-0
14 Gao XW , Qian Y .Prediction of multidrug-resistant TB from CT pulmonary images based on deep learning techniques[J].Mol Pharm,2018,15(10):4326-4335.
doi: 10.1021/acs.molpharmaceut.7b00875
15 Onishi Y , Teramoto A , Tsujimoto M , et al.Automated pulmonary nodule classification in computed tomography images using a deep convolutional neural network trained by generative adversarial networks[J].Biomed Res Int,2019,2019,6051939.
16 刘想, 谢辉辉, 许玉峰, 等.AI软件自动检出胸部CT图像上肋骨骨折的诊断效能研究[J].临床放射学杂志,2021,40(7):1369-1374.
17 Zhou QQ , Wang J , Tang W , et al.Automatic detection and classification of rib fractures on thoracic CT using convolutional neural network: Accuracy and feasibility[J].Korean J Radiol,2020,21(7):869-879.
doi: 10.3348/kjr.2019.0651
18 张武平, 王春风, 王琳, 等.人工智能辅助在提高CT肋骨骨折检出率作用分析[J].影像研究与医学应用,2020,4(13):211-212.
19 杨超朋, 赵俊彦, 何光龙, 等.基于深度学习的人体肋骨骨折智能检测技术[J].刑事技术,2021,46(2):134-139.
20 周清清, 王佳硕, 唐雯, 等.基于卷积神经网络成人肋骨骨折CT自动检测和分类的应用研究[J].影像诊断与介入放射学,2020,29(1):27-31.
21 Chapman BC , Herbert B , Rodil M , et al.RibScore: A novel radiographic score based on fracture pattern that predicts pneumonia, respiratory failure, and tracheostomy[J].J Trauma Acute Care Surg,2016,80(1):95-101.
doi: 10.1097/TA.0000000000000867
22 王建林, 李晓芬, 费峻眙.多层螺旋CT "一站式"图像后处理模式在肋骨骨折诊断以及司法鉴定中的应用探讨[J].现代医用影像学,2018,27(8):2755-2756.
23 羊鸿钧.多层螺旋CT三维重建在肋骨骨折诊断及法医临床鉴定中的应用[J].法制博览,2020,(8):155-156.
[1] Wen DU, Wenbo ZHANG, Yao YU, Shuo LIU, Huiyu SU, Leihao HU, Zunan TANG, Binzhang WU, Zhen CHEN, Jiaqi LI, Hao WANG, Xin PENG. Exploration and clinical application of the "digital and intelligent surgery" diagnosis and treatment workflow for oral and maxillofacial tumors [J]. Journal of Peking University (Health Sciences), 2026, 58(2): 278-284.
[2] Jiale GAO, Zhongtao ZHANG. Current status and future perspectives of precision treatment for locally advanced rectal cancer [J]. Journal of Peking University (Health Sciences), 2026, 58(2): 247-250.
[3] Jiafu JI, Jingtao WEI, Ke JI, Zhaode BU. Bottlenecks and breakthroughs in gastric cancer diagnosis and treatment: Towards a new era of precision and intelligent integration [J]. Journal of Peking University (Health Sciences), 2026, 58(2): 231-238.
[4] Zhemin LI, Jiafu JI, Guoxin LI, Ziyu LI, Zhaode BU, Xiangyu GAO, Di DONG, Lei TANG, Xiaofang XING, Shuqin JIA, Ting GUO, Lianhai ZHANG, Fei SHAN, Xin JI, Anqiang WANG. Development and dissemination of precision medicine approaches in gastric cancer management [J]. Journal of Peking University (Health Sciences), 2025, 57(5): 864-867.
[5] SUN Yu-chun,GUO Yu-qing,CHEN Hu,DENG Ke-hui,LI Wei-wei. Independent innovation research, development and transformation of precise bionic repair technology for oral prosthesis [J]. Journal of Peking University (Health Sciences), 2022, 54(1): 7-12.
[6] Fu-zheng GUO,Feng-xue ZHU,Jiu-xu DENG,Zhe DU,Xiu-juan ZHAO. Risk factors for mechanical ventilation in patients with severe multiple trauma [J]. Journal of Peking University (Health Sciences), 2020, 52(4): 738-742.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
Copyright © Journal of Peking University (Health Sciences), All Rights Reserved.
Powered by Beijing Magtech Co. Ltd