基于U型卷积神经网络学习的前列腺癌影像重建模型在手术导航中的应用

doi:10.19723/j.issn.1671-167X.2019.03.033

北京大学学报（医学版） ›› 2019, Vol. 51 ›› Issue (3): 596-601. doi: 10.19723/j.issn.1671-167X.2019.03.033

基于U型卷积神经网络学习的前列腺癌影像重建模型在手术导航中的应用

颜野^1,^*,夏海缀^1,^*,李旭升²,何为³,朱学华¹,张智荧¹,肖春雷¹,刘余庆¹,黄华⁴,何良华²,卢剑^1△()

收稿日期:2019-03-18 出版日期:2019-05-22 发布日期:2019-06-26
通讯作者: 颜野,夏海缀 E-mail:lujian@bjmu.edu.cn
作者简介:卢剑,北京大学第三医院泌尿外科主任医师、副教授、博士生导师。主要学术兼职有:中华医学会泌尿外科分会泌尿男科工程委员会委员、国家药品监督管理局医疗器械分类技术委员会和审评专家咨询委员会委员、中国医师协会超声分子影像与人工智能委员会委员、中国性学会性医学专业委员会委员、中国医药教育协会加速康复外科分会委员和北京医学会肿瘤分会及罕见病分会泌尿专业组委员等。主要研究方向包括肿瘤发病机制与医学图像处理。长期从事泌尿外科临床与基础研究,具备丰富的临床工作经验和较强的医学-信息科学交叉学科理论基础。主持国家重点研发计划数字诊疗专项、国家自然科学基金、北京市自然科学基金等多项国家及省部级科研课题;曾获教育部高等学校科学研究优秀成果奖(科学技术)科技进步二等奖和中华医学科技奖等省部级奖励3项。在The Journal of Urology等发表SCI论文16篇;在《北京大学学报(医学版)》《中华泌尿外科杂志》等国内核心期刊发表论文110篇,参编专业著作4部。
基金资助:
北京市自然科学基金(L172012)、国家自然科学基金(61871004)、北大医学青年科技创新培育基金(BMU2018ZHYL012)-中央高校基本科研业务费、同济大学“中央高校基本科研业务费”专项基金(kx0080020173428)

Application of U-shaped convolutional neural network in auto segmentation and reconstruction of 3D prostate model in laparoscopic prostatectomy navigation

Ye YAN^1,^*,Hai-zhui XIA^1,^*,Xu-sheng LI²,Wei HE³,Xue-hua ZHU¹,Zhi-ying ZHANG¹,Chun-lei XIAO¹,Yu-qing LIU¹,Hua HUANG⁴,Liang-hua HE²,Jian LU^1△()

Received:2019-03-18 Online:2019-05-22 Published:2019-06-26
Contact: Ye YAN,Hai-zhui XIA E-mail:lujian@bjmu.edu.cn
Supported by:
Supported by Beijing Natural Science Foundation (L172012), the National Natural Science Foundation of China (61871004), the Fundamental Research Funds for the Central Universities: Peking University Medicine Fund of Fostering Young Scholars’ Scientific & Technological Innovation (BMU2018ZHYL012), the Fundamental Research Funds for the Central Universities: Tongji University (kx0080020173428)

摘要/Abstract

摘要： 目的探讨基于U型卷积神经网络(U-shaped convolutional neural network, U-net)建立的前列腺磁共振图像自动化分割和重建3D模型对腹腔镜前列腺癌根治术进行术中认知导航的效果。方法应用含有人工注释的共5 000张前列腺癌磁共振影像训练集,训练U-net,构建了一套以临床需求为导向,稳定高效的全卷积神经网络算法模型,对前列腺磁共振图像进行区域化、多结构和精细自动化分割,并将分割数据使用医学影像处理交互平台(Medical Image Interaction Tool Kit,MITK)自动重建,以STL格式输出建模信息,应用平板电脑在术中展示前列腺模型,进行认知导航。结果基于201例前列腺癌患者的磁共振图像训练样本,在经典U-net基础上通过适应性改良,建立了一套结构简单、性能优秀的U-net,可以实现对前列腺、肿瘤、精囊腺、直肠等重要结构的单独分割,并进行三维可视化,直观地显示手术关键部位的结构关系和肿瘤侵犯程度。术中通过平板电脑同步展示3D模型,成功进行认知导航。结论通过改良的U-net可以自动化完成前列腺磁共振图像的结构化分割,通过重建局部解剖部位的3D模型用于术中认知融合导航,可以达到肿瘤可视化、降低手术切缘阳性率、提高手术效果的作用。

关键词: 卷积神经网络, 前列腺肿瘤, 成像, 三维, 外科手术, 计算机辅助

Abstract: Objective: To investigate the efficacy of intraoperative cognitive navigation on laparoscopic radical prostatectomy using 3D prostatic models created by U-shaped convolutional neural network (U-net) and reconstructed through Medical Image Interaction Tool Kit (MITK) platform. Methods: A total of 5 000 pieces of prostate cancer magnetic resonance (MR) imaging discovery sets with manual annotations were used to train a modified U-net, and a set of clinically demand-oriented, stable and efficient full convolutional neural network algorithm was constructed. The MR images were cropped and segmented automatically by using modified U-net, and the segmentation data were automatically reconstructed using MITK platform according to our own protocols. The modeling data were output as STL format, and the prostate models were simultaneously displayed on an android tablet during the operation to help achieving cognitive navigation. Results: Based on original U-net architecture, we established a modified U-net from a 201-case MR imaging training set. The network performance was tested and compared with human segmentations and other segmentation networks by using one certain testing data set. Auto segmentation of multi-structures (such as prostate, prostate tumors, seminal vesicles, rectus, neurovascular bundles and dorsal venous complex) were successfully achieved. Secondary automatic 3D reconstruction had been carried out through MITK platform. During the surgery, 3D models of prostatic area were simultaneously displayed on an android tablet, and the cognitive navigation was successfully achieved. Intra-operation organ visualization demonstrated the structural relationships among the key structures in great detail and the degree of tumor invasion was visualized directly. Conclusion: The modified U-net was able to achieve automatic segmentations of important structures of prostate area. Secondary 3D model reconstruction and demonstration could provide intraoperative visualization of vital structures of prostate area, which could help achieve cognitive fusion navigation for surgeons. The application of these techniques could finally reduce positive surgical margin rates, and may improve the efficacy and oncological outcomes of laparoscopic prostatectomy.

Key words: Convolutional neural network, Prostatic neoplasms, Imaging, three-dimensional, Surgery, computer-assisted

中图分类号:

R737.25

颜野,夏海缀,李旭升,何为,朱学华,张智荧,肖春雷,刘余庆,黄华,何良华,卢剑. 基于U型卷积神经网络学习的前列腺癌影像重建模型在手术导航中的应用[J]. 北京大学学报（医学版）, 2019, 51(3): 596-601.

Ye YAN,Hai-zhui XIA,Xu-sheng LI,Wei HE,Xue-hua ZHU,Zhi-ying ZHANG,Chun-lei XIAO,Yu-qing LIU,Hua HUANG,Liang-hua HE,Jian LU. Application of U-shaped convolutional neural network in auto segmentation and reconstruction of 3D prostate model in laparoscopic prostatectomy navigation[J]. Journal of Peking University(Health Sciences), 2019, 51(3): 596-601.

图/表 5

图1

图2

表1

图3

图4

参考文献 21

[1]	Siegel RL, Miller KD, Jemal A.Cancer statistics, 2018[J]. CA Cancer J Clin, 2018, 68(1): 7-30. doi: 10.3322/caac.21442
[2]	Simmons MN, Stephenson AJ, Klein EA.Natural history of biochemical recurrence after radical prostatectomy: risk assessment for secondary therapy[J]. Eur Urol, 2007, 51(5): 1175-1184. doi: 10.1016/j.eururo.2007.01.015
[3]	Van den Broeck T, van den Bergh R, Arfi N, et al. Prognostic value of biochemical recurrence following treatment with curative intent for prostate cancer: A systematic review [J/OL]. Eur Urol,(2018-10-17) [2019-02-15]. https://doi.org/10.1016/j.eururo.2018.10.011.
[4]	Ukimura O, Aron M, Nakamoto M, et al.Three-dimensional surgical navigation model with TilePro display during robot-assisted radical prostatectomy[J]. J Endourol, 2014, 28(6): 625-630. doi: 10.1089/end.2013.0749
[5]	Hughes-Hallett A, Mayer EK, Marcus HJ, et al.Augmented rea-lity partial nephrectomy: examining the current status and future perspectives[J]. Urology, 2014, 83(2): 266-273. doi: 10.1016/j.urology.2013.08.049
[6]	王燕, 高旭, 阳青松, 等. 3D打印技术辅助认知融合在前列腺穿刺活检术中的应用[J]. 临床泌尿外科杂志, 2016, 31(2): 104-107.
[7]	邵叶秦, 杨新. 基于随机森林的CT前列腺分割[J]. CT理论与应用研究, 2015, 24(5): 647-655.
[8]	Ronneberger O, Fischer P, Brox T.U-net: Convolutional networks for biomedical image segmentation[C]. International Conference on Medical image computing and computer-assisted intervention. Cham: Springer, 2015: 234-241.
[9]	詹曙, 梁植程, 谢栋栋. 前列腺磁共振图像分割的反卷积神经网络方法[J]. 中国图象图形学报, 2017, 22(4): 516-522.
[10]	Neher PF, Stieltjes B, Reisert M, et al.MITK global tractography[C]. Proceedings of SPIE: The International Society for Optical Engineering, 2012: 83144D. doi: 10.1117/12.911215.
[11]	Lecun Y, Bottou L, Bengio Y, et al.Gradient-based learning applied to document recognition[J]. Proceedings of the IEEE, 1998, 86(11): 2278-2324. doi: 10.1109/5.726791
[12]	Mahapatra D, Buhmann JM.Prostate MRI segmentation using learned semantic knowledge and graph cuts[J]. IEEE Transactions on Biomedical Engineering, 2014, 61(3): 756-764. doi: 10.1109/TBME.2013.2289306
[13]	Korez R, Likar B, Pernuš F, et al.Model-based segmentation of vertebral bodies from MR images with 3D CNNs[C]. International Conference on Medical Image Computing and Computer-Assisted Intervention. Cham: Springer, 2016: 433-441.
[14]	Brosch T, Tang LY, Yoo Y, et al.Deep 3D convolutional encoder networks with shortcuts for multiscale feature integration applied to multiple sclerosis lesion segmentation[J]. IEEE Transactions on Medical Imaging, 2016, 35(5): 1229-1239. doi: 10.1109/TMI.2016.2528821
[15]	Martínez F, Romero E, Dréan G, et al.Segmentation of pelvic structures for planning CT using a geometrical shape model tuned by a multi-scale edge detector[J]. Phys Med Biol, 2014, 59(6): 1471-1484. doi: 10.1088/0031-9155/59/6/1471
[16]	凌彤, 杨琬琪, 杨明. 利用多模态U形网络的CT图像前列腺分割[J]. 智能系统学报, 2018, 13(6): 981-988.
[17]	Ebbing J, Jäderling F, Collins JW, et al.Comparison of 3D printed prostate models with standard radiological information to aid understanding of the precise location of prostate cancer: A construct validation study[J]. PLoS One, 2018, 13(6): e199477.
[18]	Volonté F, Pugin F, Bucher P, et al.Augmented reality and image overlay navigation with OsiriX in laparoscopic and robotic surgery: not only a matter of fashion[J]. J Hepatobiliary Pancreat Sci, 2011, 18(4): 506-509. doi: 10.1007/s00534-011-0385-6
[19]	Teber D, Guven S, Simpfendorfer T, et al.Augmented reality: a new tool to improve surgical accuracy during laparoscopic partial nephrectomy? Preliminary in vitro and in vivo results[J]. Eur Urol, 2009, 56(2): 332-338. doi: 10.1016/j.eururo.2009.05.017
[20]	Porpiglia F, Fiori C, Checcucci E, et al.Augmented reality robot-assisted radical prostatectomy: Preliminary experience[J]. Urology, 2018, 115(5): 184. doi: 10.1016/j.urology.2018.01.028
[21]	Porpiglia F, Checcucci E, Amparore D, et al.Augmented-reality robot-assisted radical prostatectomy using hyper-accuracy three-dimensional reconstruction (HA 3DTM) technology: a radiological and pathological study[J]. BJU international, 2018, 123(5): 834-845.

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed

Group name	Warping error	Rand error	Pixel error
Human values	0.000 004	0.001 9	0.000 9
Zhan^[9]	0.000 410	0.036 7	0.068 8
U-net	0.000 340	0.036 0	0.059 9

基于U型卷积神经网络学习的前列腺癌影像重建模型在手术导航中的应用

Application of U-shaped convolutional neural network in auto segmentation and reconstruction of 3D prostate model in laparoscopic prostatectomy navigation

RICH HTML

PDF (PC)

摘要/Abstract

引用本文

使用本文

图/表 5

参考文献 21

相关文章 15

Metrics

本文评价

推荐阅读 10

[1]	刘思民,赵一姣,王晓燕,王祖华. 动态导航下不同深度环钻定位精确度的体外评价[J]. 北京大学学报（医学版）, 2022, 54(1): 146-152.
[2]	李怡,王丽瑜,刘晓强,周倜,吕季喆,谭建国. 不同材料及厚度椅旁CAD/CAM瓷贴面的边缘特征[J]. 北京大学学报（医学版）, 2022, 54(1): 140-145.
[3]	邱淑婷,朱玉佳,王时敏,王飞龙,叶红强,赵一姣,刘云松,王勇,周永胜. 姿势微笑位口唇对称参考平面的数字化构建及初步应用验证[J]. 北京大学学报（医学版）, 2022, 54(1): 193-199.
[4]	苏俊琪,宋扬,谢尚. 口腔鳞状细胞癌患者修复重建术后感染的病原学特征及感染风险预测模型的构建[J]. 北京大学学报（医学版）, 2022, 54(1): 68-76.
[5]	任国勇,吴雪梅,李颖,李婕妤,孙伟平,黄一宁. 大血管闭塞性脑卒中亚急性期磁敏感血管征的表现[J]. 北京大学学报（医学版）, 2021, 53(6): 1133-1138.
[6]	李媛,林红,张铁军. 对比传统成像与数字成像对牙科复合树脂X射线阻射性的影响[J]. 北京大学学报（医学版）, 2021, 53(5): 995-1001.
[7]	杨刚,胡文杰,曹洁,柳登高. 牙周健康的上颌前牙唇侧嵴顶上牙龈的三维形态分析[J]. 北京大学学报（医学版）, 2021, 53(5): 990-994.
[8]	邵振兴,宋庆法,赵宇晴,崔国庆. 一种结合线袢固定的关节镜下“嵌入式”喙突移位术:手术技术及术后影像学分析[J]. 北京大学学报（医学版）, 2021, 53(5): 896-901.
[9]	吴一凡,张晓圆,任爽,玉应香,常翠青. 基于磁共振的青年男性股四头肌的测量和评估[J]. 北京大学学报（医学版）, 2021, 53(5): 843-849.
[10]	朱敬先,鲁胜楠,蒋艳芳,姜玲,王健全. 老年肩袖损伤手术患者术前肺功能的影响因素[J]. 北京大学学报（医学版）, 2021, 53(5): 902-906.
[11]	林国中, 马长城, 王振宇, 谢京城, 刘彬, 陈晓东. 颈_1~2硬膜外神经鞘瘤的显微微创治疗[J]. 北京大学学报（医学版）, 2021, 53(3): 586-589.
[12]	李新飞, 彭意吉, 余霄腾, 熊盛炜, 程嗣达, 丁光璞, 杨昆霖, 唐琦, 米悦, 吴静云, 张鹏, 谢家馨, 郝瀚, 王鹤, 邱建星, 杨建, 李学松, 周利群. 肾部分切除术前CT三维可视化评估标准的初步探究[J]. 北京大学学报（医学版）, 2021, 53(3): 613-622.
[13]	杨阳,肖锋,王进,宋波,李西慧,张师杰,何志嵩,张寰,尹玲. 同期手术治疗心脏病和非心脏疾病[J]. 北京大学学报（医学版）, 2021, 53(2): 327-331.
[14]	胡迪,张苗,康惠颖,彭芸. 0~2岁婴幼儿磁共振脑白质模板的建立及验证[J]. 北京大学学报（医学版）, 2021, 53(2): 341-347.
[15]	陈迪,徐翔宇,汪明睿,李芮,臧根奥,张悦,钱浩楠,闫光荣,范田园. 熔融沉积成型3D打印盐酸维拉帕米胃漂浮制剂的制备与体外评价[J]. 北京大学学报（医学版）, 2021, 53(2): 348-354.