Email Alert | RSS

北京大学学报（医学版） ›› 2019, Vol. 51 ›› Issue (3): 596-601. doi: 10.19723/j.issn.1671-167X.2019.03.033

• • 上一篇    下一篇

基于U型卷积神经网络学习的前列腺癌影像重建模型在手术导航中的应用

颜野1,*,夏海缀1,*,李旭升2,何为3,朱学华1,张智荧1,肖春雷1,刘余庆1,黄华4,何良华2,卢剑1△()   

  • 收稿日期:2019-03-18 出版日期:2019-05-22 发布日期:2019-06-26
  • 通讯作者: 颜野,夏海缀 E-mail:lujian@bjmu.edu.cn
  • 作者简介:卢剑,北京大学第三医院泌尿外科主任医师、副教授、博士生导师。主要学术兼职有:中华医学会泌尿外科分会泌尿男科工程委员会委员、国家药品监督管理局医疗器械分类技术委员会和审评专家咨询委员会委员、中国医师协会超声分子影像与人工智能委员会委员、中国性学会性医学专业委员会委员、中国医药教育协会加速康复外科分会委员和北京医学会肿瘤分会及罕见病分会泌尿专业组委员等。主要研究方向包括肿瘤发病机制与医学图像处理。长期从事泌尿外科临床与基础研究,具备丰富的临床工作经验和较强的医学-信息科学交叉学科理论基础。主持国家重点研发计划数字诊疗专项、国家自然科学基金、北京市自然科学基金等多项国家及省部级科研课题;曾获教育部高等学校科学研究优秀成果奖(科学技术)科技进步二等奖和中华医学科技奖等省部级奖励3项。在The Journal of Urology等发表SCI论文16篇;在《北京大学学报(医学版)》《中华泌尿外科杂志》等国内核心期刊发表论文110篇,参编专业著作4部。
  • 基金资助:
    北京市自然科学基金(L172012)、国家自然科学基金(61871004)、北大医学青年科技创新培育基金(BMU2018ZHYL012)-中央高校基本科研业务费、同济大学“中央高校基本科研业务费”专项基金(kx0080020173428)

Application of U-shaped convolutional neural network in auto segmentation and reconstruction of 3D prostate model in laparoscopic prostatectomy navigation

Ye YAN1,*,Hai-zhui XIA1,*,Xu-sheng LI2,Wei HE3,Xue-hua ZHU1,Zhi-ying ZHANG1,Chun-lei XIAO1,Yu-qing LIU1,Hua HUANG4,Liang-hua HE2,Jian LU1△()   

  • Received:2019-03-18 Online:2019-05-22 Published:2019-06-26
  • Contact: Ye YAN,Hai-zhui XIA E-mail:lujian@bjmu.edu.cn
  • Supported by:
    Supported by Beijing Natural Science Foundation (L172012), the National Natural Science Foundation of China (61871004), the Fundamental Research Funds for the Central Universities: Peking University Medicine Fund of Fostering Young Scholars’ Scientific & Technological Innovation (BMU2018ZHYL012), the Fundamental Research Funds for the Central Universities: Tongji University (kx0080020173428)

RICH HTML

   

PDF (PC)

摘要： 目的 探讨基于U型卷积神经网络(U-shaped convolutional neural network, U-net)建立的前列腺磁共振图像自动化分割和重建3D模型对腹腔镜前列腺癌根治术进行术中认知导航的效果。方法 应用含有人工注释的共5 000张前列腺癌磁共振影像训练集,训练U-net,构建了一套以临床需求为导向,稳定高效的全卷积神经网络算法模型,对前列腺磁共振图像进行区域化、多结构和精细自动化分割,并将分割数据使用医学影像处理交互平台(Medical Image Interaction Tool Kit,MITK)自动重建,以STL格式输出建模信息,应用平板电脑在术中展示前列腺模型,进行认知导航。结果 基于201例前列腺癌患者的磁共振图像训练样本,在经典U-net基础上通过适应性改良,建立了一套结构简单、性能优秀的U-net,可以实现对前列腺、肿瘤、精囊腺、直肠等重要结构的单独分割,并进行三维可视化,直观地显示手术关键部位的结构关系和肿瘤侵犯程度。术中通过平板电脑同步展示3D模型,成功进行认知导航。结论 通过改良的U-net可以自动化完成前列腺磁共振图像的结构化分割,通过重建局部解剖部位的3D模型用于术中认知融合导航,可以达到肿瘤可视化、降低手术切缘阳性率、提高手术效果的作用。

关键词: 卷积神经网络, 前列腺肿瘤, 成像, 三维, 外科手术, 计算机辅助

Abstract: Objective: To investigate the efficacy of intraoperative cognitive navigation on laparoscopic radical prostatectomy using 3D prostatic models created by U-shaped convolutional neural network (U-net) and reconstructed through Medical Image Interaction Tool Kit (MITK) platform. Methods: A total of 5 000 pieces of prostate cancer magnetic resonance (MR) imaging discovery sets with manual annotations were used to train a modified U-net, and a set of clinically demand-oriented, stable and efficient full convolutional neural network algorithm was constructed. The MR images were cropped and segmented automatically by using modified U-net, and the segmentation data were automatically reconstructed using MITK platform according to our own protocols. The modeling data were output as STL format, and the prostate models were simultaneously displayed on an android tablet during the operation to help achieving cognitive navigation. Results: Based on original U-net architecture, we established a modified U-net from a 201-case MR imaging training set. The network performance was tested and compared with human segmentations and other segmentation networks by using one certain testing data set. Auto segmentation of multi-structures (such as prostate, prostate tumors, seminal vesicles, rectus, neurovascular bundles and dorsal venous complex) were successfully achieved. Secondary automatic 3D reconstruction had been carried out through MITK platform. During the surgery, 3D models of prostatic area were simultaneously displayed on an android tablet, and the cognitive navigation was successfully achieved. Intra-operation organ visualization demonstrated the structural relationships among the key structures in great detail and the degree of tumor invasion was visualized directly. Conclusion: The modified U-net was able to achieve automatic segmentations of important structures of prostate area. Secondary 3D model reconstruction and demonstration could provide intraoperative visualization of vital structures of prostate area, which could help achieve cognitive fusion navigation for surgeons. The application of these techniques could finally reduce positive surgical margin rates, and may improve the efficacy and oncological outcomes of laparoscopic prostatectomy.

Key words: Convolutional neural network, Prostatic neoplasms, Imaging, three-dimensional, Surgery, computer-assisted

中图分类号: 

  • R737.25

图1

改良U-net的架构"

图2

U-net进行前列腺及相邻结构分割示例"

表1

U-net性能与其他分割效果对比"

Group nameWarping errorRand errorPixel error
Human values0.000 0040.001 90.000 9
Zhan[9]0.000 4100.036 70.068 8
U-net0.000 3400.036 00.059 9

图3

患者前列腺磁共振T2WI图像"

图4

分割重建实体标本"

[1] Siegel RL, Miller KD, Jemal A.Cancer statistics, 2018[J]. CA Cancer J Clin, 2018, 68(1): 7-30.
doi: 10.3322/caac.21442
[2] Simmons MN, Stephenson AJ, Klein EA.Natural history of biochemical recurrence after radical prostatectomy: risk assessment for secondary therapy[J]. Eur Urol, 2007, 51(5): 1175-1184.
doi: 10.1016/j.eururo.2007.01.015
[3] Van den Broeck T, van den Bergh R, Arfi N, et al. Prognostic value of biochemical recurrence following treatment with curative intent for prostate cancer: A systematic review [J/OL]. Eur Urol,(2018-10-17) [2019-02-15]. https://doi.org/10.1016/j.eururo.2018.10.011.
[4] Ukimura O, Aron M, Nakamoto M, et al.Three-dimensional surgical navigation model with TilePro display during robot-assisted radical prostatectomy[J]. J Endourol, 2014, 28(6): 625-630.
doi: 10.1089/end.2013.0749
[5] Hughes-Hallett A, Mayer EK, Marcus HJ, et al.Augmented rea-lity partial nephrectomy: examining the current status and future perspectives[J]. Urology, 2014, 83(2): 266-273.
doi: 10.1016/j.urology.2013.08.049
[6] 王燕, 高旭, 阳青松, 等. 3D打印技术辅助认知融合在前列腺穿刺活检术中的应用[J]. 临床泌尿外科杂志, 2016, 31(2): 104-107.
[7] 邵叶秦, 杨新. 基于随机森林的CT前列腺分割[J]. CT理论与应用研究, 2015, 24(5): 647-655.
[8] Ronneberger O, Fischer P, Brox T.U-net: Convolutional networks for biomedical image segmentation[C]. International Conference on Medical image computing and computer-assisted intervention. Cham: Springer, 2015: 234-241.
[9] 詹曙, 梁植程, 谢栋栋. 前列腺磁共振图像分割的反卷积神经网络方法[J]. 中国图象图形学报, 2017, 22(4): 516-522.
[10] Neher PF, Stieltjes B, Reisert M, et al.MITK global tractography[C]. Proceedings of SPIE: The International Society for Optical Engineering, 2012: 83144D. doi: 10.1117/12.911215.
[11] Lecun Y, Bottou L, Bengio Y, et al.Gradient-based learning applied to document recognition[J]. Proceedings of the IEEE, 1998, 86(11): 2278-2324.
doi: 10.1109/5.726791
[12] Mahapatra D, Buhmann JM.Prostate MRI segmentation using learned semantic knowledge and graph cuts[J]. IEEE Transactions on Biomedical Engineering, 2014, 61(3): 756-764.
doi: 10.1109/TBME.2013.2289306
[13] Korez R, Likar B, Pernuš F, et al.Model-based segmentation of vertebral bodies from MR images with 3D CNNs[C]. International Conference on Medical Image Computing and Computer-Assisted Intervention. Cham: Springer, 2016: 433-441.
[14] Brosch T, Tang LY, Yoo Y, et al.Deep 3D convolutional encoder networks with shortcuts for multiscale feature integration applied to multiple sclerosis lesion segmentation[J]. IEEE Transactions on Medical Imaging, 2016, 35(5): 1229-1239.
doi: 10.1109/TMI.2016.2528821
[15] Martínez F, Romero E, Dréan G, et al.Segmentation of pelvic structures for planning CT using a geometrical shape model tuned by a multi-scale edge detector[J]. Phys Med Biol, 2014, 59(6): 1471-1484.
doi: 10.1088/0031-9155/59/6/1471
[16] 凌彤, 杨琬琪, 杨明. 利用多模态U形网络的CT图像前列腺分割[J]. 智能系统学报, 2018, 13(6): 981-988.
[17] Ebbing J, Jäderling F, Collins JW, et al.Comparison of 3D printed prostate models with standard radiological information to aid understanding of the precise location of prostate cancer: A construct validation study[J]. PLoS One, 2018, 13(6): e199477.
[18] Volonté F, Pugin F, Bucher P, et al.Augmented reality and image overlay navigation with OsiriX in laparoscopic and robotic surgery: not only a matter of fashion[J]. J Hepatobiliary Pancreat Sci, 2011, 18(4): 506-509.
doi: 10.1007/s00534-011-0385-6
[19] Teber D, Guven S, Simpfendorfer T, et al.Augmented reality: a new tool to improve surgical accuracy during laparoscopic partial nephrectomy? Preliminary in vitro and in vivo results[J]. Eur Urol, 2009, 56(2): 332-338.
doi: 10.1016/j.eururo.2009.05.017
[20] Porpiglia F, Fiori C, Checcucci E, et al.Augmented reality robot-assisted radical prostatectomy: Preliminary experience[J]. Urology, 2018, 115(5): 184.
doi: 10.1016/j.urology.2018.01.028
[21] Porpiglia F, Checcucci E, Amparore D, et al.Augmented-reality robot-assisted radical prostatectomy using hyper-accuracy three-dimensional reconstruction (HA 3DTM) technology: a radiological and pathological study[J]. BJU international, 2018, 123(5): 834-845.
[1] FarinEbrahimi, 冯志强, FarazEbrahimi, 韩玮华, 于子杨, 贾宽宽, 安金刚. 上颌药物相关性颌骨坏死的不同分期手术治疗效果[J]. 北京大学学报（医学版）, 2026, 58(1): 107-114.
[2] 温奥楠, 张晓会, 杨咏涛, 高梓翔, 李文博, 单珅瑶, 商相宜, 田淯文, 郭殊玮, 王艺蓁, 王勇, 赵一姣. 基于非刚性配准构建三维颜面微笑仿真序列数据的方法[J]. 北京大学学报（医学版）, 2026, 58(1): 139-144.
[3] 于录, 吴灵, 刘筱菁, 李自力. 基于数据库相似性检索的正颌外科手术规划技术流程可行性研究: 随机对照试验[J]. 北京大学学报（医学版）, 2026, 58(1): 145-152.
[4] 邵梁, 马雯洁, 陈莹, 丁茜, 张磊. 上颌切牙前伸和正中咬合接触解剖特征的数字化测量与分析[J]. 北京大学学报（医学版）, 2026, 58(1): 99-106.
[5] 王翠萍, 陈哲, 程永静. 超微血流成像评估与膝骨关节炎临床症状的关联研究[J]. 北京大学学报（医学版）, 2025, 57(6): 1096-1100.
[6] 刘艳华, 陆敏, 赵旭阳, 张宽根, 武睿, 梅放, 戴志豪, 由江峰, 裴斐. 肿瘤转移抑制基因LASS2去磷酸化对液泡型ATP酶活性及前列腺癌侵袭性的影响[J]. 北京大学学报（医学版）, 2025, 57(6): 1113-1123.
[7] 杨小勇, 张帆, 马潞林, 刘承. 前列腺导管腺癌临床特征及腺外侵犯的影响因素[J]. 北京大学学报（医学版）, 2025, 57(5): 956-960.
[8] 宋凤岐, 徐心雨, 刘筱菁, 李自力. 上颌骨前部和整体顺时针旋转改善骨性Ⅲ类牙颌面畸形患者鼻旁凹陷的对比[J]. 北京大学学报（医学版）, 2025, 57(5): 980-988.
[9] 肖宇嘉, 毛渤淳, 周彦恒. 姿势性微笑的三维形态学研究[J]. 北京大学学报（医学版）, 2025, 57(5): 989-995.
[10] 宁家昕, 王浩然, 罗书航, 敬吉波, 王建业, 侯惠民, 刘明. 氧化应激相关基因与前列腺癌关系的多组学分析[J]. 北京大学学报（医学版）, 2025, 57(4): 633-643.
[11] 王泽远, 于栓宝, 郑浩轲, 陶金, 范雅峰, 张雪培. 基于临床特征和多参数MRI的前列腺癌盆腔淋巴结转移的术前预测模型[J]. 北京大学学报（医学版）, 2025, 57(4): 684-691.
[12] 宁圆, 张晓盈, 李雪, 李原, 何菁, 金月波. 干燥综合征并发乳腺淋巴瘤1例[J]. 北京大学学报（医学版）, 2025, 57(4): 808-811.
[13] 孙建军, 马千权, 尹晓亮, 杨辰龙, 张嘉, 陈素华, 吴超, 谢京城, 韩芸峰, 林国中, 司雨, 杨军, 邬海博, 赵强. 任意维度重建磁共振对骶管囊肿进行精准分型对于指导微创手术和康复的意义[J]. 北京大学学报（医学版）, 2025, 57(2): 303-308.
[14] 仇师禹, 练洋, 康一帆, 张雷, 蔡义望, 单小峰, 蔡志刚. 基于下颌骨数据库和全连接神经网络的三维检索模型辅助下的下颌骨个性化重建[J]. 北京大学学报（医学版）, 2025, 57(2): 360-368.
[15] 马丽娟, 腾雍辉, 王勇, 赵一姣, 张馨月, 秦庆钊, 尹东. 乳牙缺失数字化丝圈间隙保持器的三维有限元分析[J]. 北京大学学报（医学版）, 2025, 57(2): 376-383.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
ISSN 1671-167X
CN 11-4691/R
主管单位：中华人民共和国教育部
主办单位：北京大学
地址： 北京市海淀区学院路38号 北京大学医学部 《北京大学学报（医学版）》
邮编：100191
电话：010-82801551
Email： xbbjb2@bjmu.edu.cn

《北京大学学报（医学版）》
微信公众号
版权所有 © 2018 《北京大学学报（医学版）》编辑部