Email Alert | RSS

北京大学学报（医学版） ›› 2019, Vol. 51 ›› Issue (3): 596-601. doi: 10.19723/j.issn.1671-167X.2019.03.033

• • 上一篇    下一篇

基于U型卷积神经网络学习的前列腺癌影像重建模型在手术导航中的应用

颜野1,*,夏海缀1,*,李旭升2,何为3,朱学华1,张智荧1,肖春雷1,刘余庆1,黄华4,何良华2,卢剑1△()   

  • 收稿日期:2019-03-18 出版日期:2019-05-22 发布日期:2019-06-26
  • 通讯作者: 颜野,夏海缀 E-mail:lujian@bjmu.edu.cn
  • 作者简介:卢剑,北京大学第三医院泌尿外科主任医师、副教授、博士生导师。主要学术兼职有:中华医学会泌尿外科分会泌尿男科工程委员会委员、国家药品监督管理局医疗器械分类技术委员会和审评专家咨询委员会委员、中国医师协会超声分子影像与人工智能委员会委员、中国性学会性医学专业委员会委员、中国医药教育协会加速康复外科分会委员和北京医学会肿瘤分会及罕见病分会泌尿专业组委员等。主要研究方向包括肿瘤发病机制与医学图像处理。长期从事泌尿外科临床与基础研究,具备丰富的临床工作经验和较强的医学-信息科学交叉学科理论基础。主持国家重点研发计划数字诊疗专项、国家自然科学基金、北京市自然科学基金等多项国家及省部级科研课题;曾获教育部高等学校科学研究优秀成果奖(科学技术)科技进步二等奖和中华医学科技奖等省部级奖励3项。在The Journal of Urology等发表SCI论文16篇;在《北京大学学报(医学版)》《中华泌尿外科杂志》等国内核心期刊发表论文110篇,参编专业著作4部。
  • 基金资助:
    北京市自然科学基金(L172012)、国家自然科学基金(61871004)、北大医学青年科技创新培育基金(BMU2018ZHYL012)-中央高校基本科研业务费、同济大学“中央高校基本科研业务费”专项基金(kx0080020173428)

Application of U-shaped convolutional neural network in auto segmentation and reconstruction of 3D prostate model in laparoscopic prostatectomy navigation

Ye YAN1,*,Hai-zhui XIA1,*,Xu-sheng LI2,Wei HE3,Xue-hua ZHU1,Zhi-ying ZHANG1,Chun-lei XIAO1,Yu-qing LIU1,Hua HUANG4,Liang-hua HE2,Jian LU1△()   

  • Received:2019-03-18 Online:2019-05-22 Published:2019-06-26
  • Contact: Ye YAN,Hai-zhui XIA E-mail:lujian@bjmu.edu.cn
  • Supported by:
    Supported by Beijing Natural Science Foundation (L172012), the National Natural Science Foundation of China (61871004), the Fundamental Research Funds for the Central Universities: Peking University Medicine Fund of Fostering Young Scholars’ Scientific & Technological Innovation (BMU2018ZHYL012), the Fundamental Research Funds for the Central Universities: Tongji University (kx0080020173428)

RICH HTML

   

PDF (PC)

摘要： 目的 探讨基于U型卷积神经网络(U-shaped convolutional neural network, U-net)建立的前列腺磁共振图像自动化分割和重建3D模型对腹腔镜前列腺癌根治术进行术中认知导航的效果。方法 应用含有人工注释的共5 000张前列腺癌磁共振影像训练集,训练U-net,构建了一套以临床需求为导向,稳定高效的全卷积神经网络算法模型,对前列腺磁共振图像进行区域化、多结构和精细自动化分割,并将分割数据使用医学影像处理交互平台(Medical Image Interaction Tool Kit,MITK)自动重建,以STL格式输出建模信息,应用平板电脑在术中展示前列腺模型,进行认知导航。结果 基于201例前列腺癌患者的磁共振图像训练样本,在经典U-net基础上通过适应性改良,建立了一套结构简单、性能优秀的U-net,可以实现对前列腺、肿瘤、精囊腺、直肠等重要结构的单独分割,并进行三维可视化,直观地显示手术关键部位的结构关系和肿瘤侵犯程度。术中通过平板电脑同步展示3D模型,成功进行认知导航。结论 通过改良的U-net可以自动化完成前列腺磁共振图像的结构化分割,通过重建局部解剖部位的3D模型用于术中认知融合导航,可以达到肿瘤可视化、降低手术切缘阳性率、提高手术效果的作用。

关键词: 卷积神经网络, 前列腺肿瘤, 成像, 三维, 外科手术, 计算机辅助

Abstract: Objective: To investigate the efficacy of intraoperative cognitive navigation on laparoscopic radical prostatectomy using 3D prostatic models created by U-shaped convolutional neural network (U-net) and reconstructed through Medical Image Interaction Tool Kit (MITK) platform. Methods: A total of 5 000 pieces of prostate cancer magnetic resonance (MR) imaging discovery sets with manual annotations were used to train a modified U-net, and a set of clinically demand-oriented, stable and efficient full convolutional neural network algorithm was constructed. The MR images were cropped and segmented automatically by using modified U-net, and the segmentation data were automatically reconstructed using MITK platform according to our own protocols. The modeling data were output as STL format, and the prostate models were simultaneously displayed on an android tablet during the operation to help achieving cognitive navigation. Results: Based on original U-net architecture, we established a modified U-net from a 201-case MR imaging training set. The network performance was tested and compared with human segmentations and other segmentation networks by using one certain testing data set. Auto segmentation of multi-structures (such as prostate, prostate tumors, seminal vesicles, rectus, neurovascular bundles and dorsal venous complex) were successfully achieved. Secondary automatic 3D reconstruction had been carried out through MITK platform. During the surgery, 3D models of prostatic area were simultaneously displayed on an android tablet, and the cognitive navigation was successfully achieved. Intra-operation organ visualization demonstrated the structural relationships among the key structures in great detail and the degree of tumor invasion was visualized directly. Conclusion: The modified U-net was able to achieve automatic segmentations of important structures of prostate area. Secondary 3D model reconstruction and demonstration could provide intraoperative visualization of vital structures of prostate area, which could help achieve cognitive fusion navigation for surgeons. The application of these techniques could finally reduce positive surgical margin rates, and may improve the efficacy and oncological outcomes of laparoscopic prostatectomy.

Key words: Convolutional neural network, Prostatic neoplasms, Imaging, three-dimensional, Surgery, computer-assisted

中图分类号: 

  • R737.25

图1

改良U-net的架构"

图2

U-net进行前列腺及相邻结构分割示例"

表1

U-net性能与其他分割效果对比"

Group nameWarping errorRand errorPixel error
Human values0.000 0040.001 90.000 9
Zhan[9]0.000 4100.036 70.068 8
U-net0.000 3400.036 00.059 9

图3

患者前列腺磁共振T2WI图像"

图4

分割重建实体标本"

[1] Siegel RL, Miller KD, Jemal A.Cancer statistics, 2018[J]. CA Cancer J Clin, 2018, 68(1): 7-30.
doi: 10.3322/caac.21442
[2] Simmons MN, Stephenson AJ, Klein EA.Natural history of biochemical recurrence after radical prostatectomy: risk assessment for secondary therapy[J]. Eur Urol, 2007, 51(5): 1175-1184.
doi: 10.1016/j.eururo.2007.01.015
[3] Van den Broeck T, van den Bergh R, Arfi N, et al. Prognostic value of biochemical recurrence following treatment with curative intent for prostate cancer: A systematic review [J/OL]. Eur Urol,(2018-10-17) [2019-02-15]. https://doi.org/10.1016/j.eururo.2018.10.011.
[4] Ukimura O, Aron M, Nakamoto M, et al.Three-dimensional surgical navigation model with TilePro display during robot-assisted radical prostatectomy[J]. J Endourol, 2014, 28(6): 625-630.
doi: 10.1089/end.2013.0749
[5] Hughes-Hallett A, Mayer EK, Marcus HJ, et al.Augmented rea-lity partial nephrectomy: examining the current status and future perspectives[J]. Urology, 2014, 83(2): 266-273.
doi: 10.1016/j.urology.2013.08.049
[6] 王燕, 高旭, 阳青松, 等. 3D打印技术辅助认知融合在前列腺穿刺活检术中的应用[J]. 临床泌尿外科杂志, 2016, 31(2): 104-107.
[7] 邵叶秦, 杨新. 基于随机森林的CT前列腺分割[J]. CT理论与应用研究, 2015, 24(5): 647-655.
[8] Ronneberger O, Fischer P, Brox T.U-net: Convolutional networks for biomedical image segmentation[C]. International Conference on Medical image computing and computer-assisted intervention. Cham: Springer, 2015: 234-241.
[9] 詹曙, 梁植程, 谢栋栋. 前列腺磁共振图像分割的反卷积神经网络方法[J]. 中国图象图形学报, 2017, 22(4): 516-522.
[10] Neher PF, Stieltjes B, Reisert M, et al.MITK global tractography[C]. Proceedings of SPIE: The International Society for Optical Engineering, 2012: 83144D. doi: 10.1117/12.911215.
[11] Lecun Y, Bottou L, Bengio Y, et al.Gradient-based learning applied to document recognition[J]. Proceedings of the IEEE, 1998, 86(11): 2278-2324.
doi: 10.1109/5.726791
[12] Mahapatra D, Buhmann JM.Prostate MRI segmentation using learned semantic knowledge and graph cuts[J]. IEEE Transactions on Biomedical Engineering, 2014, 61(3): 756-764.
doi: 10.1109/TBME.2013.2289306
[13] Korez R, Likar B, Pernuš F, et al.Model-based segmentation of vertebral bodies from MR images with 3D CNNs[C]. International Conference on Medical Image Computing and Computer-Assisted Intervention. Cham: Springer, 2016: 433-441.
[14] Brosch T, Tang LY, Yoo Y, et al.Deep 3D convolutional encoder networks with shortcuts for multiscale feature integration applied to multiple sclerosis lesion segmentation[J]. IEEE Transactions on Medical Imaging, 2016, 35(5): 1229-1239.
doi: 10.1109/TMI.2016.2528821
[15] Martínez F, Romero E, Dréan G, et al.Segmentation of pelvic structures for planning CT using a geometrical shape model tuned by a multi-scale edge detector[J]. Phys Med Biol, 2014, 59(6): 1471-1484.
doi: 10.1088/0031-9155/59/6/1471
[16] 凌彤, 杨琬琪, 杨明. 利用多模态U形网络的CT图像前列腺分割[J]. 智能系统学报, 2018, 13(6): 981-988.
[17] Ebbing J, Jäderling F, Collins JW, et al.Comparison of 3D printed prostate models with standard radiological information to aid understanding of the precise location of prostate cancer: A construct validation study[J]. PLoS One, 2018, 13(6): e199477.
[18] Volonté F, Pugin F, Bucher P, et al.Augmented reality and image overlay navigation with OsiriX in laparoscopic and robotic surgery: not only a matter of fashion[J]. J Hepatobiliary Pancreat Sci, 2011, 18(4): 506-509.
doi: 10.1007/s00534-011-0385-6
[19] Teber D, Guven S, Simpfendorfer T, et al.Augmented reality: a new tool to improve surgical accuracy during laparoscopic partial nephrectomy? Preliminary in vitro and in vivo results[J]. Eur Urol, 2009, 56(2): 332-338.
doi: 10.1016/j.eururo.2009.05.017
[20] Porpiglia F, Fiori C, Checcucci E, et al.Augmented reality robot-assisted radical prostatectomy: Preliminary experience[J]. Urology, 2018, 115(5): 184.
doi: 10.1016/j.urology.2018.01.028
[21] Porpiglia F, Checcucci E, Amparore D, et al.Augmented-reality robot-assisted radical prostatectomy using hyper-accuracy three-dimensional reconstruction (HA 3DTM) technology: a radiological and pathological study[J]. BJU international, 2018, 123(5): 834-845.
[1] 李志存, 吴天俣, 梁磊, 范宇, 孟一森, 张骞. 穿刺活检单针阳性前列腺癌术后病理升级的危险因素分析及列线图模型构建[J]. 北京大学学报（医学版）, 2024, 56(5): 896-901.
[2] 邢念增,王明帅,杨飞亚,尹路,韩苏军. 前列腺免活检创新理念的临床实践及其应用前景[J]. 北京大学学报（医学版）, 2024, 56(4): 565-566.
[3] 田宇轩,阮明健,刘毅,李德润,吴静云,沈棋,范宇,金杰. 双参数MRI改良PI-RADS评分4分和5分病灶的最大径对临床有意义前列腺癌的预测效果[J]. 北京大学学报（医学版）, 2024, 56(4): 567-574.
[4] 姚凯烽,阮明健,李德润,田宇轩,陈宇珂,范宇,刘毅. 靶向穿刺联合区域系统穿刺对PI-RADS 4~5分患者的前列腺癌诊断效能[J]. 北京大学学报（医学版）, 2024, 56(4): 575-581.
[5] 欧俊永,倪坤明,马潞林,王国良,颜野,杨斌,李庚午,宋昊东,陆敏,叶剑飞,张树栋. 肌层浸润性膀胱癌合并中高危前列腺癌患者的预后因素[J]. 北京大学学报（医学版）, 2024, 56(4): 582-588.
[6] 王明瑞,刘军,熊六林,于路平,胡浩,许克新,徐涛. 经皮微通道-微电子肾镜-微超声探针碎石术治疗1.5~2.5 cm肾结石的疗效和安全性[J]. 北京大学学报（医学版）, 2024, 56(4): 605-609.
[7] 应沂岑,杜毅聪,李志华,张一鸣,李新飞,王冰,张鹏,朱宏建,周利群,杨昆霖,李学松. 机器人辅助腹腔镜下颊黏膜补片输尿管成形术治疗复杂输尿管狭窄[J]. 北京大学学报（医学版）, 2024, 56(4): 640-645.
[8] 薄士仕,高承志. 基于卷积神经网络实现锥形束CT牙齿分割及牙位标定[J]. 北京大学学报（医学版）, 2024, 56(4): 735-740.
[9] 唐祖南,胡耒豪,陈震,于尧,章文博,彭歆. 增强现实技术在口腔颌面颈部解剖识别中的应用评价[J]. 北京大学学报（医学版）, 2024, 56(3): 541-545.
[10] 吕梁,张铭津,温奧楠,赵一姣,王勇,李晶,杨庚辰,柳大为. 应用三维软组织空间线角模板法评价颏部对称性[J]. 北京大学学报（医学版）, 2024, 56(1): 106-110.
[11] 毛渤淳,田雅婧,王雪东,李晶,周彦恒. 骨性Ⅱ类高角患者拔牙矫治前后的面部软硬组织变化[J]. 北京大学学报（医学版）, 2024, 56(1): 111-119.
[12] 凌晓彤,屈留洋,郑丹妮,杨静,闫雪冰,柳登高,高岩. 牙源性钙化囊肿与牙源性钙化上皮瘤的三维影像特点[J]. 北京大学学报（医学版）, 2024, 56(1): 131-137.
[13] 徐心雨,吴灵,宋凤岐,李自力,张益,刘筱菁. 基于下颌运动轨迹的正颌外科术中下颌骨髁突定位方法及初步精度验证[J]. 北京大学学报（医学版）, 2024, 56(1): 57-65.
[14] 蔡安东,王晓霞,周文娟,柳忠豪. 下颌前突畸形患者上颌骨及髁突虚拟位置与术后现实位置的比较[J]. 北京大学学报（医学版）, 2024, 56(1): 74-80.
[15] 李穗,马雯洁,王时敏,丁茜,孙瑶,张磊. 上前牙种植单冠修复体切导的数字化设计正确度[J]. 北京大学学报（医学版）, 2024, 56(1): 81-87.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
ISSN 1671-167X
CN 11-4691/R
主管单位：中华人民共和国教育部
主办单位：北京大学
地址： 北京市海淀区学院路38号 北京大学医学部 《北京大学学报（医学版）》
邮编：100191
电话：010-82801551
Email： xbbjb2@bjmu.edu.cn

《北京大学学报（医学版）》
微信公众号
版权所有 © 2018 《北京大学学报（医学版）》编辑部