Email Alert | RSS

Journal of Peking University(Health Sciences) ›› 2019, Vol. 51 ›› Issue (3): 596-601. doi: 10.19723/j.issn.1671-167X.2019.03.033

Previous Articles     Next Articles

Application of U-shaped convolutional neural network in auto segmentation and reconstruction of 3D prostate model in laparoscopic prostatectomy navigation

Ye YAN1,*,Hai-zhui XIA1,*,Xu-sheng LI2,Wei HE3,Xue-hua ZHU1,Zhi-ying ZHANG1,Chun-lei XIAO1,Yu-qing LIU1,Hua HUANG4,Liang-hua HE2,Jian LU1△()   

  • Received:2019-03-18 Online:2019-05-22 Published:2019-06-26
  • Contact: Ye YAN,Hai-zhui XIA E-mail:lujian@bjmu.edu.cn
  • Supported by:
    Supported by Beijing Natural Science Foundation (L172012), the National Natural Science Foundation of China (61871004), the Fundamental Research Funds for the Central Universities: Peking University Medicine Fund of Fostering Young Scholars’ Scientific & Technological Innovation (BMU2018ZHYL012), the Fundamental Research Funds for the Central Universities: Tongji University (kx0080020173428)

RICH HTML

   

PDF (PC)

Abstract: Objective: To investigate the efficacy of intraoperative cognitive navigation on laparoscopic radical prostatectomy using 3D prostatic models created by U-shaped convolutional neural network (U-net) and reconstructed through Medical Image Interaction Tool Kit (MITK) platform. Methods: A total of 5 000 pieces of prostate cancer magnetic resonance (MR) imaging discovery sets with manual annotations were used to train a modified U-net, and a set of clinically demand-oriented, stable and efficient full convolutional neural network algorithm was constructed. The MR images were cropped and segmented automatically by using modified U-net, and the segmentation data were automatically reconstructed using MITK platform according to our own protocols. The modeling data were output as STL format, and the prostate models were simultaneously displayed on an android tablet during the operation to help achieving cognitive navigation. Results: Based on original U-net architecture, we established a modified U-net from a 201-case MR imaging training set. The network performance was tested and compared with human segmentations and other segmentation networks by using one certain testing data set. Auto segmentation of multi-structures (such as prostate, prostate tumors, seminal vesicles, rectus, neurovascular bundles and dorsal venous complex) were successfully achieved. Secondary automatic 3D reconstruction had been carried out through MITK platform. During the surgery, 3D models of prostatic area were simultaneously displayed on an android tablet, and the cognitive navigation was successfully achieved. Intra-operation organ visualization demonstrated the structural relationships among the key structures in great detail and the degree of tumor invasion was visualized directly. Conclusion: The modified U-net was able to achieve automatic segmentations of important structures of prostate area. Secondary 3D model reconstruction and demonstration could provide intraoperative visualization of vital structures of prostate area, which could help achieve cognitive fusion navigation for surgeons. The application of these techniques could finally reduce positive surgical margin rates, and may improve the efficacy and oncological outcomes of laparoscopic prostatectomy.

Key words: Convolutional neural network, Prostatic neoplasms, Imaging, three-dimensional, Surgery, computer-assisted

CLC Number: 

  • R737.25

Figure 1

Modified U-net architecture"

Figure 2

U-net auto segmentation for prostate with adjacent structures A, original magnetic resonance images; B, human segmentation; C, priority assignment; D, U-net segmentation"

Table 1

Comparison of U-net performance with other segmentation methods"

Group nameWarping errorRand errorPixel error
Human values0.000 0040.001 90.000 9
Zhan[9]0.000 4100.036 70.068 8
U-net0.000 3400.036 00.059 9

Figure 3

T2WI images of demo patients"

Figure 4

Segmentation, reconstruction model and gross specimen A, U-net segmentations on original magnetic resonance images; B, left sagittal view of auto reconstructed 3D model; C, axil view of 3D model (cephalad to caudal); D, gross specimen (cephalad to caudal)"

[1] Siegel RL, Miller KD, Jemal A.Cancer statistics, 2018[J]. CA Cancer J Clin, 2018, 68(1): 7-30.
doi: 10.3322/caac.21442
[2] Simmons MN, Stephenson AJ, Klein EA.Natural history of biochemical recurrence after radical prostatectomy: risk assessment for secondary therapy[J]. Eur Urol, 2007, 51(5): 1175-1184.
doi: 10.1016/j.eururo.2007.01.015
[3] Van den Broeck T, van den Bergh R, Arfi N, et al. Prognostic value of biochemical recurrence following treatment with curative intent for prostate cancer: A systematic review [J/OL]. Eur Urol,(2018-10-17) [2019-02-15]. https://doi.org/10.1016/j.eururo.2018.10.011.
[4] Ukimura O, Aron M, Nakamoto M, et al.Three-dimensional surgical navigation model with TilePro display during robot-assisted radical prostatectomy[J]. J Endourol, 2014, 28(6): 625-630.
doi: 10.1089/end.2013.0749
[5] Hughes-Hallett A, Mayer EK, Marcus HJ, et al.Augmented rea-lity partial nephrectomy: examining the current status and future perspectives[J]. Urology, 2014, 83(2): 266-273.
doi: 10.1016/j.urology.2013.08.049
[6] 王燕, 高旭, 阳青松, 等. 3D打印技术辅助认知融合在前列腺穿刺活检术中的应用[J]. 临床泌尿外科杂志, 2016, 31(2): 104-107.
[7] 邵叶秦, 杨新. 基于随机森林的CT前列腺分割[J]. CT理论与应用研究, 2015, 24(5): 647-655.
[8] Ronneberger O, Fischer P, Brox T.U-net: Convolutional networks for biomedical image segmentation[C]. International Conference on Medical image computing and computer-assisted intervention. Cham: Springer, 2015: 234-241.
[9] 詹曙, 梁植程, 谢栋栋. 前列腺磁共振图像分割的反卷积神经网络方法[J]. 中国图象图形学报, 2017, 22(4): 516-522.
[10] Neher PF, Stieltjes B, Reisert M, et al.MITK global tractography[C]. Proceedings of SPIE: The International Society for Optical Engineering, 2012: 83144D. doi: 10.1117/12.911215.
[11] Lecun Y, Bottou L, Bengio Y, et al.Gradient-based learning applied to document recognition[J]. Proceedings of the IEEE, 1998, 86(11): 2278-2324.
doi: 10.1109/5.726791
[12] Mahapatra D, Buhmann JM.Prostate MRI segmentation using learned semantic knowledge and graph cuts[J]. IEEE Transactions on Biomedical Engineering, 2014, 61(3): 756-764.
doi: 10.1109/TBME.2013.2289306
[13] Korez R, Likar B, Pernuš F, et al.Model-based segmentation of vertebral bodies from MR images with 3D CNNs[C]. International Conference on Medical Image Computing and Computer-Assisted Intervention. Cham: Springer, 2016: 433-441.
[14] Brosch T, Tang LY, Yoo Y, et al.Deep 3D convolutional encoder networks with shortcuts for multiscale feature integration applied to multiple sclerosis lesion segmentation[J]. IEEE Transactions on Medical Imaging, 2016, 35(5): 1229-1239.
doi: 10.1109/TMI.2016.2528821
[15] Martínez F, Romero E, Dréan G, et al.Segmentation of pelvic structures for planning CT using a geometrical shape model tuned by a multi-scale edge detector[J]. Phys Med Biol, 2014, 59(6): 1471-1484.
doi: 10.1088/0031-9155/59/6/1471
[16] 凌彤, 杨琬琪, 杨明. 利用多模态U形网络的CT图像前列腺分割[J]. 智能系统学报, 2018, 13(6): 981-988.
[17] Ebbing J, Jäderling F, Collins JW, et al.Comparison of 3D printed prostate models with standard radiological information to aid understanding of the precise location of prostate cancer: A construct validation study[J]. PLoS One, 2018, 13(6): e199477.
[18] Volonté F, Pugin F, Bucher P, et al.Augmented reality and image overlay navigation with OsiriX in laparoscopic and robotic surgery: not only a matter of fashion[J]. J Hepatobiliary Pancreat Sci, 2011, 18(4): 506-509.
doi: 10.1007/s00534-011-0385-6
[19] Teber D, Guven S, Simpfendorfer T, et al.Augmented reality: a new tool to improve surgical accuracy during laparoscopic partial nephrectomy? Preliminary in vitro and in vivo results[J]. Eur Urol, 2009, 56(2): 332-338.
doi: 10.1016/j.eururo.2009.05.017
[20] Porpiglia F, Fiori C, Checcucci E, et al.Augmented reality robot-assisted radical prostatectomy: Preliminary experience[J]. Urology, 2018, 115(5): 184.
doi: 10.1016/j.urology.2018.01.028
[21] Porpiglia F, Checcucci E, Amparore D, et al.Augmented-reality robot-assisted radical prostatectomy using hyper-accuracy three-dimensional reconstruction (HA 3DTM) technology: a radiological and pathological study[J]. BJU international, 2018, 123(5): 834-845.
[1] Zhaode BU, Mengyu FENG, Ke JI. Practice and reflection on sentinel lymph node navigation surgery for early gastric cancer [J]. Journal of Peking University (Health Sciences), 2026, 58(2): 239-243.
[2] Wen DU, Wenbo ZHANG, Yao YU, Shuo LIU, Huiyu SU, Leihao HU, Zunan TANG, Binzhang WU, Zhen CHEN, Jiaqi LI, Hao WANG, Xin PENG. Exploration and clinical application of the "digital and intelligent surgery" diagnosis and treatment workflow for oral and maxillofacial tumors [J]. Journal of Peking University (Health Sciences), 2026, 58(2): 278-284.
[3] Jingheng WU, Yunhao XUE, Shanlin CHEN, Yintao GUO, Yuntao LIU, Wei ZHANG. Super microsurgical lymphaticovenular anastomosis for limb lymphedema: An outcome analysis based on clinical stage and indocyanine green pattern [J]. Journal of Peking University (Health Sciences), 2026, 58(2): 359-364.
[4] Aonan WEN, Xiaohui ZHANG, Yongtao YANG, Zixiang GAO, Wenbo LI, Shenyao SHAN, Xiangyi SHANG, Yuwen TIAN, Shuwei GUO, Yizhen WANG, Yong WANG, Yijiao ZHAO. Method of constructing 3D facial smile simulation sequence data based on non-rigid registration [J]. Journal of Peking University (Health Sciences), 2026, 58(1): 139-144.
[5] Lu YU, Ling WU, Xiaojing LIU, Zili LI. Feasibility study of a surgical planning protocol for orthognathic surgery utilizing similarity retrieval from database: A randomized controlled trial [J]. Journal of Peking University (Health Sciences), 2026, 58(1): 145-152.
[6] Yuting YANG, Liuyang QU, Danni ZHENG, Xiaotong LING, Xiaoyun XU, Denggao LIU. Demographic characteristic and clinical features in 1 812 patients with salivary gland stones [J]. Journal of Peking University (Health Sciences), 2026, 58(1): 153-159.
[7] Rentao TANG, Liuchang YANG, Jie NIE, Xiaoyan WANG. Microbial communities in extraradicular infections of post-treatment apical periodontitis without or with sinus tracts [J]. Journal of Peking University (Health Sciences), 2026, 58(1): 43-49.
[8] Liang SHAO, Wenjie MA, Ying CHEN, Qian DING, Lei ZHANG. Digital measurement and analysis of anatomical characteristics of protrusive and intercuspal position occlusal contacts in maxillary incisors [J]. Journal of Peking University (Health Sciences), 2026, 58(1): 99-106.
[9] Cuiping WANG, Zhe CHEN, Yongjing CHENG. Correlation study of superb microvascular imaging on knee osteoarthritis [J]. Journal of Peking University (Health Sciences), 2025, 57(6): 1096-1100.
[10] Yanhua LIU, Min LU, Xuyang ZHAO, Kuan'gen ZHANG, Rui WU, Fang MEI, Zhihao DAI, Jiangfeng YOU, Fei PEI. Effect of dephosphorylation of tumor metastasis suppressor gene LASS2 on vacuolar ATPase activity and invasiveness of prostate cancer [J]. Journal of Peking University (Health Sciences), 2025, 57(6): 1113-1123.
[11] Zhemin LI, Jiafu JI, Guoxin LI, Ziyu LI, Zhaode BU, Xiangyu GAO, Di DONG, Lei TANG, Xiaofang XING, Shuqin JIA, Ting GUO, Lianhai ZHANG, Fei SHAN, Xin JI, Anqiang WANG. Development and dissemination of precision medicine approaches in gastric cancer management [J]. Journal of Peking University (Health Sciences), 2025, 57(5): 864-867.
[12] Bowen LI, Qiang ZHANG, Yixin SUN. Establishment and validation of a risk prediction model for scoliosis after Nuss procedure in children and young adults with pectus excavatum [J]. Journal of Peking University (Health Sciences), 2025, 57(5): 941-946.
[13] Xiaoyong YANG, Fan ZHANG, Lulin MA, Cheng LIU. Clinical characteristics and influencing factors of extraglandular invasion of prostatic ductal adenocarcinoma [J]. Journal of Peking University (Health Sciences), 2025, 57(5): 956-960.
[14] Yujia XIAO, Bochun MAO, Yanheng ZHOU. Three-dimensional morphological analysis of posed smile [J]. Journal of Peking University (Health Sciences), 2025, 57(5): 989-995.
[15] Jiaxin NING, Haoran WANG, Shuhang LUO, Jibo JING, Jianye WANG, Huimin HOU, Ming LIU. Multi-omics analysis of the relationship between oxidative stress-related gene and prostate cancer [J]. Journal of Peking University (Health Sciences), 2025, 57(4): 633-643.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
Copyright © Journal of Peking University (Health Sciences), All Rights Reserved.
Powered by Beijing Magtech Co. Ltd