Email Alert | RSS

Journal of Peking University(Health Sciences) ›› 2019, Vol. 51 ›› Issue (3): 596-601. doi: 10.19723/j.issn.1671-167X.2019.03.033

Previous Articles     Next Articles

Application of U-shaped convolutional neural network in auto segmentation and reconstruction of 3D prostate model in laparoscopic prostatectomy navigation

Ye YAN1,*,Hai-zhui XIA1,*,Xu-sheng LI2,Wei HE3,Xue-hua ZHU1,Zhi-ying ZHANG1,Chun-lei XIAO1,Yu-qing LIU1,Hua HUANG4,Liang-hua HE2,Jian LU1△()   

  • Received:2019-03-18 Online:2019-05-22 Published:2019-06-26
  • Contact: Ye YAN,Hai-zhui XIA E-mail:lujian@bjmu.edu.cn
  • Supported by:
    Supported by Beijing Natural Science Foundation (L172012), the National Natural Science Foundation of China (61871004), the Fundamental Research Funds for the Central Universities: Peking University Medicine Fund of Fostering Young Scholars’ Scientific & Technological Innovation (BMU2018ZHYL012), the Fundamental Research Funds for the Central Universities: Tongji University (kx0080020173428)

RICH HTML

 10   

PDF (PC)

255

Abstract: Objective: To investigate the efficacy of intraoperative cognitive navigation on laparoscopic radical prostatectomy using 3D prostatic models created by U-shaped convolutional neural network (U-net) and reconstructed through Medical Image Interaction Tool Kit (MITK) platform. Methods: A total of 5 000 pieces of prostate cancer magnetic resonance (MR) imaging discovery sets with manual annotations were used to train a modified U-net, and a set of clinically demand-oriented, stable and efficient full convolutional neural network algorithm was constructed. The MR images were cropped and segmented automatically by using modified U-net, and the segmentation data were automatically reconstructed using MITK platform according to our own protocols. The modeling data were output as STL format, and the prostate models were simultaneously displayed on an android tablet during the operation to help achieving cognitive navigation. Results: Based on original U-net architecture, we established a modified U-net from a 201-case MR imaging training set. The network performance was tested and compared with human segmentations and other segmentation networks by using one certain testing data set. Auto segmentation of multi-structures (such as prostate, prostate tumors, seminal vesicles, rectus, neurovascular bundles and dorsal venous complex) were successfully achieved. Secondary automatic 3D reconstruction had been carried out through MITK platform. During the surgery, 3D models of prostatic area were simultaneously displayed on an android tablet, and the cognitive navigation was successfully achieved. Intra-operation organ visualization demonstrated the structural relationships among the key structures in great detail and the degree of tumor invasion was visualized directly. Conclusion: The modified U-net was able to achieve automatic segmentations of important structures of prostate area. Secondary 3D model reconstruction and demonstration could provide intraoperative visualization of vital structures of prostate area, which could help achieve cognitive fusion navigation for surgeons. The application of these techniques could finally reduce positive surgical margin rates, and may improve the efficacy and oncological outcomes of laparoscopic prostatectomy.

Key words: Convolutional neural network, Prostatic neoplasms, Imaging, three-dimensional, Surgery, computer-assisted

CLC Number: 

  • R737.25

Figure 1

Modified U-net architecture"

Figure 2

U-net auto segmentation for prostate with adjacent structures A, original magnetic resonance images; B, human segmentation; C, priority assignment; D, U-net segmentation"

Table 1

Comparison of U-net performance with other segmentation methods"

Group nameWarping errorRand errorPixel error
Human values0.000 0040.001 90.000 9
Zhan[9]0.000 4100.036 70.068 8
U-net0.000 3400.036 00.059 9

Figure 3

T2WI images of demo patients"

Figure 4

Segmentation, reconstruction model and gross specimen A, U-net segmentations on original magnetic resonance images; B, left sagittal view of auto reconstructed 3D model; C, axil view of 3D model (cephalad to caudal); D, gross specimen (cephalad to caudal)"

[1] Siegel RL, Miller KD, Jemal A.Cancer statistics, 2018[J]. CA Cancer J Clin, 2018, 68(1): 7-30.
doi: 10.3322/caac.21442
[2] Simmons MN, Stephenson AJ, Klein EA.Natural history of biochemical recurrence after radical prostatectomy: risk assessment for secondary therapy[J]. Eur Urol, 2007, 51(5): 1175-1184.
doi: 10.1016/j.eururo.2007.01.015
[3] Van den Broeck T, van den Bergh R, Arfi N, et al. Prognostic value of biochemical recurrence following treatment with curative intent for prostate cancer: A systematic review [J/OL]. Eur Urol,(2018-10-17) [2019-02-15]. https://doi.org/10.1016/j.eururo.2018.10.011.
[4] Ukimura O, Aron M, Nakamoto M, et al.Three-dimensional surgical navigation model with TilePro display during robot-assisted radical prostatectomy[J]. J Endourol, 2014, 28(6): 625-630.
doi: 10.1089/end.2013.0749
[5] Hughes-Hallett A, Mayer EK, Marcus HJ, et al.Augmented rea-lity partial nephrectomy: examining the current status and future perspectives[J]. Urology, 2014, 83(2): 266-273.
doi: 10.1016/j.urology.2013.08.049
[6] 王燕, 高旭, 阳青松, 等. 3D打印技术辅助认知融合在前列腺穿刺活检术中的应用[J]. 临床泌尿外科杂志, 2016, 31(2): 104-107.
[7] 邵叶秦, 杨新. 基于随机森林的CT前列腺分割[J]. CT理论与应用研究, 2015, 24(5): 647-655.
[8] Ronneberger O, Fischer P, Brox T.U-net: Convolutional networks for biomedical image segmentation[C]. International Conference on Medical image computing and computer-assisted intervention. Cham: Springer, 2015: 234-241.
[9] 詹曙, 梁植程, 谢栋栋. 前列腺磁共振图像分割的反卷积神经网络方法[J]. 中国图象图形学报, 2017, 22(4): 516-522.
[10] Neher PF, Stieltjes B, Reisert M, et al.MITK global tractography[C]. Proceedings of SPIE: The International Society for Optical Engineering, 2012: 83144D. doi: 10.1117/12.911215.
[11] Lecun Y, Bottou L, Bengio Y, et al.Gradient-based learning applied to document recognition[J]. Proceedings of the IEEE, 1998, 86(11): 2278-2324.
doi: 10.1109/5.726791
[12] Mahapatra D, Buhmann JM.Prostate MRI segmentation using learned semantic knowledge and graph cuts[J]. IEEE Transactions on Biomedical Engineering, 2014, 61(3): 756-764.
doi: 10.1109/TBME.2013.2289306
[13] Korez R, Likar B, Pernuš F, et al.Model-based segmentation of vertebral bodies from MR images with 3D CNNs[C]. International Conference on Medical Image Computing and Computer-Assisted Intervention. Cham: Springer, 2016: 433-441.
[14] Brosch T, Tang LY, Yoo Y, et al.Deep 3D convolutional encoder networks with shortcuts for multiscale feature integration applied to multiple sclerosis lesion segmentation[J]. IEEE Transactions on Medical Imaging, 2016, 35(5): 1229-1239.
doi: 10.1109/TMI.2016.2528821
[15] Martínez F, Romero E, Dréan G, et al.Segmentation of pelvic structures for planning CT using a geometrical shape model tuned by a multi-scale edge detector[J]. Phys Med Biol, 2014, 59(6): 1471-1484.
doi: 10.1088/0031-9155/59/6/1471
[16] 凌彤, 杨琬琪, 杨明. 利用多模态U形网络的CT图像前列腺分割[J]. 智能系统学报, 2018, 13(6): 981-988.
[17] Ebbing J, Jäderling F, Collins JW, et al.Comparison of 3D printed prostate models with standard radiological information to aid understanding of the precise location of prostate cancer: A construct validation study[J]. PLoS One, 2018, 13(6): e199477.
[18] Volonté F, Pugin F, Bucher P, et al.Augmented reality and image overlay navigation with OsiriX in laparoscopic and robotic surgery: not only a matter of fashion[J]. J Hepatobiliary Pancreat Sci, 2011, 18(4): 506-509.
doi: 10.1007/s00534-011-0385-6
[19] Teber D, Guven S, Simpfendorfer T, et al.Augmented reality: a new tool to improve surgical accuracy during laparoscopic partial nephrectomy? Preliminary in vitro and in vivo results[J]. Eur Urol, 2009, 56(2): 332-338.
doi: 10.1016/j.eururo.2009.05.017
[20] Porpiglia F, Fiori C, Checcucci E, et al.Augmented reality robot-assisted radical prostatectomy: Preliminary experience[J]. Urology, 2018, 115(5): 184.
doi: 10.1016/j.urology.2018.01.028
[21] Porpiglia F, Checcucci E, Amparore D, et al.Augmented-reality robot-assisted radical prostatectomy using hyper-accuracy three-dimensional reconstruction (HA 3DTM) technology: a radiological and pathological study[J]. BJU international, 2018, 123(5): 834-845.
[1] LIU Si-min,ZHAO Yi-jiao,WANG Xiao-yan,WANG Zu-hua. In vitro evaluation of positioning accuracy of trephine bur at different depths by dynamic navigation [J]. Journal of Peking University (Health Sciences), 2022, 54(1): 146-152.
[2] QIU Shu-ting,ZHU Yu-jia,WANG Shi-min,WANG Fei-long,YE Hong-qiang,ZHAO Yi-jiao,LIU Yun-song,WANG Yong,ZHOU Yong-sheng. Preliminary clinical application verification of complete digital workflow of design lips symmetry reference plane based on posed smile [J]. Journal of Peking University (Health Sciences), 2022, 54(1): 193-199.
[3] Guo-yong REN,Xue-mei WU, ,Jie-yu LI,Wei-ping SUN,Yi-ning HUANG. Susceptibility vessel sign in subacute stroke patients with large vessel occlusion [J]. Journal of Peking University (Health Sciences), 2021, 53(6): 1133-1138.
[4] Yuan LI,Hong LIN,Tie-jun ZHANG. Comparative study on radio-opacity of dental composite resin materials’determination using film imaging and digital imaging [J]. Journal of Peking University (Health Sciences), 2021, 53(5): 995-1001.
[5] Gang YANG,Wen-jie HU,Jie CAO,Deng-gao LIU. Three-dimensional morphology analysis of the supraosseous gingival profile of periodontally healthy maxillary anterior teeth [J]. Journal of Peking University (Health Sciences), 2021, 53(5): 990-994.
[6] Zhen-xing SHAO,Qing-fa SONG,Yu-qing ZHAO,Guo-qing CUI. An arthroscopic “inlay” Bristow procedure with suture button fixation: Surgical technique and radiology evaluation [J]. Journal of Peking University (Health Sciences), 2021, 53(5): 896-901.
[7] Yi-fan WU,Xiao-yuan ZHANG,Shuang REN,Ying-xiang YU,Cui-qing CHANG. Measurement and evaluation of the quadriceps muscle mass in young men based on magnetic resonance imaging [J]. Journal of Peking University (Health Sciences), 2021, 53(5): 843-849.
[8] HAN Song-chen,HUANG Zi-xiong,LIU Hui-xin,XU Tao. Renal functional compensation after unilateral radical nephrectomy of renal cell carcinoma [J]. Journal of Peking University (Health Sciences), 2021, 53(4): 680-685.
[9] WANG Xin-guang,GENG Xiao,LI Yang,WU Tian-chen,LI Zi-jian,TIAN Hua. Comparison of alignment and operative time between portable accelerometer-based navigation device and computer assisted surgery in total knee arthroplasty [J]. Journal of Peking University (Health Sciences), 2021, 53(4): 728-733.
[10] YUE Lei,WANG Yue-tian,BAI Chun-bi,CHEN Hao,FU Hao-yong,YU Zheng-rong,LI Chun-de,SUN Hao-lin. Analysis of surgical strategy of percutaneous endoscopic lumbar discectomy in young and middle-aged double-segment patients with lumbar disc herniation [J]. Journal of Peking University (Health Sciences), 2021, 53(4): 734-739.
[11] LIN Guo-zhong, MA Chang-cheng, WANG Zhen-yu, XIE Jing-cheng, LIU Bin, CHEN Xiao-dong. Minimally invasive treatment of cervical1-2 epidural neurilemmoma [J]. Journal of Peking University (Health Sciences), 2021, 53(3): 586-589.
[12] LI Xin-fei, PENG Yi-ji, YU Xiao-teng, XIONG Sheng-wei, CHENG Si-da, DING Guang-pu, YANG Kun-lin, TANG Qi, MI Yue, WU Jing-yun, ZHANG Peng, XIE Jia-xin, HAO Han, WANG He, QIU Jian-xing, YANG Jian, LI Xue-song, ZHOU Li-qun. Three dimensional nephrometry system for partial nephrectomy: Our initial exploration [J]. Journal of Peking University (Health Sciences), 2021, 53(3): 613-622.
[13] HU Di,ZHANG Miao,KANG Hui-ying,PENG Yun. Investigation and validation of magnetic resonance white matter atlas for 0 to 2 years old infants [J]. Journal of Peking University (Health Sciences), 2021, 53(2): 341-347.
[14] CHEN Di,XU Xiang-yu,WANG Ming-rui,LI Rui,ZANG Gen-ao,ZHANG Yue,QIAN Hao-nan,YAN Guang-rong,FAN Tian-yuan. Preparation and in vitro evaluation of fused deposition modeling 3D printed verapa-mil hydrochloride gastric floating formulations [J]. Journal of Peking University (Health Sciences), 2021, 53(2): 348-354.
[15] MENG Yuan,ZHANG Li-qi,ZHAO Ya-ning,LIU Deng-gao,ZHANG Zu-yan,GAO Yan. Three-dimentional radiographic features of 67 maxillary radicular cysts [J]. Journal of Peking University (Health Sciences), 2021, 53(2): 396-401.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
[1] Author. English Title Test[J]. Journal of Peking University(Health Sciences), 2010, 42(1): 1 -10 .
[2] . [J]. Journal of Peking University(Health Sciences), 2009, 41(2): 188 -191 .
[3] . [J]. Journal of Peking University(Health Sciences), 2009, 41(3): 376 -379 .
[4] . [J]. Journal of Peking University(Health Sciences), 2009, 41(4): 459 -462 .
[5] . [J]. Journal of Peking University(Health Sciences), 2010, 42(1): 82 -84 .
[6] . [J]. Journal of Peking University(Health Sciences), 2007, 39(3): 319 -322 .
[7] . [J]. Journal of Peking University(Health Sciences), 2007, 39(3): 333 -336 .
[8] . [J]. Journal of Peking University(Health Sciences), 2007, 39(3): 337 -340 .
[9] . [J]. Journal of Peking University(Health Sciences), 2007, 39(3): 225 -328 .
[10] . [J]. Journal of Peking University(Health Sciences), 2007, 39(4): 346 -350 .
Copyright © Journal of Peking University (Health Sciences), All Rights Reserved.
Powered by Beijing Magtech Co. Ltd