北京大学学报(医学版) ›› 2025, Vol. 57 ›› Issue (1): 113-120. doi: 10.19723/j.issn.1671-167X.2025.01.017

• 论著 • 上一篇    下一篇

三维颌面对称参考平面智能构建的深度学习算法

朱玉佳1, 沈华2, 温奥楠1, 高梓翔1, 秦庆钊1, 单珅瑶3, 李文博3, 傅湘玲2, 赵一姣1,3,*(), 王勇1,*()   

  1. 1. 北京大学口腔医学院·口腔医院口腔医学数字化研究中心,国家口腔医 学中心,国家口腔疾病临床医学研究中心,口腔生物材料和数字诊疗装备国家工程研究中心,口腔数字医学北京市重点实验室,国家卫生健康委员会口腔医学计算机应用工程技术研究中心,北京 100081
    2. 北京邮电大学计算机学院(国家示范性软件学院),北京邮电大学可信分布式计算与服务教育部重点实验室,北京 100876
    3. 北京大学医学部医学技 术研究院,北京 100191
  • 收稿日期:2024-10-09 出版日期:2025-02-18 发布日期:2025-01-25
  • 通讯作者: 赵一姣,王勇 E-mail:kqcadcs@bjmu.edu.cn;kqcadc@bjmu.edu.cn
  • 作者简介:第一联系人:

    * These authors contributed equally to this work

  • 基金资助:
    国家自然科学基金(82071171);国家自然科学基金(82271039);国家重点研发计划(2022YFC2405401);北京市自然科学基金(L232100);北京市自然科学基金(L242132);北京大学口腔医院开放课题(PKUSS20230201)

Deep learning algorithms for intelligent construction of a three-dimensional maxillofacial symmetry reference plane

Yujia ZHU1, Hua SHEN2, Aonan WEN1, Zixiang GAO1, Qingzhao QIN1, Shenyao SHAN3, Wenbo LI3, Xiangling FU2, Yijiao ZHAO1,3,*(), Yong WANG1,*()   

  1. 1. Center for Digital Dentistry, Peking University School and Hospital of Stomatology & National Center for Stomatology & National Clinical Research Center for Oral Diseases & National Engineering Research Center of Oral Biomaterials and Digi-tal Medical Devices & Beijing Key Laboratory of Digital Stomatology & NHC Research Center of Engineering and Technology for Computerized Dentistry, Beijing 100081, China
    2. School of Computer Science, Beijing University of Posts and Telecommunications (National Pilot Software Engineering School); Key Laboratory of Trustworthy Distributed Computing and Service, Ministry of Education, Beijing University of Posts and Telecommunications, Beijing 100876, China
    3. Institute of Medical Technology, Peking University Health Science Center, Beijing 100191, China
  • Received:2024-10-09 Online:2025-02-18 Published:2025-01-25
  • Contact: Yijiao ZHAO, Yong WANG E-mail:kqcadcs@bjmu.edu.cn;kqcadc@bjmu.edu.cn
  • Supported by:
    the National Natural Science Foundation of China(82071171);the National Natural Science Foundation of China(82271039);National Key Research and Development Program of China(2022YFC2405401);Beijing Natural Science Foundation(L232100);Beijing Natural Science Foundation(L242132);Open Subject Foundation of Peking University Hospital of Stomatology(PKUSS20230201)

RICH HTML

  

摘要:

目的: 建立一种可实现三维颌面点云数据智能配准的本体-镜像关联深度学习算法,基于颌面动态图结构的配准网络(maxillofacial dynamic graph registration network,MDGR-Net)模型,实现三维颌面对称参考平面的自动化构建,以期为口腔临床数字化设计与分析提供参考。方法: 收集2018年10月至2022年10月就诊于北京大学口腔医院无显著颌面畸形临床患者400例,通过数据增强的方式获得2 000例三维颌面数据用于MDGR-Net算法训练与测试,其中训练集1 600例、验证集200例、内部测试集200例,MDGR-Net模型包含构造本体与镜像点云(X和Y)中关键点的特征向量,基于特征向量获取点云X和Y中关键点的对应关系,以及通过奇异值分解(singular value decomposition,SVD)计算旋转和平移矩阵R,t。基于MDGR-Net模型实现本体点云与镜像点云的智能配准,获得本体-镜像联合点云,并采用主成分分析(principal component analysis,PCA)算法获得MDGR-Net关联法对称参考平面。基于决定系数(coefficient of determination,R2)指标对内部测试集平移及旋转矩阵进行模型评价,并对200例内部测试集与40例外部测试集临床数据,基于MDGR-Net关联法与“真值”迭代最近点(iterative closest point,ICP)关联法构建的三维颌面对称参考平面进行角度误差评价。结果: 基于200例内部测试集三维颌面数据测试MDGR-Net旋转矩阵R2为0.91,平移矩阵R2为0.98。在内部与外部测试集上,角度误差平均值分别为0.84°±0.55°、0.58°±0.43°,临床构建40例三维颌面对称参考平面仅需3 s,在正畸骨性Ⅲ类、高角、安氏Ⅲ类错牙合畸形受试者表现最佳。结论: 基于点云智能配准的MDGR-Net关联法为口腔临床三维颌面对称参考平面构建提供了新的解决方案,可显著提升诊疗效率和效果,降低专家依赖性。

关键词: 颌面部, 对称参考平面, 成像,三维, 深度学习, 算法

Abstract:

Objective: To develop an original-mirror alignment associated deep learning algorithm for intelligent registration of three-dimensional maxillofacial point cloud data, by utilizing a dynamic graph-based registration network model (maxillofacial dynamic graph registration network, MDGR-Net), and to provide a valuable reference for digital design and analysis in clinical dental applications. Methods: Four hundred clinical patients without significant deformities were recruited from Peking University School of Stomatology from October 2018 to October 2022. Through data augmentation, a total of 2 000 three-dimensional maxillofacial datasets were generated for training and testing the MDGR-Net algorithm. These were divided into a training set (1 400 cases), a validation set (200 cases), and an internal test set (200 cases). The MDGR-Net model constructed feature vectors for key points in both original and mirror point clouds (X, Y), established correspondences between key points in the X and Y point clouds based on these feature vectors, and calculated rotation and translation matrices using singular value decomposition (SVD). Utilizing the MDGR-Net model, intelligent registration of the original and mirror point clouds were achieved, resulting in a combined point cloud. The principal component analysis (PCA) algorithm was applied to this combined point cloud to obtain the symmetry reference plane associated with the MDGR-Net methodology. Model evaluation for the translation and rotation matrices on the test set was performed using the coefficient of determination (R2). Angle error evaluations for the three-dimensional maxillofacial symmetry reference planes were constructed using the MDGR-Net-associated method and the "ground truth" iterative closest point (ICP)-associated method were conducted on 200 cases in the internal test set and 40 cases in an external test set. Results: Based on testing with the three-dimensional maxillofacial data from the 200-case internal test set, the MDGR-Net model achieved an R2 value of 0.91 for the rotation matrix and 0.98 for the translation matrix. The average angle error on the internal and external test sets were 0.84°±0.55° and 0.58°±0.43°, respectively. The construction of the three-dimensional maxillofacial symmetry reference plane for 40 clinical cases took only 3 seconds, with the model performing optimally in the patients with skeletal Class Ⅲ malocclusion, high angle cases, and Angle Class Ⅲ orthodontic patients. Conclusion: This study proposed the MDGR-Net association method based on intelligent point cloud registration as a novel solution for constructing three-dimensional maxillofacial symmetry reference planes in clinical dental applications, which can significantly enhance diagnostic and therapeutic efficiency and outcomes, while reduce expert dependence.

Key words: Maxillofacial, Symmetry reference plane, Imaging, three-dimensional, Deep learning, Algorithm

中图分类号: 

  • R78

图1

三维颌面数据增强与数据归一化处理"

图2

MDGR-Net网络框架图"

图3

颌面关键点特征图"

图4

MDGR-Net网络训练过程损失函数变化曲线图"

表1

MDGR-Net关联法构建三维颌面对称参考平面临床评价结果"

Dataset Angle error/(°) Accuracy rate/%
${\bar x}$±s Minimum Maximum < 1° < 2°
Internal test set 0.84±0.55 0.02 3.75 71 96
External test set 0.58±0.43 0.02 1.97 93 100

图5

MDGR-Net关联法和ICP关联法构建三维颌面对称参考平面效果示意图"

表2

外部测试集MDGR-Net关联法构建三维颌面对称参考平面临床评价结果"

Items n Angle error/(°),${\bar x}$±s
Sagittal skeletal patterns
Skeletal class Ⅰ 18 0.71±0.54
Skeletal class Ⅱ 10 0.56±0.32
Skeletal class Ⅲ 12 0.39±0.27
Vertical skeletal patterns
High angle 11 0.33±0.25
Average angle 17 0.59±0.46
Low angle 12 0.66±0.52
Angle classification
Angle class Ⅰ 18 0.56±0.30
Angle class Ⅱ 11 0.58±0.55
Angle class Ⅲ 11 0.44±0.28

图6

外部测试集MDGR-Net关联法构建三维颌面对称参考平面角度误差箱状图"

1 Khambay BS , Lowney CJ , Hsung T , et al. Fluctuating asymmetry of dynamic smiles in normal individuals[J]. Int J Oral Maxillofac Surg, 2019, 48 (10): 1372- 1379.
doi: 10.1016/j.ijom.2019.01.032
2 Hoenn M , Goez G . The ideal of facial beauty: A review[J]. J Orofac Orthop, 2007, 68 (1): 6- 16.
doi: 10.1007/s00056-007-0604-6
3 Huang CS , Liu XQ , Chen YR . Facial asymmetry index in normal young adults[J]. Orthod Craniofac Res, 2013, 16 (2): 97- 104.
doi: 10.1111/ocr.12010
4 Arora KS , Bansal R , Mohapatra S , et al. Review and classification update: Unilateral condylar hyperplasia[J]. BMJ Case Rep, 2019, 12 (2): e227569.
5 Brown JS , Barry C , Ho M , et al. A new classification for mandi-bular defects after oncological resection[J]. Lancet Oncol, 2016, 17 (1): E23- E30.
doi: 10.1016/S1470-2045(15)00310-1
6 Li Y , Shao Z , Zhu Y , et al. Virtual surgical planning for successful second-stage mandibular defect reconstruction using vascula-rized iliac crest bone flap a valid and reliable method[J]. Ann Plast Surg, 2020, 84 (2): 183- 187.
doi: 10.1097/SAP.0000000000002102
7 顾晓宇, 陈晓波, 焦婷, 等. 三维打印数字化阴模辅助制作口腔颌面缺损赝复体的临床应用[J]. 中华口腔医学杂志, 2017, 52 (6): 336- 341.
doi: 10.3760/cma.j.issn.1002-0098.2017.06.003
8 Maesschalck TD , Courvoisier DS , Scolozzi P . Computer-assisted versus traditional freehand technique in fibular free flap mandibular reconstruction: A morphological comparative study[J]. Eur Arch Otorhinolaryngol, 2017, 274 (1): 517- 526.
doi: 10.1007/s00405-016-4246-4
9 Shin SM , Kim Y , Kim N , et al. Statistical shape analysis-based determination of optimal midsagittal reference plane for evaluation of facial asymmetry[J]. Am J Orthod Dentofacial Orthop, 2016, 150 (2): 252- 260.
doi: 10.1016/j.ajodo.2016.01.017
10 Dobai A , Markella Z , Vizkelety T , et al. Landmark-based midsagittal plane analysis in patients with facial symmetry and asymmetry based on CBCT analysis tomography[J]. J Orofac Orthop, 2018, 79 (6): 371- 379.
doi: 10.1007/s00056-018-0151-3
11 Klingenberg CP , Barluenga M , Meyer A . Shape analysis of symmetric structures: Quantifying variation among individuals and asymmetry[J]. Evolution, 2002, 56 (10): 1909- 1920.
12 Hartmann J , Meyer-Marcotty P , Benz M , et al. Reliability of a method for computing facial symmetry plane and degree of asymmetry based on 3D-data[J]. J Orofac Orthop, 2007, 68 (6): 477- 490.
doi: 10.1007/s00056-007-0652-y
13 刘筱菁, 李倩倩, 王晓霞, 等. 基于本体-镜像关联的三维头颅正中矢状面自动构建法[J]. 中华口腔正畸学杂志, 2014, 21 (3): 148- 150.
doi: 10.3760/cma.j.issn.1674-5760.2014.03.006
14 Djordjevic J , Pirttiniemi P , Harila V , et al. Three-dimensional longitudinal assessment of facial symmetry in adolescents[J]. Eur J Orthod, 2013, 35 (2): 143- 151.
doi: 10.1093/ejo/cjr006
15 浙江大学, 中国食品药品检定研究院, 海军军医大学第二附属医院. 人工智能医疗器械性能评价通用方法专家共识(2023)[J]. 协和医学杂志, 2023, 14 (3): 494- 503.
16 Du S , Xu Y , Wan T , et al. Robust iterative closest point algorithm based on global reference point for rotation invariant registration[J]. PLoS One, 2017, 12 (11): e18803911.
17 Xiong Y , Zhao Y , Yang H , et al. Comparison between interactive closest point and procrustes analysis for determining the median sagittal plane of three-dimensional facial data[J]. J Craniofac Surg, 2016, 27 (2): 441- 444.
doi: 10.1097/SCS.0000000000002376
18 Zelditch , Leah M . Geometric morphometrics for biologists: A primer[M]. New York and London: Elsevier Academic Press, 2004.
19 Redmon J, Divvala S, Girshick R, et al. You only look once: Unified, real-time object detection[C]/ /Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016. Las Vegas, Nevada, USA: Institute of Electrical and Electronics Engineers, 2016: 779-788.
20 Ronneberger O, Fischer P, Brox T. U-Net: Convolutional networks for biomedical image segmentation[C]/ /Navab N, Hornegger J, Wells WM, et al. Proccedings of the International Confe-rence on Medical Image Computing and Computer-Assisted Intervention (MICCAI) 2015. Germany: Springer International Publishing, 2015: 234-241.
21 Schwendicke F , Singh T , Lee J , et al. Artificial intelligence in dental research: Checklist for authors, reviewers, readers[J]. J Dent, 2021, 107, 103610.
doi: 10.1016/j.jdent.2021.103610
22 Schwendicke F , Samek W , Krois J . Artificial intelligence in dentistry: Chances and challenges[J]. J Dent Res, 2020, 99 (7): 769- 774.
doi: 10.1177/0022034520915714
23 Wang Y , Sun Y , Liu Z , et al. Dynamic graph CNN for learning on point clouds[J]. ACM Transactions On Graphics (TOG), 2019, 38 (5): 1- 12.
24 Vaswani A, Shazeer N, Parmar N, et al. Attention is all you need[C]/ /Proceedings of the 31st International Conference on Neural Information Processing Systems (NIPS 2017). Red Hook, NY, USA: Curran Associates, Inc., 2017.
25 Wu J , Heike C , Birgfeld C , et al. Measuring symmetry in children with unrepaired cleft lip: Defining a standard for the three-dimensional midfacial reference plane[J]. Cleft Palate Craniofac J, 2016, 53 (6): 695- 704.
doi: 10.1597/15-053
[1] 许克新,丁泽华. 人工智能在功能泌尿外科的应用[J]. 北京大学学报(医学版), 2023, 55(5): 771-774.
[2] 王哲,孙伟,杨雪,宋颖,姬爱平,白洁. 口腔急诊颌面部感染患者临床分析[J]. 北京大学学报(医学版), 2023, 55(3): 543-547.
[3] 宿骞,彭歆,周传香,俞光岩. 369例口腔颌面部非霍奇金淋巴瘤的临床病理特点及预后[J]. 北京大学学报(医学版), 2023, 55(1): 13-21.
[4] 熊士凯,史尉利,王安鸿,谢兴,郭秦炜. 腓骨远端撕脱骨折的影像学诊断:踝关节X线与CT三维重建的比较[J]. 北京大学学报(医学版), 2023, 55(1): 156-159.
[5] 马芮,宣岩,段瑶,帅婷. 口腔颌面部恶性肿瘤患者术后正念水平调查及影响因素分析[J]. 北京大学学报(医学版), 2022, 54(4): 727-734.
[6] 朱玉佳,许晴,赵一姣,张磊,付子旺,温奥楠,高梓翔,张昀,傅湘玲,王勇. 深度学习算法辅助构建三维颜面正中矢状平面[J]. 北京大学学报(医学版), 2022, 54(1): 134-139.
[7] 李新飞, 彭意吉, 余霄腾, 熊盛炜, 程嗣达, 丁光璞, 杨昆霖, 唐琦, 米悦, 吴静云, 张鹏, 谢家馨, 郝瀚, 王鹤, 邱建星, 杨建, 李学松, 周利群. 肾部分切除术前CT三维可视化评估标准的初步探究[J]. 北京大学学报(医学版), 2021, 53(3): 613-622.
[8] 朱玉佳,赵一姣,郑盛文,温奥楠,傅湘玲,王勇. 基于赋权形态学分析的三维面部对称参考平面构建方法[J]. 北京大学学报(医学版), 2021, 53(1): 220-226.
[9] 唐祖南,Hui Yuh Soh,胡耒豪,于尧,章文博,彭歆. 混合现实技术在口腔颌面部肿瘤手术中的应用[J]. 北京大学学报(医学版), 2020, 52(6): 1124-1129.
[10] 王顺吉,章文博,于尧,谢晓艳,杨宏宇,彭歆. 术前虚拟设计在股前外侧皮瓣修复口腔颌面部缺损中的应用[J]. 北京大学学报(医学版), 2020, 52(1): 119-123.
[11] 戴帆帆,刘怡,许天民,陈贵. 探索成人正畸前后下颌三维数字化模型的重叠方法[J]. 北京大学学报(医学版), 2018, 50(2): 271-278.
[12] 瞿经纬,吕肖庆,刘振明,廖媛,孙鹏晖,王蓓,汤帜. 一种基于图塌缩的药物分子检索方法[J]. 北京大学学报(医学版), 2018, 50(2): 368-374.
[13] 程明轩,姜婷,孙玉春,张皓羽. 比较口内扫描和模型扫描对数字化牙列模型咬合定量分析的影响[J]. 北京大学学报(医学版), 2018, 50(1): 136-140.
[14] 魏菱,陈虎,周永胜,孙玉春,潘韶霞. 数字化全口义齿个别托盘制作与临床应用时间[J]. 北京大学学报(医学版), 2017, 49(1): 86-091.
[15] 王斯维,黎敏,杨慧芳,赵一姣,王勇,刘怡. 3种生成大视野锥形束CT数据正中矢状面方法的比较[J]. 北京大学学报(医学版), 2016, 48(2): 330-335.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!