DESR: Dynamic Endoscopic Scene Reconstruction Based on Gaussian Splatting

doi:10.1117/12.3053129

科研成果详情

题名	DESR: Dynamic Endoscopic Scene Reconstruction Based on Gaussian Splatting
作者	Fu，Yihang; Dong，Xiao; Miu，Runjie; Yang，Can; Wang，Haowei; Xu，Xinmu
发表日期	2024
会议名称	9th International Workshop on Pattern Recognition
会议录名称	Proceedings of SPIE - The International Society for Optical Engineering
ISSN	0277-786X
卷号	13399
会议日期	2024-06-21——2024-06-23
会议地点	Xiamen
摘要	Reconstructing soft tissues in endoscopic videos requires the support of advanced 3D reconstruction technology, which plays a key role in robot-assisted minimally invasive surgery. In this paper, we focus on how to reconstruct dynamic soft tissues as realistically as possible with surgical instruments removed. Due to the limited viewing angle and complex soft tissue deformation, current methods suffer from reconstruction blur and long training time. Inspired by the powerful rendering ability of Gaussian Splatting, we propose a dynamic endoscopic scene reconstruction method which yields better rendering quality and real-time rendering speed. We first utilize depth information to project image to point cloud and give 3D Gaussian points initial positions and colors. The properties of Gaussian points at a certain time stamp are predicted by a MLP deformation module. Our network then renders 3D Gaussian points to get 2D images using a tile-based rasterizer. We design reconstruction losses and color variation loss to optimize the rendering results. Extensive experiments show that our network generates high-fidelity reconstructed endoscopic videos with real-time rendering speed.
关键词	Endoscopic scene reconstruction Gaussian Splatting real-time
DOI	10.1117/12.3053129
URL	查看来源
语种	英语English
Scopus入藏号	2-s2.0-85208924471
引用统计
文献类型	会议论文
条目标识符	https://repository.uic.edu.cn/handle/39GCC9TT/13845
专题	北师香港浸会大学
通讯作者	Dong，Xiao
作者单位	Guangdong Provincial Key Laboratory IRADS,Department of Computer Science,BNU-HKBU United International College,Zhuhai,519087,China
第一作者单位	北师香港浸会大学
通讯作者单位	北师香港浸会大学
推荐引用方式 GB/T 7714	Fu，Yihang,Dong，Xiao,Miu，Runjieet al. DESR: Dynamic Endoscopic Scene Reconstruction Based on Gaussian Splatting[C], 2024.