单目深度模糊感知估计的少视图三维高斯重建

doi:10.11996/JG.j.2095-302X.2026040704

摘要/Abstract

摘要：

三维高斯溅射(3DGS)技术是神经辐射场(NeRF)理论提出之后，三维重建和新视图合成领域的又一重大突破。3DGS通过将多视角图像转换为数以百万计的高斯基元建模显示的场景表示以及可微分渲染技术实现了近乎实时的视图渲染，但仅依靠外观颜色优化高斯基元往往需要不同视角的大量视图提供全局几何线索提升场景重建质量，否则在稀疏视图输入下由于高斯核函数自身的局部支持以及缺少全局几何约束容易过拟合训练视图，导致重建场景出现空洞、浮动伪影等问题。为此，提出了一种在稀疏视图输入情况下的三维高斯重建算法以进一步拓展3DGS应用场景。针对稀疏输入场景设置，首先利用单目深度模糊感知技术，通过条件隐式极大似然估计学习深度估计的多模态分布，以提取多模态深度稠密点云用于初始化高斯基元，为场景重建引入全局几何线索。然后，设计了一种空间雕刻损失解决高斯基元空间位置在初始化阶段保留的不确定性和模糊性，捕获单目深度分布中全局立体一致的模式子集，获取全局场景几何结构，进而缓解稀疏视图下的场景过拟合现象并有效提升场景重建质量。实验结果表明，该算法在各个数据集下的PSNR性能均值较同期算法提升了11%~ 54%不等，同时训练时长较基于NeRF的算法大幅缩短，实现了良好的场景重建效果与新视图合成质量的提升。

关键词: 三维重建, 稀疏输入, 新视图合成, 三维高斯溅射, 单目深度估计

Abstract:

After Neural Radiance Fields (NeRF), which predominantly use coordinate-based models to map spatial coordinates to pixel values, 3D Gaussian Splatting (3DGS) has emerged as an important breakthrough technique in the realm of three-dimensional reconstruction and novel view synthesis. By transforming multi-view images into millions of learnable 3D Gaussians to model an explicit scene representation, which introduces unprecedented levels of editability, and using differentiable rendering, 3DGS achieves near real-time view rendering. However, 3DGS is prone to overfitting the training views when a small number of images are available due to Gaussian splats’ local nature. Moreover, 3DGS optimizes independent splats only under multi-view color supervision, without global geometric structure. This problem becomes more pronounced as the number of images used for 3D scene optimization decreases, because sufficient images that can offer global geometric cues are unavailable. As a result, reconstructed scenes under sparse-view input often suffer from problems such as holes and floating artifacts. To address this issue, an algorithm of few-shot 3D Gaussian splatting based on monocular depth ambiguity-aware estimation was proposed to improve the quality of scene reconstruction and the effect of novel view synthesis, thereby further expanding the 3DGS application scenario. For sparse-input settings, Conditional Implicit Maximum Likelihood Estimation (CIMLE) was used to learn the multimodal distribution of the depth estimation to avoid the unimodal distribution that may be caused by the conditional GAN mode collapse. This monocular depth ambiguity-aware technique was used to extract a multi-modal depth dense point cloud for initialization of Gaussian primitives, and to introduce global geometric cues for scene reconstruction. Then, a space-carving loss was designed, along with color-based photometric loss optimization to jointly supervise and optimize the scene structure, thereby resolving the inherent uncertainty and ambiguity retained by the spatial positions of Gaussian primitives in the initialization phase. The multi-modal depth distribution was used to constrain the optimization of Gaussian primitives, with the goal of finding a globally consistent subset of patterns captured from the monocular depth ambiguity-aware distribution of each view. In this way, information from different views was fused together, since the inherent ambiguity can only be resolved through multi-view information. This allowed a common shape consistent across all views and the object surface to be “snapped” together. This approach alleviated scene overfitting under sparse views and effectively improved the quality of scene reconstruction. The experimental results showed that the average PSNR performance of the proposed three-dimensional Gaussian reconstruction algorithm was 11%-54% higher than that of the same period algorithm. At the same time, training time was greatly reduced compared with the NeRF-based algorithm, and improved scene reconstruction and novel-view-synthesis quality was achieved.

Key words: three-dimensional reconstruction, sparse input, novel view synthesis, 3D Gaussian splatting, monocular depth estimation

中图分类号:

TP391

王紫威, 王录涛, 李桉同, 沈艳. 单目深度模糊感知估计的少视图三维高斯重建[J]. 图学学报, 2026, 47(4): 704-713.

WANG Ziwei, WANG Lutao, LI Antong, SHEN Yan. Few-shot 3D Gaussian splatting based on monocular depth ambiguity-aware estimation[J]. Journal of Graphics, 2026, 47(4): 704-713.

图/表 15

参考文献 26

[1]	曹振中, 光金正, 张千一, 等. 基于3D高斯溅射的3维重建技术综述[J]. 机器人, 2024, 46(5): 611-622.
	CAO Z Z, GUANG J Z, ZHANG Q Y, et al. Survey of 3D reconstruction techniques based on 3D Gaussian splatting[J]. Robot, 2024, 46(5): 611-622 (in Chinese).
[2]	MILDENHALL B, SRINIVASAN P P, TANCIK M, et al. NeRF: representing scenes as neural radiance fields for view synthesis[C]// The 16th European Conference on Computer Vision. Cham: Springer, 2020: 405-421.
[3]	KERBL B, KOPANAS G, LEIMKÜHLER T, et al. 3D Gaussian splatting for real-time radiance field rendering[J]. ACM Transactions on Graphics, 2023, 42(4): 139.
[4]	CHUNG J, OH J, LEE K M. Depth-regularized optimization for 3D Gaussian splatting in few-shot images[C]// 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. New York: IEEE Press, 2024: 811-820.
[5]	于航, 付彦伟, 姜柏言, 等. 基于少量图像的三维重建综述[J]. 智能科学与技术学报, 2022, 4(4): 544-559. DOI
	YU H, FU Y W, JIANG B Y, et al. A survey of image-based few-shot 3D reconstruction[J]. Chinese Journal of Intelligent Science and Technology, 2022, 4(4): 544-559 (in Chinese).
[6]	BAO Y Q, DING T Y, HUO J, et al. 3D Gaussian splatting: survey, technologies, challenges, and opportunities[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2025, 35(7): 6832-6852. DOI URL
[7]	SCHÖNBERGER J L, FRAHM J M. Structure-from-motion revisited[C]// 2016 IEEE Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2016: 4104-4113.
[8]	PARK H, RYU G, KIM W. DropGaussian: structural regularization for sparse-view Gaussian splatting[C]// 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2025: 21600-21609.
[9]	高建, 陈林卓, 沈秋, 等. 基于三维高斯溅射技术的可微分渲染研究进展(特邀)[J]. 激光与光电子学进展, 2024, 61(16): 1611010.
	GAO J, CHEN L Z, SHEN Q, et al. Advances in differentiable rendering based on three-dimensional Gaussian splatting (Invited)[J]. Laser & Optoelectronics Progress, 2024, 61(16): 1611010 (in Chinese).
[10]	肖强, 陈铭林, 张晔, 等. 室内稀疏全景图的神经辐射场重建[J]. 中国图象图形学报, 2024, 29(9): 2596-2609.
	XIAO Q, CHEN M L, ZHANG Y, et al. Neural radiance field reconstruction for sparse indoor panoramas[J]. Journal of Image and Graphics, 2024, 29(9): 2596-2609 (in Chinese). DOI URL
[11]	ZHU Z H, FAN Z W, JIANG Y F, et al. FSGS: real-time few-shot view synthesis using Gaussian splatting[C]// The 18th European Conference on Computer Vision. Cham: Springer, 2025: 145-163.
[12]	LI J H, ZHANG J W, BAI X, et al. DNGaussian: optimizing sparse-view 3D Gaussian radiance fields with global-local depth normalization[C]// 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2024: 20775-20785.
[13]	CHARATAN D, LI S L, TAGLIASACCHI A, et al. PixelSplat: 3D Gaussian splats from image pairs for scalable generalizable 3D reconstruction[C]// 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2024: 19457-19467.
[14]	AZINOVIĆ D, MARTIN-BRUALLA R, GOLDMAN D B, et al. Neural RGB-D surface reconstruction[C]// 2022 IEEE/ CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2022: 6280-6291.
[15]	ZHANG Z Y. Microsoft Kinect sensor and its effect[J]. IEEE MultiMedia, 2012, 19(2): 4-10.
[16]	YIN W, ZHANG J M, WANG O, et al. Learning to recover 3D scene shape from a single image[C]// 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2021: 204-213.
[17]	PHILION J, FIDLER S. Lift, splat, shoot: encoding images from arbitrary camera rigs by implicitly unprojecting to 3D[C]// The 16th European Conference on Computer Vision. Cham: Springer, 2020: 194-210.
[18]	LI K, PENG S C, ZHANG T H, et al. Multimodal image synthesis with conditional implicit maximum likelihood estimation[J]. International Journal of Computer Vision, 2020, 128(10/11): 2607-2628. DOI
[19]	HUANG X, BELONGIE S. Arbitrary style transfer in real-time with adaptive instance normalization[C]// 2017 IEEE International Conference on Computer Vision. New York: IEEE Press, 2017: 1510-1519.
[20]	UY M A, MARTIN-BRUALLA R, GUIBAS L, et al. SCADE: NeRFs from space carving with ambiguity-aware depth estimates[C]// 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2023: 16518-16527.
[21]	FU Y, WANG X L, LIU S F, et al. COLMAP-free 3D Gaussian splatting[C]// 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2024: 20796-20805.
[22]	DAI A, CHANG A X, SAVVA M, et al. ScanNet: richly- annotated 3D reconstructions of indoor scenes[C]// 2017 IEEE Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2017: 2432-2443.
[23]	KNAPITSCH A, PARK J, ZHOU Q Y, et al. Tanks and temples: benchmarking large-scale scene reconstruction[J]. ACM Transactions on Graphics, 2017, 36(4): 78.
[24]	ROESSLE B, BARRON J T, MILDENHALL B, et al. Dense depth priors for neural radiance fields from sparse input views[C]// 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2022: 12882-12891.
[25]	WANG Z, BOVIK A C, SHEIKH H R, et al. Image quality assessment: from error visibility to structural similarity[J]. IEEE Transactions on Image Processing, 2004, 13(4): 600-612. DOI PMID
[26]	ZHANG R, ISOLA P, EFROS A A, et al. The unreasonable effectiveness of deep features as a perceptual metric[C]// 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2018: 586-595.

方法	In_the_wild（所有场景）			Basement			Kitchen			Lounge
方法	PSNR↑	SSIM↑	LPIPS↓	PSNR↑	SSIM↑	LPIPS↓	PSNR↑	SSIM↑	LPIPS↓	PSNR↑	SSIM↑	LPIPS↓
3DGS^[2]	15.18	0.60	0.50	14.27	0.64	0.47	14.37	0.59	0.53	16.91	0.57	0.51
SCADE^[20]	19.03	0.68	0.61	19.33	0.72	0.58	19.00	0.67	0.60	18.75	0.64	0.65
DRGS^[4]	11.19	0.58	0.65	10.45	0.59	0.59	10.84	0.56	0.65	12.29	0.58	0.70
DropGS^[8]	16.57	0.55	0.38	16.14	0.56	0.38	14.83	0.44	0.45	18.74	0.65	0.32
本文方法1	20.66	0.73	0.39	20.20	0.76	0.36	20.93	0.74	0.39	20.86	0.70	0.42
本文方法2	20.69	0.69	0.39	20.48	0.74	0.36	20.45	0.69	0.39	21.14	0.63	0.41

方法	In_the_wild（所有场景）			Basement			Kitchen			Lounge
方法	PSNR↑	SSIM↑	LPIPS↓	PSNR↑	SSIM↑	LPIPS↓	PSNR↑	SSIM↑	LPIPS↓	PSNR↑	SSIM↑	LPIPS↓
3DGS^[2]	15.18	0.60	0.50	14.27	0.64	0.47	14.37	0.59	0.53	16.91	0.57	0.51
SCADE^[20]	19.03	0.68	0.61	19.33	0.72	0.58	19.00	0.67	0.60	18.75	0.64	0.65
DRGS^[4]	11.19	0.58	0.65	10.45	0.59	0.59	10.84	0.56	0.65	12.29	0.58	0.70
DropGS^[8]	16.57	0.55	0.38	16.14	0.56	0.38	14.83	0.44	0.45	18.74	0.65	0.32
本文方法1	20.66	0.73	0.39	20.20	0.76	0.36	20.93	0.74	0.39	20.86	0.70	0.42
本文方法2	20.69	0.69	0.39	20.48	0.74	0.36	20.45	0.69	0.39	21.14	0.63	0.41

方法	Scannet（所有场景）			Scene710			Scene758			Scene781
方法	PSNR↑	SSIM↑	LPIPS↓	PSNR↑	SSIM↑	LPIPS↓	PSNR↑	SSIM↑	LPIPS↓	PSNR↑	SSIM↑	LPIPS↓
3DGS^[2]	11.85	0.47	0.59	12.00	0.42	0.59	10.31	0.45	0.60	13.23	0.54	0.58
SCADE^[20]	19.75	0.65	0.58	19.33	0.62	0.63	19.42	0.68	0.53	20.50	0.65	0.57
DRGS^[4]	11.87	0.57	0.63	12.54	0.57	0.61	9.63	0.51	0.66	13.44	0.62	0.63
DropGS^[8]	16.16	0.54	0.38	16.36	0.52	0.38	14.33	0.46	0.44	17.79	0.63	0.32
本文方法1	15.91	0.59	0.49	16.75	0.57	0.47	14.24	0.58	0.51	16.73	0.61	0.49
本文方法2	16.93	0.56	0.49	16.91	0.55	0.48	15.27	0.55	0.50	17.00	0.59	0.49

方法	Scannet（所有场景）			Scene710			Scene758			Scene781
方法	PSNR↑	SSIM↑	LPIPS↓	PSNR↑	SSIM↑	LPIPS↓	PSNR↑	SSIM↑	LPIPS↓	PSNR↑	SSIM↑	LPIPS↓
3DGS^[2]	11.85	0.47	0.59	12.00	0.42	0.59	10.31	0.45	0.60	13.23	0.54	0.58
SCADE^[20]	19.75	0.65	0.58	19.33	0.62	0.63	19.42	0.68	0.53	20.50	0.65	0.57
DRGS^[4]	11.87	0.57	0.63	12.54	0.57	0.61	9.63	0.51	0.66	13.44	0.62	0.63
DropGS^[8]	16.16	0.54	0.38	16.36	0.52	0.38	14.33	0.46	0.44	17.79	0.63	0.32
本文方法1	15.91	0.59	0.49	16.75	0.57	0.47	14.24	0.58	0.51	16.73	0.61	0.49
本文方法2	16.93	0.56	0.49	16.91	0.55	0.48	15.27	0.55	0.50	17.00	0.59	0.49

方法	Tanks & Temples（所有场景）			Auditorium			Church			Courtroom
方法	PSNR↑	SSIM↑	LPIPS↓	PSNR↑	SSIM↑	LPIPS↓	PSNR↑	SSIM↑	LPIPS↓	PSNR↑	SSIM↑	LPIPS↓
3DGS^[2]	15.70	0.53	0.49	18.05	0.70	0.41	13.18	0.39	0.56	15.88	0.49	0.51
SCADE^[20]	17.46	0.55	0.69	19.2	0.71	0.62	15.91	0.46	0.71	17.26	0.47	0.75
DRGS^[4]	12.53	0.48	0.73	15.46	0.64	0.65	9.81	0.36	0.79	12.32	0.43	0.74
DropGS^[8]	17.59	0.62	0.31	19.83	0.69	0.27	16.00	0.55	0.35	16.94	0.61	0.30
本文方法1	18.10	0.63	0.44	20.01	0.76	0.37	15.88	0.54	0.46	18.40	0.59	0.49
本文方法2	18.44	0.60	0.41	20.23	0.73	0.37	16.08	0.50	0.46	19.01	0.58	0.41