联合空频监督与多视角一致性剪枝的高斯泼溅优化方法

doi:10.11996/JG.j.2095-302X.2026051062

摘要/Abstract

摘要：

针对3D高斯泼溅(3DGS)在复杂场景新视角合成中存在弱纹理区域渲染模糊、浮空伪影丛生及模型存储大的问题，提出了一种联合空频监督与多视角一致性剪枝的优化方法。在几何上设计多视角一致性剪枝算法，即基于跨视角投影一致性算法计算高斯基元局部置信度，结合体素化邻域共识信息平滑几何空间，定期剔除冗余高斯；设计空域与频域联合监督方法：空域上设计空间自适应L1损失函数，构建特征提取器生成空间权重图，增强高误差区域的梯度贡献，采用能量守恒缩放因子控制训练量级；频域上设计基于双树复小波变换(DTCWT)的幅值正则化方法，利用近似平移不变性和多方向特性，在精细化训练阶段约束多尺度子带幅值差异，平稳调整先验引导高斯属性，解决相位约束导致的震荡和点数激增问题。在Mip-NeRF360等3个公开数据集的实验结果表明，相较于3DGS基准模型，本文方法在渲染质量与模型轻量化均取得有效提升：结构相似性(SSIM)最高由0.851提升至0.860，峰值信噪比(PSNR)最高由23.75 dB提升至24.17 dB，学习感知图像块相似度(LPIPS)最高由0.169降低至0.148；模型存储指标上，Deep Blending数据集上由628 MB降低至417 MB，压缩幅度约为33.6%。综合而言，本文的空频域联合监督与物理剪枝方法实现了高保真重建与紧凑存储。

关键词: 三维重建, 3D高斯泼溅, 联合空频监督, 多视角一致性剪枝, 双树复小波变换

Abstract:

To address the challenges of blurry rendering in low-texture regions, pervasive floating artifacts, and large model storage in novel view synthesis of complex scenes using 3D Gaussian Splatting (3DGS), an optimization method integratingjoint spatial-frequency supervision with multi-view consistency pruning was proposed. In terms of geometry, a multi-view consistency pruning algorithm was designed, computing the local confidence of each Gaussian primitive based on a cross-view projection consistency criterion. This is further combined with voxelized neighborhood consensus information to smooth the geometric space and periodically eliminate redundant Gaussians. For supervision, a joint spatial-domain and frequency-domain strategy was introduced. In the spatial domain, a spatially adaptive L1 loss function was devised, in which a feature extractor was constructed to generate a spatial weight map that enhances gradient contributions in high-error regions, while an energy-conservation scaling factor is employed to control the training magnitude. In the frequency domain, a magnitude regularization method based on the Dual-Tree Complex Wavelet Transform (DTCWT) was proposed, which leverages its approximate shift-invariance and multi-directional properties to constrain multi-scale subband magnitude differences during the fine-tuning stage. This smoothly adjusts the Gaussian attributes under prior guidance, mitigating the oscillation and surge in point counts caused by phase constraints. Experimental results on three public datasets, including Mip-NeRF360, demonstrated that the proposed method achieves effective improvements in both rendering quality and model compactness over the baseline 3DGS model: Structural Similarity (SSIM) is improved from 0.851 to 0.860, Peak Signal-to-Noise Ratio (PSNR) increased from 23.75 dB to 24.17 dB, and Learned Perceptual Image Patch Similarity (LPIPS) decreased from 0.169 to 0.148. In terms of model storage, the model size on the Deep Blending dataset was reduced from 628 MB to 417 MB, achieving a compression rate of approximately 33.6%. Overall, the proposed joint spatial-frequency supervision and physically grounded pruning method enabled both high-fidelity reconstruction and compact storage.

Key words: 3D reconstruction, 3D Gaussian splatting, joint spatial-frequency supervision, multi-view consistency pruning, dual-tree complex wavelet transform

中图分类号:

TP 391.41

林科, 宋晓晖, 龙石钦, 覃志松. 联合空频监督与多视角一致性剪枝的高斯泼溅优化方法[J]. 图学学报, 2026, 47(5): 1062-1071.

LIN Ke, SONG Xiaohui, LONG Shiqin, QIN Zhisong. Gaussian splatting optimization via joint spatial-frequency supervision and multi-view consistency pruning[J]. Journal of Graphics, 2026, 47(5): 1062-1071.

图/表 9

参考文献 37

[1]	MILDENHALL B, SRINIVASAN P P, TANCIK M, et al. NeRF: representing scenes as neural radiance fields for view synthesis[J]. Communications of the ACM, 2021, 65(1): 99-106.
[2]	LI Z S, MÜLLER T, EVANS A, et al. Neuralangelo: high-fidelity neural surface reconstruction[C]// 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2023: 8456-8465.
[3]	WANG P, LIU L J, LIU Y, et al. Neus: learning neural implicit surfaces by volume rendering for multi-view reconstruction[C]// The 35th International Conference on Neural Information Processing Systems. Red Hook: Curran Associates Inc., 2021, 34: 2081.
[4]	MÜLLER T, EVANS A, SCHIED C, et al. Instant neural graphics primitives with a multiresolution hash encoding[J]. ACM Transactions on Graphics, 2022, 41(4): 102.
[5]	WANG Y M, HAN Q, HABERMANN M, et al. NeuS2: fast learning of neural implicit surfaces for multi-view reconstruction[C]// 2023 IEEE/CVF International Conference on Computer Vision. New York: IEEE Press, 2023: 3272-3283.
[6]	NEFF T, STADLBAUER P, PARGER M, et al. DONeRF: towards real-time rendering of compact neural radiance fields using depth oracle networks[J]. Computer Graphics Forum, 2021, 40(4): 45-59.
[7]	KERBL B, KOPANAS G, LEIMKUEHLER T, et al. 3D gaussian splatting for real-time radiance field rendering[J]. ACM Transactions on Graphics, 2023, 42(4): 139.
[8]	ZWICKER M, PFISTER H, VAN BAAR J, et al. EWA volume splatting[C]// Visualization, 2001. New York: IEEE Press, 2001: 29-538.
[9]	BARRON J T, MILDENHALL B, VERBIN D, et al. Mip-NeRF 360: unbounded anti-aliased neural radiance fields[C]// 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2022: 5460-5469.
[10]	YU Z H, SATTLER T, GEIGER A. Gaussian opacity fields: efficient adaptive surface reconstruction in unbounded scenes[J]. ACM Transactions on Graphics, 2024, 43(6): 271.
[11]	RHO D, LEE B, NAM S, et al. Masked wavelet representation for compact neural radiance fields[C]// 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2023: 20680-20690.
[12]	XU M Y, ZHAN F N, ZHANG J H, et al. WaveNeRF: wavelet-based generalizable neural radiance fields[C]// 2023 IEEE/CVF International Conference on Computer Vision. New York: IEEE Press, 2023: 18149-18158.
[13]	CHAN E R, MONTEIRO M, KELLNHOFER P, et al. pi-GAN: periodic implicit generative adversarial networks for 3D-aware image synthesis[C]// 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2021: 5795-5805.
[14]	CHEN A, XU Z X, ZHAO F Q, et al. MVSNeRF: fast generalizable radiance field reconstruction from multi-view stereo[C]// 2021 IEEE/CVF International Conference on Computer Vision. New York: IEEE Press, 2021: 14104-14113.
[15]	KHATIB R, GIRYES R. TriNeRFLet: a wavelet based triplane NeRF representation[C]// The 18th European Conference Computer Vision. Cham: Springer, 2024: 358-374.
[16]	LOU A G, PLANCHE B, GAO Z P, et al. DaReNeRF: direction-aware representation for dynamic scenes[C]// 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2024: 5031-5042.
[17]	YANG J W, PAVONE M, WANG Y. FreeNeRF: improving few-shot neural rendering with free frequency regularization[C]// 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2023: 8254-8263.
[18]	ZHAO B Z, ZHOU Y F, YU S C, et al. Wavelet-GS: 3D gaussian splatting with wavelet decomposition[C]// The 33rd ACM International Conference on Multimedia. New York: ACM, 2025: 8616-8625.
[19]	NGUYEN H, LI R F, LE A, et al. DWTGS: rethinking frequency regularization for sparse-view 3D gaussian splatting[C]// 2025 International Conference on Visual Communications and Image Processing. New York: IEEE Press, 2025: 1-5.
[20]	LU T, YU M L, XU L N, et al. Scaffold-GS: structured 3D Gaussians for view-adaptive rendering[C]// 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2024: 20654-20664.
[21]	FAN Z W, WANG K, WEN K R, et al. LightGaussian: unbounded 3D Gaussian compression with 15x reduction and 200+ FPS[C]// The 38th International Conference on Neural Information Processing Systems. Red Hook: Curran Associates Inc., 2024: 140138-140158.
[22]	LEE J C, RHO D, SUN X Y, et al. Compact 3D gaussian representation for radiance field[C]// 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2024: 21719-21728.
[23]	PAPANTONAKIS P, KOPANAS G, KERBL B, et al. Reducing the memory footprint of 3D gaussian splatting[J]. Proceedings of the ACM on Computer Graphics and Interactive Techniques, 2024, 7(1): 16.
[24]	XIE S Z, ZHANG W X, TANG C, et al. MesonGS: post-training compression of 3D gaussians via efficient attribute transformation[C]// The 18th European Conference on Computer Vision. Cham: Springer, 2024: 434-452.
[25]	NIEDERMAYR S, STUMPFEGGER J, WESTERMANN R. Compressed 3D Gaussian splatting for accelerated novel view synthesis[C]// 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2024: 10349-10358.
[26]	WANG H R, HUANG G X, ZHANG F, et al. Prune wisely, reconstruct sharply: compact 3D Gaussian splatting via adaptive pruning and difference-of-Gaussian primitives[C]// IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2026: 11716-11725.
[27]	NAVANEET K L, POURAHMADI MEIBODI K, ABBASI KOOHPAYEGANI S, et al. CompGS: smaller and faster gaussian splatting with vector quantization[C]// The 18th European Conference on Computer Vision. Cham: Springer, 2024: 330-349.
[28]	CHEN Y H, WU Q Y, LIN W Y, et al. HAC: hash-grid assisted context for 3D Gaussian splatting compression[C]// The 18th European Conference Computer Vision. Cham: Springer, 2024: 422-438.
[29]	KERBL B, MEULEMAN A, KOPANAS G, et al. A hierarchical 3D Gaussian representation for real-time rendering of very large datasets[J]. ACM Transactions on Graphics, 2024, 43(4): 62.
[30]	HUANG L, GUO J, DAN J L, et al. Spectral-GS: taming 3D Gaussian splatting with spectral entropy[C]// SIGGRAPH Asia 2025 Conference Papers. New York: ACM, 2025: 199.
[31]	SHANG H X, CHEN M Y, FENG K N, et al. Enhanced 3D Gaussian splatting for real-scene reconstruction via depth priors, adaptive densification, and denoising[J]. Sensors, 2025, 25(22): 6999. DOI URL
[32]	ZHANG J H, ZHAN F N, XU M Y, et al. FreGS: 3D Gaussian splatting with progressive frequency regularization[C]// 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2024: 21424-21433.
[33]	YE Z X, LI W Y, LIU S D, et al. AbsGS: recovering fine details in 3D Gaussian splatting[C]// The 32nd ACM International Conference on Multimedia. New York: ACM, 2024: 1053-1061.
[34]	KNAPITSCH A, PARK J, ZHOU Q Y, et al. Tanks and temples: benchmarking large-scale scene reconstruction[J]. ACM Transactions on Graphics, 2017, 36(4): 78.
[35]	HEDMAN P, PHILIP J, PRICE T, et al. Deep blending for free-viewpoint image-based rendering[J]. ACM Transactions on Graphics, 2018, 37(6): 257.
[36]	ZHANG Z, HU W B, LAO Y X, et al. Pixel-GS: density control with pixel-aware gradient for 3D Gaussian splatting[C]// The 18th European Conference on Computer Vision. Cham: Springer, 2024: 326-342.
[37]	FANG G C, WANG B. Efficient scene modeling via structure-aware and region-prioritized 3D Gaussians[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2026, 48(4): 4623-4641. DOI URL

方法	Mip-NeRF360					Tanks & Temples
方法	SSIM↑	PSNR↑	LPIPS↓	Mem.	FPS	SSIM↑	PSNR↑	LPIPS↓	Mem.	FPS
Instant-NGP	0.751	26.75	0.298			0.745	21.92	0.305
Mip-NeRF360	0.842	29.07	0.207			0.759	22.22	0.257
3DGS	0.872	29.08	0.181	677 MB	147	0.851	23.75	0.169	423 MB	183
Pixel-GS	0.875	29.16	0.168	1.32 GB	109	0.853	23.78	0.151	1.06 GB	113
Abs-GS	0.876	29.08	0.167	719 MB	176	0.851	23.67	0.162	307 MB	245
Scaffold-GS	0.868	29.27	0.192	179 MB	145	0.823	23.96	0.177	87 MB	184
Mini-Splatting2-D	0.882	29.13	0.158	855 MB	108	0.851	23.28	0.151	641 MB	133
Ours	0.880	29.33	0.162	648 MB	127	0.860	24.17	0.148	417 MB	171

方法	Mip-NeRF360					Tanks & Temples
方法	SSIM↑	PSNR↑	LPIPS↓	Mem.	FPS	SSIM↑	PSNR↑	LPIPS↓	Mem.	FPS
Instant-NGP	0.751	26.75	0.298			0.745	21.92	0.305
Mip-NeRF360	0.842	29.07	0.207			0.759	22.22	0.257
3DGS	0.872	29.08	0.181	677 MB	147	0.851	23.75	0.169	423 MB	183
Pixel-GS	0.875	29.16	0.168	1.32 GB	109	0.853	23.78	0.151	1.06 GB	113
Abs-GS	0.876	29.08	0.167	719 MB	176	0.851	23.67	0.162	307 MB	245
Scaffold-GS	0.868	29.27	0.192	179 MB	145	0.823	23.96	0.177	87 MB	184
Mini-Splatting2-D	0.882	29.13	0.158	855 MB	108	0.851	23.28	0.151	641 MB	133
Ours	0.880	29.33	0.162	648 MB	127	0.860	24.17	0.148	417 MB	171

方法	Deep Blending
方法	SSIM↑	PSNR↑	LPIPS↓	Mem.	FPS
Instant-NGP	0.817	24.96	0.390
Mip-NeRF360	0.901	29.40	0.245
3DGS	0.906	29.82	0.238	628MB	152
Pixel-GS	0.892	28.91	0.250	1.09GB	112
Abs-GS	0.905	29.71	0.243	440MB	231
Scaffold-GS	0.906	30.21	0.254	66MB	189
Mini-Splatting2-D	0.908	29.74	0.223	979MB	149
Ours	0.909	29.87	0.229	417MB	166

方法	Deep Blending
方法	SSIM↑	PSNR↑	LPIPS↓	Mem.	FPS
Instant-NGP	0.817	24.96	0.390
Mip-NeRF360	0.901	29.40	0.245
3DGS	0.906	29.82	0.238	628MB	152
Pixel-GS	0.892	28.91	0.250	1.09GB	112
Abs-GS	0.905	29.71	0.243	440MB	231
Scaffold-GS	0.906	30.21	0.254	66MB	189
Mini-Splatting2-D	0.908	29.74	0.223	979MB	149
Ours	0.909	29.87	0.229	417MB	166

场景	SSIM↑		PSNR↑		LPIPS↓		Memory/MB
场景	3DGS	Ours	3DGS	Ours	3DGS	Ours	3DGS	Ours
Bicycle	0.765	0.788	25.23	25.48	0.207	0.173	1 252	1 169
Garden	0.868	0.873	27.51	27.65	0.105	0.095	1 116	935
Stump	0.771	0.791	26.65	26.97	0.215	0.196	1 062	960
Truck	0.884	0.894	25.44	25.96	0.141	0.110	577	502
Train	0.818	0.827	22.07	22.39	0.198	0.185	270	332
Bonsai	0.943	0.945	32.39	32.64	0.202	0.170	279	337
Room	0.919	0.924	31.44	31.72	0.217	0.196	340	416
Kitchen	0.927	0.929	31.22	31.61	0.125	0.117	429	424
Counter	0.909	0.909	29.13	29.23	0.197	0.189	286	298
Drjohnson	0.905	0.907	29.42	29.53	0.235	0.229	799	502
Playroom	0.909	0.911	30.22	30.23	0.240	0.228	457	332