Gaussian splatting optimization via joint spatial-frequency supervision and multi-view consistency pruning

doi:10.11996/JG.j.2095-302X.2026051062

Abstract

Abstract:

To address the challenges of blurry rendering in low-texture regions, pervasive floating artifacts, and large model storage in novel view synthesis of complex scenes using 3D Gaussian Splatting (3DGS), an optimization method integratingjoint spatial-frequency supervision with multi-view consistency pruning was proposed. In terms of geometry, a multi-view consistency pruning algorithm was designed, computing the local confidence of each Gaussian primitive based on a cross-view projection consistency criterion. This is further combined with voxelized neighborhood consensus information to smooth the geometric space and periodically eliminate redundant Gaussians. For supervision, a joint spatial-domain and frequency-domain strategy was introduced. In the spatial domain, a spatially adaptive L1 loss function was devised, in which a feature extractor was constructed to generate a spatial weight map that enhances gradient contributions in high-error regions, while an energy-conservation scaling factor is employed to control the training magnitude. In the frequency domain, a magnitude regularization method based on the Dual-Tree Complex Wavelet Transform (DTCWT) was proposed, which leverages its approximate shift-invariance and multi-directional properties to constrain multi-scale subband magnitude differences during the fine-tuning stage. This smoothly adjusts the Gaussian attributes under prior guidance, mitigating the oscillation and surge in point counts caused by phase constraints. Experimental results on three public datasets, including Mip-NeRF360, demonstrated that the proposed method achieves effective improvements in both rendering quality and model compactness over the baseline 3DGS model: Structural Similarity (SSIM) is improved from 0.851 to 0.860, Peak Signal-to-Noise Ratio (PSNR) increased from 23.75 dB to 24.17 dB, and Learned Perceptual Image Patch Similarity (LPIPS) decreased from 0.169 to 0.148. In terms of model storage, the model size on the Deep Blending dataset was reduced from 628 MB to 417 MB, achieving a compression rate of approximately 33.6%. Overall, the proposed joint spatial-frequency supervision and physically grounded pruning method enabled both high-fidelity reconstruction and compact storage.

Key words: 3D reconstruction, 3D Gaussian splatting, joint spatial-frequency supervision, multi-view consistency pruning, dual-tree complex wavelet transform

CLC Number:

TP 391.41

LIN Ke, SONG Xiaohui, LONG Shiqin, QIN Zhisong. Gaussian splatting optimization via joint spatial-frequency supervision and multi-view consistency pruning[J]. Journal of Graphics, 2026, 47(5): 1062-1071.

Figures/Tables 9

References 37

[1]	MILDENHALL B, SRINIVASAN P P, TANCIK M, et al. NeRF: representing scenes as neural radiance fields for view synthesis[J]. Communications of the ACM, 2021, 65(1): 99-106.
[2]	LI Z S, MÜLLER T, EVANS A, et al. Neuralangelo: high-fidelity neural surface reconstruction[C]// 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2023: 8456-8465.
[3]	WANG P, LIU L J, LIU Y, et al. Neus: learning neural implicit surfaces by volume rendering for multi-view reconstruction[C]// The 35th International Conference on Neural Information Processing Systems. Red Hook: Curran Associates Inc., 2021, 34: 2081.
[4]	MÜLLER T, EVANS A, SCHIED C, et al. Instant neural graphics primitives with a multiresolution hash encoding[J]. ACM Transactions on Graphics, 2022, 41(4): 102.
[5]	WANG Y M, HAN Q, HABERMANN M, et al. NeuS2: fast learning of neural implicit surfaces for multi-view reconstruction[C]// 2023 IEEE/CVF International Conference on Computer Vision. New York: IEEE Press, 2023: 3272-3283.
[6]	NEFF T, STADLBAUER P, PARGER M, et al. DONeRF: towards real-time rendering of compact neural radiance fields using depth oracle networks[J]. Computer Graphics Forum, 2021, 40(4): 45-59.
[7]	KERBL B, KOPANAS G, LEIMKUEHLER T, et al. 3D gaussian splatting for real-time radiance field rendering[J]. ACM Transactions on Graphics, 2023, 42(4): 139.
[8]	ZWICKER M, PFISTER H, VAN BAAR J, et al. EWA volume splatting[C]// Visualization, 2001. New York: IEEE Press, 2001: 29-538.
[9]	BARRON J T, MILDENHALL B, VERBIN D, et al. Mip-NeRF 360: unbounded anti-aliased neural radiance fields[C]// 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2022: 5460-5469.
[10]	YU Z H, SATTLER T, GEIGER A. Gaussian opacity fields: efficient adaptive surface reconstruction in unbounded scenes[J]. ACM Transactions on Graphics, 2024, 43(6): 271.
[11]	RHO D, LEE B, NAM S, et al. Masked wavelet representation for compact neural radiance fields[C]// 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2023: 20680-20690.
[12]	XU M Y, ZHAN F N, ZHANG J H, et al. WaveNeRF: wavelet-based generalizable neural radiance fields[C]// 2023 IEEE/CVF International Conference on Computer Vision. New York: IEEE Press, 2023: 18149-18158.
[13]	CHAN E R, MONTEIRO M, KELLNHOFER P, et al. pi-GAN: periodic implicit generative adversarial networks for 3D-aware image synthesis[C]// 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2021: 5795-5805.
[14]	CHEN A, XU Z X, ZHAO F Q, et al. MVSNeRF: fast generalizable radiance field reconstruction from multi-view stereo[C]// 2021 IEEE/CVF International Conference on Computer Vision. New York: IEEE Press, 2021: 14104-14113.
[15]	KHATIB R, GIRYES R. TriNeRFLet: a wavelet based triplane NeRF representation[C]// The 18th European Conference Computer Vision. Cham: Springer, 2024: 358-374.
[16]	LOU A G, PLANCHE B, GAO Z P, et al. DaReNeRF: direction-aware representation for dynamic scenes[C]// 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2024: 5031-5042.
[17]	YANG J W, PAVONE M, WANG Y. FreeNeRF: improving few-shot neural rendering with free frequency regularization[C]// 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2023: 8254-8263.
[18]	ZHAO B Z, ZHOU Y F, YU S C, et al. Wavelet-GS: 3D gaussian splatting with wavelet decomposition[C]// The 33rd ACM International Conference on Multimedia. New York: ACM, 2025: 8616-8625.
[19]	NGUYEN H, LI R F, LE A, et al. DWTGS: rethinking frequency regularization for sparse-view 3D gaussian splatting[C]// 2025 International Conference on Visual Communications and Image Processing. New York: IEEE Press, 2025: 1-5.
[20]	LU T, YU M L, XU L N, et al. Scaffold-GS: structured 3D Gaussians for view-adaptive rendering[C]// 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2024: 20654-20664.
[21]	FAN Z W, WANG K, WEN K R, et al. LightGaussian: unbounded 3D Gaussian compression with 15x reduction and 200+ FPS[C]// The 38th International Conference on Neural Information Processing Systems. Red Hook: Curran Associates Inc., 2024: 140138-140158.
[22]	LEE J C, RHO D, SUN X Y, et al. Compact 3D gaussian representation for radiance field[C]// 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2024: 21719-21728.
[23]	PAPANTONAKIS P, KOPANAS G, KERBL B, et al. Reducing the memory footprint of 3D gaussian splatting[J]. Proceedings of the ACM on Computer Graphics and Interactive Techniques, 2024, 7(1): 16.
[24]	XIE S Z, ZHANG W X, TANG C, et al. MesonGS: post-training compression of 3D gaussians via efficient attribute transformation[C]// The 18th European Conference on Computer Vision. Cham: Springer, 2024: 434-452.
[25]	NIEDERMAYR S, STUMPFEGGER J, WESTERMANN R. Compressed 3D Gaussian splatting for accelerated novel view synthesis[C]// 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2024: 10349-10358.
[26]	WANG H R, HUANG G X, ZHANG F, et al. Prune wisely, reconstruct sharply: compact 3D Gaussian splatting via adaptive pruning and difference-of-Gaussian primitives[C]// IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2026: 11716-11725.
[27]	NAVANEET K L, POURAHMADI MEIBODI K, ABBASI KOOHPAYEGANI S, et al. CompGS: smaller and faster gaussian splatting with vector quantization[C]// The 18th European Conference on Computer Vision. Cham: Springer, 2024: 330-349.
[28]	CHEN Y H, WU Q Y, LIN W Y, et al. HAC: hash-grid assisted context for 3D Gaussian splatting compression[C]// The 18th European Conference Computer Vision. Cham: Springer, 2024: 422-438.
[29]	KERBL B, MEULEMAN A, KOPANAS G, et al. A hierarchical 3D Gaussian representation for real-time rendering of very large datasets[J]. ACM Transactions on Graphics, 2024, 43(4): 62.
[30]	HUANG L, GUO J, DAN J L, et al. Spectral-GS: taming 3D Gaussian splatting with spectral entropy[C]// SIGGRAPH Asia 2025 Conference Papers. New York: ACM, 2025: 199.
[31]	SHANG H X, CHEN M Y, FENG K N, et al. Enhanced 3D Gaussian splatting for real-scene reconstruction via depth priors, adaptive densification, and denoising[J]. Sensors, 2025, 25(22): 6999. DOI URL
[32]	ZHANG J H, ZHAN F N, XU M Y, et al. FreGS: 3D Gaussian splatting with progressive frequency regularization[C]// 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2024: 21424-21433.
[33]	YE Z X, LI W Y, LIU S D, et al. AbsGS: recovering fine details in 3D Gaussian splatting[C]// The 32nd ACM International Conference on Multimedia. New York: ACM, 2024: 1053-1061.
[34]	KNAPITSCH A, PARK J, ZHOU Q Y, et al. Tanks and temples: benchmarking large-scale scene reconstruction[J]. ACM Transactions on Graphics, 2017, 36(4): 78.
[35]	HEDMAN P, PHILIP J, PRICE T, et al. Deep blending for free-viewpoint image-based rendering[J]. ACM Transactions on Graphics, 2018, 37(6): 257.
[36]	ZHANG Z, HU W B, LAO Y X, et al. Pixel-GS: density control with pixel-aware gradient for 3D Gaussian splatting[C]// The 18th European Conference on Computer Vision. Cham: Springer, 2024: 326-342.
[37]	FANG G C, WANG B. Efficient scene modeling via structure-aware and region-prioritized 3D Gaussians[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2026, 48(4): 4623-4641. DOI URL

方法	Mip-NeRF360					Tanks & Temples
方法	SSIM↑	PSNR↑	LPIPS↓	Mem.	FPS	SSIM↑	PSNR↑	LPIPS↓	Mem.	FPS
Instant-NGP	0.751	26.75	0.298			0.745	21.92	0.305
Mip-NeRF360	0.842	29.07	0.207			0.759	22.22	0.257
3DGS	0.872	29.08	0.181	677 MB	147	0.851	23.75	0.169	423 MB	183
Pixel-GS	0.875	29.16	0.168	1.32 GB	109	0.853	23.78	0.151	1.06 GB	113
Abs-GS	0.876	29.08	0.167	719 MB	176	0.851	23.67	0.162	307 MB	245
Scaffold-GS	0.868	29.27	0.192	179 MB	145	0.823	23.96	0.177	87 MB	184
Mini-Splatting2-D	0.882	29.13	0.158	855 MB	108	0.851	23.28	0.151	641 MB	133
Ours	0.880	29.33	0.162	648 MB	127	0.860	24.17	0.148	417 MB	171

方法	Mip-NeRF360					Tanks & Temples
方法	SSIM↑	PSNR↑	LPIPS↓	Mem.	FPS	SSIM↑	PSNR↑	LPIPS↓	Mem.	FPS
Instant-NGP	0.751	26.75	0.298			0.745	21.92	0.305
Mip-NeRF360	0.842	29.07	0.207			0.759	22.22	0.257
3DGS	0.872	29.08	0.181	677 MB	147	0.851	23.75	0.169	423 MB	183
Pixel-GS	0.875	29.16	0.168	1.32 GB	109	0.853	23.78	0.151	1.06 GB	113
Abs-GS	0.876	29.08	0.167	719 MB	176	0.851	23.67	0.162	307 MB	245
Scaffold-GS	0.868	29.27	0.192	179 MB	145	0.823	23.96	0.177	87 MB	184
Mini-Splatting2-D	0.882	29.13	0.158	855 MB	108	0.851	23.28	0.151	641 MB	133
Ours	0.880	29.33	0.162	648 MB	127	0.860	24.17	0.148	417 MB	171

方法	Deep Blending
方法	SSIM↑	PSNR↑	LPIPS↓	Mem.	FPS
Instant-NGP	0.817	24.96	0.390
Mip-NeRF360	0.901	29.40	0.245
3DGS	0.906	29.82	0.238	628MB	152
Pixel-GS	0.892	28.91	0.250	1.09GB	112
Abs-GS	0.905	29.71	0.243	440MB	231
Scaffold-GS	0.906	30.21	0.254	66MB	189
Mini-Splatting2-D	0.908	29.74	0.223	979MB	149
Ours	0.909	29.87	0.229	417MB	166

方法	Deep Blending
方法	SSIM↑	PSNR↑	LPIPS↓	Mem.	FPS
Instant-NGP	0.817	24.96	0.390
Mip-NeRF360	0.901	29.40	0.245
3DGS	0.906	29.82	0.238	628MB	152
Pixel-GS	0.892	28.91	0.250	1.09GB	112
Abs-GS	0.905	29.71	0.243	440MB	231
Scaffold-GS	0.906	30.21	0.254	66MB	189
Mini-Splatting2-D	0.908	29.74	0.223	979MB	149
Ours	0.909	29.87	0.229	417MB	166

场景	SSIM↑		PSNR↑		LPIPS↓		Memory/MB
场景	3DGS	Ours	3DGS	Ours	3DGS	Ours	3DGS	Ours
Bicycle	0.765	0.788	25.23	25.48	0.207	0.173	1 252	1 169
Garden	0.868	0.873	27.51	27.65	0.105	0.095	1 116	935
Stump	0.771	0.791	26.65	26.97	0.215	0.196	1 062	960
Truck	0.884	0.894	25.44	25.96	0.141	0.110	577	502
Train	0.818	0.827	22.07	22.39	0.198	0.185	270	332
Bonsai	0.943	0.945	32.39	32.64	0.202	0.170	279	337
Room	0.919	0.924	31.44	31.72	0.217	0.196	340	416
Kitchen	0.927	0.929	31.22	31.61	0.125	0.117	429	424
Counter	0.909	0.909	29.13	29.23	0.197	0.189	286	298
Drjohnson	0.905	0.907	29.42	29.53	0.235	0.229	799	502
Playroom	0.909	0.911	30.22	30.23	0.240	0.228	457	332