Acceleration method for neural implicit surface reconstruction with joint point cloud priors

doi:10.11996/JG.j.2095-302X.2025040807

Abstract

Abstract:

To address the practical problem that the current neural implicit surface reconstruction tasks cost high training time, a sampling method guided by joint point-cloud priors we proposed, which reduced the model training time cost while ensuring the quality of surface reconstruction. Acceleration for the training of neural implicit surface reconstruction networks was achieved from three aspects: firstly, it alternated between random training pixel sampling and adaptive training pixel sampling based on point-cloud projection density to accelerat the model’s optimization for the locations of the surface to be reconstructed; secondly, by utilizing point-cloud priors and the adjacency relationship of sampled pixels, the propsed approach concentrated sampling on locations near the surface on training rays, thus reducing the number and time cost of importance sampling; in addition, it leveraged sparse point cloud prior loss to optimize the signed distance field network and periodically updated the point cloud cache with a certain iteration step. Comparative experiments conducted on ten test scenes from the DTU and Tanks-and-Temples datasets demonstrated that the proposed method can significantly reduce the training time cost of neural implicit surface reconstruction while preserving the quality of the reconstruction. When compared to the NeuS neural implicit surface reconstruction method, our approach reduced training time costs by 35%; with the same training duration, our approach achieved a 3.1% average increase in peak signal-to-noise ratio (PSNR) and a 3.4% average improvement in structural similarity index (SSIM) for new viewpoint image predictions.

Key words: surface reconstruction, neural rendering, neural implicit surface, point cloud, adaptive sampling

CLC Number:

GUO Mingce, HUANG Bei, CHENG Lechao, WANG Zhangye. Acceleration method for neural implicit surface reconstruction with joint point cloud priors[J]. Journal of Graphics, 2025, 46(4): 807-817.

Figures/Tables 21

Fig. 1 Framework diagram of our method

Fig. 2 Adaptive pixel sampling heatmap ((a) Coarsening coefficients of 1 pixel units; (b) Coarsening coefficients of 2 pixel units; (c) Coarsening coefficients of 4 pixel units; (d) Coarsening coefficients of 8 pixel units)

Fig. 3 Dheatmap adaptive pixel sampling ((a) Coarsening coefficients of 1 pixel units; (b) Coarsening coefficients of 2 pixel units; (c) Coarsening coefficients of 4 pixel units; (d) Coarsening coefficients of 8 pixel units)

Fig. 4 Dfair adaptive pixel sampling ((a) Coarsening coefficients of 1 pixel units; (b) Coarsening coefficients of 2 pixel units; (c) Coarsening coefficients of 4 pixel units; (d) Coarsening coefficients of 8 pixel units)

Fig. 5 h(η,λ) regarding the relationship diagram of changes between λ and η

Table 1 The total number of sampling points participating in training for each method

方法	$S R$
NeRF^[1]	$N ray × (S unf + S imp)$
NeuS^[5]	$N ray × (S unf + S imp + S out)$
本文	$N ray × (S unf + S imp + S out) + M × S ppg$

Table 1 The total number of sampling points participating in training for each method

方法	$S R$
NeRF^[1]	$N ray × (S unf + S imp)$
NeuS^[5]	$N ray × (S unf + S imp + S out)$
本文	$N ray × (S unf + S imp + S out) + M × S ppg$

Fig. 6 Initial point cloud, sparse scene sampling points (yellow) and uniform sampling points (green)

Table 2 Training parameters for our method

参数	数值
μ_cs	4
N^ray	256
Step^pri	10000
τ^R	0.5

Table 2 Training parameters for our method

参数	数值
μ_cs	4
N^ray	256
Step^pri	10000
τ^R	0.5

Table 3 Sampling parameters and time cost for training each method

方法	S^unf	S^imp	S^ppg	S^out	Time
NeRF^[1]	64	64	-	-	12
NeuS^[5]	64	64	-	32	17
本文方法	32	32	16	32	11

Fig. 7 Changes in the average point cloud cache size during the training of this method

Fig. 8 Partial scene reconstruction of point clouds and images ((a) Dtu_scan 24[29]; (b) Dtu_scan 37[29]; (c) Dtu_scan 55[29]; (d) Dtu_scan 63[29]; (e) Truck[32]; (f) Ignatiu[32])

Table 4 Peak signal-to-noise ratio of each method and scenario experiment

方法	场景
方法	scan_24	scan_37	scan_55	scan_63	scan_106	scan_114	scan_118	scan_122	Truck	Ignatius
NeRF^[1]	24.49	21.44	24.80	24.37	30.50	30.85	34.58	30.59	19.02	17.07
NeuS^[5]	24.50	21.79	23.92	30.62	31.82	30.45	33.43	34.38	21.14	20.03
本文方法	26.15	22.06	25.23	31.90	32.36	31.35	34.13	34.43	21.40	20.25

Table 5 Structural similarity of each method and scenario experiment

方法	场景
方法	scan_24	scan_37	scan_55	scan_63	scan_106	scan_114	scan_118	scan_122	Truck	Ignatius
NeRF^[1]	0.779 7	0.725 0	0.745 3	0.898 1	0.848 9	0.909 6	0.926 7	0.857 6	0.556 3	0.399 9
NeuS^[5]	0.789 3	0.744 5	0.735 9	0.940 9	0.896 4	0.881 4	0.900 5	0.907 2	0.675 0	0.560 2
本文方法	0.812 3	0.797 2	0.784 6	0.955 3	0.906 3	0.900 6	0.910 8	0.917 4	0.681 1	0.568 1

Fig. 9 The relationship between color loss and time in this our method and NeuS[5] method

Fig. 10 Comparison of surface reconstruction effects of various testing scenarios using different methods ((a) Real images; (b) NeRF[2]; (c) NeuS[26]; (d) Ours)

Table 6 Peak signal-to-noise ratio of each pixel sample strategy

采样策略	场景
采样策略	scan_24	scan_37	scan_55	scan_63	scan_106	scan_114	scan_118	scan_122
D^random	25.57	21.45	23.76	30.25	31.15	30.25	32.66	32.62
D^heatmap	26.06	21.89	24.55	31.79	32.33	31.29	33.91	33.45
D^fair	26.15	22.06	25.23	31.90	32.36	31.35	34.02	33.53

Table 7 Structural similarity of each pixel sample strategy

采样策略	场景
采样策略	scan_24	scan_37	scan_55	scan_63	scan_106	scan_114	scan_118	scan_122
D^random	0.804 6	0.785 2	0.774 6	0.948 7	0.901 7	0.895 8	0.907 8	0.915 4
D^heatmap	0.810 0	0.794 3	0.783 6	0.952 8	0.903 5	0.899 5	0.908 0	0.916 3
D^fair	0.811 3	0.793 2	0.784 4	0.953 6	0.906 3	0.900 6	0.910 8	0.917 4

Table 8 Peak signal-to-noise ratio of different prior optimization steps

Step^pri	场景
Step^pri	scan_24	scan_37	scan_55	scan_63	scan_106	scan_114	scan_118	scan_122
0	25.83	21.68	24.82	30.22	31.72	30.98	33.79	34.02
10 000	26.15	22.06	25.23	31.90	32.36	31.35	34.13	34.43
20 000	26.26	22.27	25.47	32.08	32.43	31.49	34.28	34.53

Table 9 Structural similarity of different prior optimization steps

Step^pri	场景
Step^pri	scan_24	scan_37	scan_55	scan_63	scan_106	scan_114	scan_118	scan_122
0	0.810 5	0.800 8	0.782 1	0.952 7	0.901 2	0.898 5	0.906 8	0.909 6
10 000	0.812 3	0.797 2	0.784 6	0.955 3	0.906 3	0.900 6	0.910 8	0.917 4
20 000	0.813 0	0.809 2	0.786 8	0.956 3	0.909 2	0.902 4	0.912 8	0.918 3
50 000	0.812 5	0.810 1	0.785 6	0.955 5	0.910 1	0.901 1	0.913 7	0.918 2

Table 10 Peak signal-to-noise ratio of different sample number from prior point cloud guide

S^ppg	场景
S^ppg	scan_24	scan_37	scan_55	scan_63	scan_106	scan_114	scan_118	scan_122
8	25.34	21.93	24.95	29.92	31.25	30.43	32.93	33.23
12	25.52	22.74	25.10	30.76	31.80	30.87	33.76	33.83
16	26.15	22.06	25.23	31.90	32.36	31.35	34.13	34.43

Table 11 Structural similarity of different sample number from prior point cloud guide

S^ppg	场景
S^ppg	scan_24	scan_37	scan_55	scan_63	scan_106	scan_114	scan_118	scan_122
8	0.802 1	0.792 1	0.782 1	0.934 1	0.892 3	0.883 6	0.889 6	0.897 2
12	0.808 4	0.793 4	0.783 4	0.947 4	0.903 5	0.890 4	0.898 7	0.908 4
16	0.812 3	0.797 2	0.784 6	0.955 3	0.906 3	0.900 6	0.910 8	0.917 4

References 32

[1]	MILDENHALL B, SRINIVASAN P P, TANCIK M, et al. NeRF: representing scenes as neural radiance fields for view synthesis[J]. Communications of the ACM, 2021, 65(1): 99-106.
[2]	YANG M D, CHAO C F, HUANG K S, et al. Image-based 3D scene reconstruction and exploration in augmented reality[J]. Automation in Construction, 2013, 33: 48-60.
[3]	MA Z L, LIU S L. A review of 3D reconstruction techniques in civil engineering and their applications[J]. Advanced Engineering Informatics, 2018, 37: 163-174.
[4]	LU Y J, WANG S, FAN S S, et al. Image-based 3D reconstruction for multi-scale civil and infrastructure projects: a review from 2012 to 2022 with new perspective from deep learning methods[J]. Advanced Engineering Informatics, 2024, 59: 102268.
[5]	WANG P, LIU L J, LIU Y, et al. NeuS: learning neural implicit surfaces by volume rendering for multi-view reconstruction[C]// The 35th International Conference on Neural Information Processing System. Red Hook: Curran Associates Inc., 2021: 2081.
[6]	LI Z S, MÜLLER T, EVANS A, et al. Neuralangelo: high-fidelity neural surface reconstruction[C]// 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2023: 8456-8465.
[7]	SUN J M, CHEN X, WANG Q Q, et al. Neural 3D reconstruction in the wild[C]// ACM SIGGRAPH 2022 Conference Proceedings. New York: ACM, 2022: 26.
[8]	HORÉ A, ZIOU D. Image quality metrics:PSNR vs. SSIM[C]// 2010 20th International Conference on Pattern Recognition. New York: IEEE Press, 2010: 2366-2369.
[9]	DELAUNAY B. Sur la sphere vide[J]. Izv Akad Nauk SSSR, Otdelenie Matematicheskii i Estestvennyka Nauk, 1934, 7: 793-800.
[10]	LEE D T, SCHACHTER B J. Two algorithms for constructing a Delaunay triangulation[J]. International Journal of Computer & Information Sciences, 1980, 9(3): 219-242.
[11]	BERNARDINI F, MITTLEMAN J, RUSHMEIER H, et al. The ball-pivoting algorithm for surface reconstruction[J]. IEEE Transactions on Visualization and Computer Graphics, 1999, 5(4): 349-359.
[12]	KAZHDAN M, BOLITHO M, HOPPE H. Poisson surface reconstruction[C]// The 4th Eurographics Symposium on Geometry Processing. Goslar: Eurographics Association, 2006: 61-70.
[13]	HOPPE H, DEROSE T, DUCHAMP T, et al. Surface reconstruction from unorganized points[C]// The 19th Annual Conference on Computer Graphics and Interactive Techniques. New York: ACM, 1992: 71-78.
[14]	LORENSEN W E, CLINE H E. Marching cubes: a high resolution 3D surface construction algorithm[C]// The 14th Annual Conference on Computer Graphics and Interactive Techniques. New York: ACM, 1987: 163-169.
[15]	SCHÖNBERGER J L, FRAHM J M. Structure-from-motion revisited[C]// 2016 IEEE Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2016: 4104-4113.
[16]	SCHÖNBERGER J L, ZHENG E L, FRAHM J M, et al. Pixelwise view selection for unstructured multi-view stereo[C]// The 14th European Conference on Computer Vision. Cham: Springer, 2016: 501-518.
[17]	MÜLLER T, EVANS A, SCHIED C, et al. Instant neural graphics primitives with a multiresolution hash encoding[J]. ACM Transactions on Graphics, 2022, 41(4): 102.
[18]	LI R L, TANCIK M, KANAZAWA A. NerfAcc: a general NeRF acceleration toolbox[EB/OL]. [2024-03-01]. https://arxiv.org/abs/2210.04847.
[19]	SUN C, SUN M, CHEN H T. Direct voxel grid optimization: super-fast convergence for radiance fields reconstruction[C]// 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2022: 5449-5459.
[20]	FRIDOVICH-KEIL S, YU A, TANCIK M, et al. Plenoxels: radiance fields without neural networks[C]// 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2022: 5491-5500.
[21]	GARBIN S J, KOWALSKI M, JOHNSON M, et al. FastNeRF: high-fidelity neural rendering at 200FPS[C]// 2021 IEEE/CVF International Conference on Computer Vision. New York: IEEE Press, 2021: 14326-14335.
[22]	BARRON J T, MILDENHALL B, TANCIK M, et al. Mip-NeRF: a multiscale representation for anti-aliasing neural radiance fields[C]// 2021 IEEE/CVF International Conference on Computer Vision. New York: IEEE Press, 2021: 5835-5844.
[23]	PARK J J, FLORENCE P, STRAUB J, et al. DeepSDF: learning continuous signed distance functions for shape representation[C]// 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2019: 165-174.
[24]	YARIV L, KASTEN Y, MORAN D, et al. Multiview neural surface reconstruction by disentangling geometry and appearance[C]// The 34th International Conference on Neural Information Processing Systems. Red Hook: Curran Associates Inc., 2020: 2492-2502.
[25]	WILLIAMS F, GOJCIC Z, KHAMIS S, et al. Neural fields as learnable kernels for 3D reconstruction[C]// 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2022: 18479-18489.
[26]	WILLIAMS F, TRAGER M, BRUNA J, et al. Neural splines: fitting 3D surfaces with infinitely-wide neural networks[C]// 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2021: 9944-9953.
[27]	HUANG J H, GOJCIC Z, ATZMON M, et al. Neural kernel surface reconstruction[C]// 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2023: 4369-4379.
[28]	WANG Y M, HAN Q, HABERMANN M, et al. NeuS2: fast learning of neural implicit surfaces for multi-view reconstruction[C]// 2023 IEEE/CVF International Conference on Computer Vision. New York: IEEE Press, 2023: 3272-3283.
[29]	JENSEN R, DAHL A, VOGIATZIS G, et al. Large scale multi-view stereopsis evaluation[C]// 2014 IEEE Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2014: 406-413.
[30]	KINGMA D P, BA J. Adam: a method for stochastic optimization[EB/OL]. [2024-04-01]. https://arxiv.org/abs/1412.6980.
[31]	WANG Z, BOVIK A C, SHEIKH H R, et al. Image quality assessment: from error visibility to structural similarity[J]. IEEE Transactions on Image Processing, 2004, 13(4): 600-612. DOI PMID
[32]	KNAPITSCH A, PARK J, ZHOU Q Y, et al. Tanks and temples: benchmarking large-scale scene reconstruction[J]. ACM Transactions on Graphics, 2017, 36(4): 78.