Visual analysis system for UAV path planning

doi:10.11996/JG.j.2095-302X.2025030655

Abstract

Abstract:

Currently, real-world image information have been used to reconstruct geometric models and generate high-quality rendered outcomes using image-based rendering methods, which has become a viable solution for acquiring high-quality art materials. A visual analysis system was designed to support UAV path planning, 3D reconstruction, and image-based rendering. In terms of engineering contributions, a blueprint editor was employed as the platform for this visual analysis system, wherein various visual analysis functions were implemented as function nodes within the editor, thereby enabling users to personalize their visual analysis workflow by simply dragging, connecting, and configuring graphic elements. In terms of algorithm innovation, an enhanced and optimized view selection algorithm based on sampling coverage was proposed. This algorithm offered improved results in terms of both algorithmic time cost and rendering coverage.

Key words: data visualization, unmanned aerial vehicles, 3D reconstruction, image-based render, visual analysis, path planning

CLC Number:

TP391
V279

HU Yue, SUN Zhida, HUANG Hui. Visual analysis system for UAV path planning[J]. Journal of Graphics, 2025, 46(3): 655-665.

Figures/Tables 14

Fig. 1 The relationship among drone data acquisition, 3D reconstruction, and image-based rendering

Fig. 2 The overall picture quality based on image rendering is affected by the absence of reference pixels

Fig. 3 Users can add functional nodes to perform any visual analysis task

Fig. 4 EDL and SSAO render effect comparison ((a) EDL; (b) SSAO)

Fig. 5 Rendering pipeline in visual analysis system

Fig. 6 In process of visual analysis of the 3D reconstruction, the relevant requirements (R) correspond to the goal (T) to be achieved, and the internal logic between the different requirements

Fig. 7 In process of the visual analysis of IBR render quality, the related requirements (R) correspond to the objectives (T) achieved, and the internal logic between the different requirements

Fig. 8 The preprocessing of point clouds can enhance the selection effect of perspectives ((a) Greedy method can not guarantee the selection of reference view combination is good enough; (b) The optimized point cloud as input can assist the algorithm to select a better reference view combination)

Fig. 9 Use unreferenced pixels in the rendering process to optimize the point cloud

Fig. 10 The visibility bitmap can be obtained by calculating whether the surface point cloud is visible in the input scene at each shooting point of the UAV

Fig. 11 Case studies processes

Table 1 Comparative experimental results

优化方案	场景类别	点云规模/万	平均可见性计算开销/ms	优/%	良/%	差/%	平均像素遮挡率/%
无优化	学校	10	5.4	76.18	22.09	1.73	0.111
无优化	小镇	10	7.3	88.92	10.89	0.18	0.043
可见性非精确计算	学校	100	1.1	79.43	19.81	0.77	0.082
可见性非精确计算	小镇	255	1.2	93.18	6.74	0.08	0.029
非均匀点云优化	学校	100	1.1	81.61	18.01	0.38	0.068
非均匀点云优化	小镇	255	1.2	93.70	6.22	0.08	0.029
采集视角补充	学校	100	1.1	86.04	13.80	0.16	0.046
采集视角补充	小镇	255	1.2	93.72	6.22	0.06	0.026

Fig. 12 The rendering case of different schemes is optimized

Table 2 Results compared with other work

方法	平均视角选择时间开销/ms	是否支持稀疏参考视角集	平均像素遮挡率/%
文献[14]	0.012	否	/
文献[16]	8.900	是	1.410
文献[19]	10.400	是	0.111
Ours	6.200	是	0.068

References 22

[1]	SHUM H, KANG S B. Review of image-based rendering techniques[EB/OL]. [2024-05-11]https://www.spiedigitallibrary.org/conference-proceedings-of-spie/4067/1/Review-of-image-based-rendering-techniques/10.1117/12.386541.short.
[2]	SCHMID K, HIRSCHMÜLLER H, DÖMEL A, et al. View planning for multi-view stereo 3D reconstruction using an autonomous multicopter[J]. Journal of Intelligent & Robotic Systems, 2012, 65(1/4): 309-323.
[3]	ZHANG H, YAO Y C, XIE K, et al. Continuous aerial path planning for 3D urban scene reconstruction[J]. ACM Transactions on Graphics, 2021, 40(6): 225.
[4]	ZHOU X H, XIE K, HUANG K, et al. Offsite aerial path planning for efficient urban scene reconstruction[J]. ACM Transactions on Graphics, 2020, 39(6): 192.
[5]	SMITH N, MOEHRLE N, GOESELE M, et al. Aerial path planning for urban scene reconstruction: a continuous optimization method and benchmark[J]. ACM Transactions on Graphics, 2018, 37(6): 183.
[6]	KNAPITSCH A, PARK J, ZHOU Q Y, et al. Tanks and temples: benchmarking large-scale scene reconstruction[J]. ACM Transactions on Graphics, 2017, 36(4): 78.
[7]	SEITZ S M, CURLESS B, DIEBEL J, et al. A comparison and evaluation of multi-view stereo reconstruction algorithms[C]// 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2006: 519-528.
[8]	LIU Y L, LIN L Q, HU Y, et al. Learning reconstructability for drone aerial path planning[J]. ACM Transactions on Graphics, 2022, 41(6): 197.
[9]	LEVOY M, HANRAHAN P. Light field rendering[C]// The 23rd Annual Conference on Computer Graphics and Interactive Techniques. New York: ACM, 1996: 31-42.
[10]	GORTLER S J, GRZESZCZUK R, SZELISKI R, et al. The lumigraph[C]// The 23rd Annual Conference on Computer Graphics and Interactive Techniques. New York: ACM, 1996: 43-54.
[11]	DEBEVEC P E, TAYLOR C J, MALIK J. Modeling and rendering architecture from photographs: a hybrid geometry- and image-based approach[C]// The 23rd Annual Conference on Computer Graphics and Interactive Techniques. New York: ACM, 1996: 11-20.
[12]	BUEHLER C, BOSSE M, MCMILLAN L, et al. Unstructured lumigraph rendering[C]// The 28th Annual Conference on Computer Graphics and Interactive Techniques. New York: ACM, 2001: 425-432.
[13]	HEDMAN P, RITSCHEL T, DRETTAKIS G, et al. Scalable inside-out image-based rendering[J]. ACM Transactions on Graphics, 2016, 35(6): 231.
[14]	XU J M, WU X C, ZHU Z H, et al. Scalable image-based indoor scene rendering with reflections[J]. ACM Transactions on Graphics, 2021, 40(4): 60.
[15]	HEDMAN P, PHILIP J, PRICE T, et al. Deep blending for free-viewpoint image-based rendering[J]. ACM Transactions on Graphics, 2018, 37(6): 257.
[16]	RIEGLER G, KOLTUN V. Free view synthesis[C]// The 16th European Conference on Computer Vision. Cham: Springer, 2020: 623-640.
[17]	RIEGLER G, KOLTUN V. Stable view synthesis[C]// 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2021: 12211-12220.
[18]	PARK J J, FLORENCE P, STRAUB J, et al. DeepSDF: learning continuous signed distance functions for shape representation[C]// 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2019: 165-174.
[19]	YI Z M, XIE K, LYU J H, et al. Where to render: studying renderability for IBR of large-scale scenes[C]// 2023 IEEE Conference on Virtual Reality and 3D User Interfaces. New York: IEEE Press, 2023: 356-366.
[20]	BOUCHENY C. Interactive scientific visualization of large datasets: towards a perceptive-based approach[D]. Grenoble: Université Joseph Fourier, 2009.
[21]	BAVOIL L, SAINZ M. Screen space ambient occlusion[EB/OL]. (2008-09-01) [2024-10-03]https://developper.download.nvidia.com/SDK/10.5/direct3d/Source/ScreenSpaceAO/doc/ScreenSpaceAO.pdf.
[22]	LIN L Q, LIU Y L, HU Y, et al. Capturing, reconstructing, and simulating: the UrbanScene3D dataset[C]// The 17th European Conference on Computer Vision. Cham: Springer, 2022: 93-109.