面向开放道路场景的实时稠密建图研究

doi:10.11996/JG.j.2095-302X.2023040747

摘要/Abstract

摘要：

针对智能驾驶领域建图效率低、建图精度差的问题，提出一种基于多传感器融合且用于室外开放道路场景的两阶段稠密建图算法。算法由外参实时标定模块和建图模块组成，外参实时标定模块基于道路场景中典型语义和几何特征构建约束并进行优化，实现对传感器间外参的实时在线标定；建图模块的核心在于一种两阶段增量式建图算法，根据智能驾驶中对不同区域的建图精度要求，先后分别对整个场景以及路面区域进行逐帧增量式粗糙建图和抽帧精细建图，粗糙建图保证了算法的实时性，精细建图实现了对交通标志等路面表面纹理的精确还原。在室外开放道路场景下进行实验，实验结果表明，该算法能在室外大尺度场景下进行实时稠密建图，且建图精度和效率较高。

关键词: 开放道路场景, 实时稠密建图, 多传感器融合, 外参标定, 增量式建图

Abstract:

In order to tackle the problems of inefficiency and inaccurate mapping prevalent in the field of intelligent driving, a two-stage dense mapping algorithm for outdoor open road scenes based on multi-sensor fusion was proposed. The proposed algorithm comprised an extrinsic parameter real-time calibration module and a mapping module. The former constructed constraints and optimized them based on typical semantic and geometric features in road scenes, achieving real-time online calibration of extrinsic parameters between sensors. The latter’s core was a two-stage incremental mapping algorithm that performed incremental coarse mapping and fine mapping for the entire scene and road face area, respectively. The rough mapping could guarantee the real-time performance of the algorithm, and fine mapping could achieve accurate restoration of road surface textures such as traffic signs. Experimental results in an outdoor open road scene demonstrated that the proposed algorithm could perform real-time dense mapping in outdoor large-scale scenes, with high accuracy and efficiency.

Key words: open road scene, real-time dense reconstruction, multi-sensor fusion, extrinsic calibration, incremental mapping

中图分类号:

TP391

李新利, 毛昊, 王武, 杨国田. 面向开放道路场景的实时稠密建图研究[J]. 图学学报, 2023, 44(4): 747-754.

LI Xin-li, MAO Hao, WANG Wu, YANG Guo-tian. Research on real-time dense reconstruction for open road scene[J]. Journal of Graphics, 2023, 44(4): 747-754.

图/表 14

图1 技术路线图

Fig. 1 Methodology diagram

图2 提取的平面和平面交线((a)平面；(b)平面交线)

Fig. 2 Extracted planes and plane intersection lines ((a) Planes; (b) Plane intersection lines)

图3 提取的圆柱和圆柱侧视线((a)圆柱；(b)圆柱侧视线)

Fig. 3 Extracted cylinders and cylinder side-view lines ((a) Cylinders; (b) Cylinder side-view lines)

图4 鸟瞰深度图边缘和路沿激光点云((a)鸟瞰深度图边缘；(b)路沿激光点云)

Fig. 4 Edge of depth map and curb point cloud ((a) Edge of depth map; (b) Curb point cloud)

图5 着色后的粗糙点云

Fig. 5 Colorized coarse point cloud

图6 图像投影结果对比((a)优化前；(b)优化后)

Fig. 6 Projection of photos ((a) Before optimization; (b) After optimization)

表1 不同标定算法的平均误差值

Table 1 Mean error of different calibration methods

场景	本文算法	算法1	算法2
场景1	0.045 m / 0.521°	0.052 m / 0.821°	0.076 m / 0.628°
场景2	0.057 m / 0.415°	0.048 m / 0.565°	0.064 m / 0.792°
场景3	0.029 m / 0.404°	0.051 m / 0.223°	0.036 m / 0.636°
场景4	0.038 m / 0.328°	0.047 m / 0.652°	0.030 m / 0.575°
场景5	0.012 m / 0.247°	0.036 m / 0.892°	0.027 m / 0.410°

表2 不同标定算法的平均收敛时间(s)

Table 2 Mean convergence time of different calibration methods (s)

场景	本文方法	算法1	算法2
场景1	0.023	0.036	0.032
场景2	0.034	0.042	0.037
场景3	0.014	0.021	0.019
场景4	0.023	0.026	0.029
场景5	0.032	0.034	0.036

图7 基于不同标定算法的外参投影结果((a)本文算法；(b)算法1；(c)算法2)

Fig. 7 Projection result of different calibration methods ((a) Ours; (b) Method 1; (c) Method 2)

图8 稠密重建结果((a)整个地图；(b)具体场景1；(c)具体场景2；(d)具体场景3)

Fig. 8 Result of dense reconstrution ((a) Whole map; (b) Specific scene 1; (c) Specific scene 2; (d) Specific scene 3)

图9 某道路场景粗糙建图和稠密建图结果((a)粗糙建图结果；(b)稠密建图结果)

Fig. 9 Coarse point cloud and fine point cloud of a road ((a) Coarse point cloud; (b) Fine point cloud)

表3 不同建图算法2次建图的PSNR值

Table 3 PSNR value of different reconstruction methods

场景	本文算法	OpenMVS	Colmap
场景1	40.101	35.237	37.878
场景2	37.920	31.818	33.532
场景3	42.335	36.190	37.977

图10 某场景下各算法建图结果

Fig. 10 Reconstruction result of different methods ((a) Ours; (b) OpenMVS; (c) Colmap)

表4 各算法平均每帧建图时长(s)

Table 4 execution time per frame of different reconstruction methods (s)

场景	本文算法	OpenMVS	Colmap
场景1	1.02	5.87	9.92
场景2	0.93	6.91	13.26
场景3	0.99	6.84	14.73

参考文献 24

[1]	ZHANG J, SINGH S. LOAM: lidar odometry and mapping in real-time[EB/OL]. [2022-09-10]. https://www.ri.cmu.edu/pub_files/2014/7/Ji_LidarMapping_RSS2014_v8.pdf.
[2]	SHAN T X, ENGLOT B. LeGO-LOAM: lightweight and rgound-optimized lidar odometry and mapping on variable terrain[C]// 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems. New York: IEEE Press, 2018: 4758-4765.
[3]	SHAN T X, ENGLOT B, MEYERS D, et al. LIO-SAM: tightly-coupled lidar inertial odometry via smoothing and mapping[C]// 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems. New York: IEEE Press, 2020: 5135-5142.
[4]	FURUKAWA Y, PONCE J. Accurate, dense, and robust multiview stereopsis[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2010, 32(8): 1362-1376. DOI URL
[5]	WU T P, YEUNG S K, JIA J Y, et al. Quasi-dense 3D reconstruction using tensor-based multiview stereo[C]// 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2010: 1482-1489.
[6]	BLEYER M, RHEMANN C, ROTHER C. PatchMatch stereo - stereo matching with slanted support Windows[EB/OL]. [2022-09-10]. http://users.utcluj.ro/-robert/ip/proiect/08_PatchMatchStereo_BMVC2011_6MB.pdf.
[7]	JI M, GALL J, ZHENG H T, et al. SurfaceNet: an end-to-end 3D neural network for multiview stereopsis[C]// 2017 IEEE International Conference on Computer Vision. New York: IEEE Press, 2017: 2326-2334.
[8]	YAO Y, LUO Z X, LI S W, et al. MVSNet: depth inference for unstructured multi-view stereo[EB/OL]. [2022-09-10]. https://openaccess.thecvf.com/content_ECCV_2018/html/Yao_Yao_MVSNet_Depth_Inference_ECCV_2018_paper.html.
[9]	WANG F J H, GALLIANI S, VOGEL C, et al. PatchmatchNet: learned multi-view patchmatch stereo[C]// 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2021: 14189-14198.
[10]	王江安, 庞大为, 黄乐, 等. 基于多尺度特征递归卷积的稠密点云重建网络[J]. 图学学报, 2022, 43(5): 875-883.
	WANG J A, PANG D W, HUANG L, et al. Dense point cloud reconstruction network using multi-scale feature recursive convolution[J]. Journal of Graphics, 2022, 43(5): 875-883 (in Chinese).
[11]	朱攀, 史健勇. 基于AISI网络的BIM三维重建方法研究[J]. 图学学报, 2020, 41(5): 839-846.
	ZHU P, SHI J Y. Research on 3D reconstruction method of BIM based on ASIS network[J]. Journal of Graphics, 2020, 41(5): 839-846 (in Chinese).
[12]	SHAUKAT A, BLACKER P C, SPITERI C, et al. Towards Camera-LIDAR fusion-based terrain modelling for planetary surfaces: review and analysis[J]. Sensors, 2016, 16(11): 1952-1975. DOI URL
[13]	PANDEY G, MCBRIDE J R, SAVARESE S, et al. Automatic extrinsic calibration of vision and lidar by maximizing mutual information[J]. Journal of Field Robotics, 2015, 32(5): 696-722. DOI URL
[14]	LEVINSON J, THRUN S. Automatic online calibration of cameras and lasers[J]. Robotics: Science and Systems, 2013, 2(7): 1-10.
[15]	WANG W M, NOBUHARA S, NAKAMURA R, et al. SOIC: semantic online initialization and calibration for LiDAR and camera[EB/OL]. [2022-09-10]. https://arxiv.53yu.com/pdf/2003.04260.pdf.
[16]	ZHU Y W, ZHENG C R, YUAN C J, et al. CamVox: a low-cost and accurate lidar-assisted visual SLAM system[C]// 2021 IEEE International Conference on Robotics and Automation. New York: IEEE Press, 2021: 5049-5055.
[17]	贾晓辉, 晁晓辉, 刘今越. 固态激光雷达与相机间外参标定方法研究[J]. 激光杂志, 2022, 43(8): 30-36.
	JIA X H, CHAO X H, LIU J Y. Research on extrinsic parameter calibration method between solid-state LiDAR-camera system[J]. Laser Journal, 2022, 43(8): 30-36 (in Chinese).
[18]	CHEN L C, ZHU Y, PAPANDREOU G, et al. Encoder-decoder with atrous separable convolution for semantic image segmentatio-n[EB/OL]. [2022-09-10]. https://openaccess.thecvf.com/content_ECCV_2018/html/Liang-Chieh_Chen_Encoder-Decoder_with_Atrous_ECCV_2018_paper.html.
[19]	XU W, ZHANG F. FAST-LIO: a fast, robust LiDAR-inertial odometry package by tightly-coupled iterated Kalman filter[J]. IEEE Robotics and Automation Letters, 2021, 6(2): 3317-3324. DOI URL
[20]	LIU X Y, YUAN C J, ZHANG F. Targetless extrinsic calibration of multiple small FoV LiDARs and cameras using adaptive voxelization[J]. IEEE Transactions on Instrumentation and Measurement, 2022, 71: 1-12.
[21]	YUAN C J, LIU X Y, HONG X P, et al. Pixel-level extrinsic self calibration of high resolution LiDAR and camera in targetless environments[EB/OL]. [2022-09-10]. https://arxiv.org/abs/2103.01627.
[22]	GUINDEL C, BELTRAN J, MARTIN D, et al. Automatic extrinsic calibration for LiDAR-stereo vehicle sensor setups[C]// IEEE International Conference on Intelligent Transportation Systems. New York: IEEE Press, 2017: 1-6.
[23]	LI S H, XIAO X W, GUO B X, et al. A novel OpenMVS-based texture reconstruction method based on the fully automatic plane segmentation for 3D mesh models[J]. Remote Sensing, 2020, 12(23): 3908-3915. DOI URL
[24]	SCHONBERGER J L, FRAHM J M. Structure-from-motion revisited[C]// IEEE Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2016: 4104-4113.