欢迎访问《图学学报》 分享到:

图学学报 ›› 2023, Vol. 44 ›› Issue (6): 1091-1103.DOI: 10.11996/JG.j.2095-302X.2023061091

• 综述 • 上一篇    下一篇

面向自动驾驶场景的神经辐射场综述

成欢1(), 王硕2, 李孟2, 秦伦明1(), 赵芳3   

  1. 1.上海电力大学电子与信息工程学院,上海 200050
    2.中国科学院上海微系统与信息技术研究所,上海 200050
    3.神鳍科技(上海)有限公司,上海 200050
  • 收稿日期:2023-06-27 接受日期:2023-09-08 出版日期:2023-12-31 发布日期:2023-12-17
  • 通讯作者: 秦伦明(1983-),男,副教授,博士。主要研究方向为计算机视觉与图像分割等。E-mail:lunming.qin@shiep.edu.cn
  • 作者简介:

    成欢(1999-),女,硕士研究生。主要研究方向为计算机视觉与计算机图形学。E-mail:chenghuan0116@gmail.com

  • 基金资助:
    国家重点研发计划项目(2021YFF1200700)

A review of neural radiance field for autonomous driving scene

CHENG Huan1(), WANG Shuo2, LI Meng2, QIN Lun-ming1(), ZHAO Fang3   

  1. 1. School of Electronic and Information Engineering, Shanghai University of Electric Power, Shanghai 200050, China
    2. Shanghai Institute of Microsystem and Information Technology, Chinese Academy of Sciences, Shanghai 200050, China
    3. DiFint Technology (Shanghai) Co., Shanghai 200050, China
  • Received:2023-06-27 Accepted:2023-09-08 Online:2023-12-31 Published:2023-12-17
  • Contact: QIN Lun-ming (1983-), associate professor, Ph.D. His main research interests cover computer vision and image segmentation, etc. E-mail:lunming.qin@shiep.edu.cn
  • About author:

    CHENG Huan (1999-), master student. Her main research interests cover computer vision and computer graphics.
    E-mail:chenghuan0116@gmail.com

  • Supported by:
    National Key Research and Development Program(2021YFF1200700)

摘要:

神经辐射场(NeRF)是一种可用于重建真实的视觉效果以及合成新颖视角的关键技术,其主要是通过摄像机捕获的二维图像数据来渲染合成三维场景,将已知视角推理到未知视角下使得用户可从不同视角观看合成视图,增强人机交互感。作为一种新颖视角合成的三维重建方法,神经辐射场技术在机器人、自动驾驶、虚拟现实和数字孪生等领域具有重要的研究与应用价值。通过NeRF技术与自动驾驶场景相结合,可实现对复杂驾驶场景的高质量重建,并模拟恶劣情况下的不同场景,从而丰富自动驾驶的训练数据,以较低成本提高自动驾驶系统的准确性和安全性,并验证自动驾驶算法的有效性。鉴于目前NeRF在自动驾驶场景有重要的应用前景以及现有的相关综述较少,首先,从传统的显式三维场景表征方法出发,引入场景的隐式表征方法——NeRF,并介绍了NeRF技术的原理;其次,对于NeRF与自动驾驶场景相结合所面临的挑战进行了探讨和分析,其中包括稀疏视角重建、大尺度场景重建、运动场景、训练加速以及合成自动驾驶场景的问题;最后,对NeRF技术进行总结,以及展望其未来发展方向。

关键词: 神经辐射场, 视图合成, 自动驾驶, 三维重建, 场景重建

Abstract:

The neural radiance field (NeRF) is a crucial technology for reconstructing realistic visual effects and synthesizing novel views of scenes. It primarily renders synthetic 3D scenes based on 2D image data captured by cameras, inferring known views to unknown views, so that users can observe synthetic views from different viewpoints to enhance human-computer interaction. As a method for novel viewpoint synthesis and 3D reconstruction, NeRF exhibited significant research and application value in the fields of robotics, autonomous driving, virtual reality, and digital twins. Its integration with autonomous driving scenarios allowed for high-quality reconstruction of complex driving scenes and the simulation of different scenes under adverse conditions. This could enrich training data for autonomous driving systems, enhance their accuracy and safety at minimal costs, and validate the effectiveness of autonomous driving algorithms. Given NeRF's current important application prospect in autonomous driving scenes and its limited coverage in existing reviews, firstly, the traditional explicit 3D scene representation method was employed to introduce the implicit representation method of scenes, namely NeRF, along with the introduction of the principle of NeRF technology. Secondly, the discussion analysis were conducted regarding the challenges encountered when combining NeRF and autonomous driving scenes, including the problems of sparse view reconstruction, large-scale scene reconstruction, motion scenes, training acceleration, and the synthesis of autonomous driving scenes. Finally, insights were provided into the future development directions of NeRF technology.

Key words: neural radiance field, view synthesis, autonomous driving, 3D reconstruction, scene reconstruction

中图分类号: