欢迎访问《图学学报》 分享到：
图学学报

图学学报 ›› 2022, Vol. 43 ›› Issue (3): 404-412.DOI: 10.11996/JG.j.2095-302X.2022030404

• 图像处理与计算机视觉 • 上一篇    下一篇

模拟真实场景的场景流预测

  

  1. 1. 山东大学控制科学与工程学院，山东 济南 250061；
    2. 中国科学院自动化研究所，北京 100190；
    3. 中国科学院大学人工智能学院，北京 100049
  • 出版日期:2022-06-30 发布日期:2022-06-28
  • 基金资助:
    国家重点研究计划项目(2016YFA0100900，2016YFA0100902)；NSFC-山东联合基金项目(U1806202)；国家自然科学基金项目(81871442，61876178，61806196，61976229，61872367)；中国科学院青年创新促进会项目(Y201930)

Scene flow prediction with simulated real scenarios

  1. 1. School of Control Science and Engineering, Shandong University, Jinan Shandong 250061, China;
    2. Institute of Automation, Chinese Academy of Sciences, Beijing 100190, China;
    3. School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing 100049, China
  • Online:2022-06-30 Published:2022-06-28
  • Supported by:
    National Key Research Programs of China (2016YFA0100900, 2016YFA0100902); Natural Science Foundation of China Under Grant
    (U1806202); Chinese National Natural Science Foundation Projects (81871442, 61876178, 61806196, 61976229, 61872367); Youth
    Innovation Promotion Association CAS (Y201930)

PDF (PC)

71

摘要：

人工智能发展至今正逐渐进入认知时代，计算机对真实物理世界的认知与推理能力亟待提高。有关物体物理属性与运动预测的现有工作多局限于简单的物体和场景，因此尝试拓展常识推理至仿真场景下物体场景流的预测。首先，为了弥补相关领域数据集的短缺，提出了一个基于仿真场景的数据集 ModernCity，从常识推理的角度出发还原了现代都市的街边景象，并提供了包括 RGB 图像、深度图、场景流数据和语义分割图在内的多种标签；此外，设计了一个物体描述子解码模型(ODD)，通过物体属性辅助预测场景流。通过消融实验证明，该模型可以在仿真的场景下通过物体的属性准确地预测物体未来的运动趋势，通过与其他 SOTA 模型的对比实验验证了该模型的性能及 ModernCity 数据集的可靠性。

关键词: 常识推理, 场景流, 仿真场景, 物体物理属性, 运动预测

Abstract:

Artificial intelligence is stepping into the age of cognition, the ability of cognizing and inferring the physical world for machines needs to be improved. Recent works about exploring the physical properties of objects and predicting the motion of objects are mostly constrained by simple objects and scenes. We attempted to predict the scene flow of objects in simulated scenarios to extend common sense cognizing. First, due to the lack of data in the related field, a dataset called ModernCity based on simulated scenarios is proposed, which contains the street scene of modern cities designed from the perspective of cognizing common sense, and provides RGB images, depth maps, scene flow, and semantic segmentations. In addition, we design an object descriptor decoder (ODD) to predict the scene flow through the properties of the objects. The model we proposed is proved to have the ability to predict future motion accurately through the properties of objects in simulated scenarios by experiments. The comparison experiment with other SOTA models demonstrates the performance of the model and the reliability of the ModernCity dataset.

Key words: common sense cognizing, scene flow, simulated scenarios, properties of objects, motion prediction

中图分类号: