Welcome to Journal of Graphics share: 

Journal of Graphics ›› 2022, Vol. 43 ›› Issue (3): 404-412.DOI: 10.11996/JG.j.2095-302X.2022030404

• Image Processing and Computer Vision • Previous Articles     Next Articles

Scene flow prediction with simulated real scenarios

  

  1. 1. School of Control Science and Engineering, Shandong University, Jinan Shandong 250061, China;
    2. Institute of Automation, Chinese Academy of Sciences, Beijing 100190, China;
    3. School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing 100049, China
  • Online:2022-06-30 Published:2022-06-28
  • Supported by:
    National Key Research Programs of China (2016YFA0100900, 2016YFA0100902); Natural Science Foundation of China Under Grant
    (U1806202); Chinese National Natural Science Foundation Projects (81871442, 61876178, 61806196, 61976229, 61872367); Youth
    Innovation Promotion Association CAS (Y201930)

Abstract:

Artificial intelligence is stepping into the age of cognition, the ability of cognizing and inferring the physical world for machines needs to be improved. Recent works about exploring the physical properties of objects and predicting the motion of objects are mostly constrained by simple objects and scenes. We attempted to predict the scene flow of objects in simulated scenarios to extend common sense cognizing. First, due to the lack of data in the related field, a dataset called ModernCity based on simulated scenarios is proposed, which contains the street scene of modern cities designed from the perspective of cognizing common sense, and provides RGB images, depth maps, scene flow, and semantic segmentations. In addition, we design an object descriptor decoder (ODD) to predict the scene flow through the properties of the objects. The model we proposed is proved to have the ability to predict future motion accurately through the properties of objects in simulated scenarios by experiments. The comparison experiment with other SOTA models demonstrates the performance of the model and the reliability of the ModernCity dataset.

Key words: common sense cognizing, scene flow, simulated scenarios, properties of objects, motion prediction

CLC Number: