Welcome to Journal of Graphics share: 

Journal of Graphics ›› 2022, Vol. 43 ›› Issue (5): 841-848.DOI: 10.11996/JG.j.2095-302X.2022050841

• Image Processing and Computer Vision • Previous Articles     Next Articles

Self-supervised optical flow estimation with attention module 

  

  1. 1. School of Artificial Intelligence, Suzhou Industrial Park Institute of Services Outsourcing, Suzhou Jiangsu 215123, China;  2. School of Economics & Management, Tongji University, Shanghai 210092, China
  • Online:2022-10-31 Published:2022-10-28
  • Supported by:
    National Natural Science Foundation of China (71272048); Jiangsu “Qing Lan Project” ([2020] 10) 

Abstract:

Optical flow estimation is the key module of many computer vision systems, which is widely utilized in motion recognition, robot positioning, and navigation. However, due to the absence of labeled optical flow datasets of real scenes, synthetic datasets were used as the main training data sources, and synthetic data could not fully represent real scenes (such as leaf movement and pedestrian reflection). Unsupervised or self-supervised methods could employ a large amount of video data for training, and at the same time facilitate fine-tuning of supervised training, which was an effective way to solve the lack of datasets. In this paper, a self-supervised learning optical flow calculation network was constructed, in which the “Teacher” module and the “Student” module adopted sparse correlation volume (SCV) network to reduce the redundancy of correlation computation, and the attention model was introduced as a node of the network, in order to enhance the dimension attribute of image feature in terms of channel and space. This paper marks the first endeavor to implement a self-supervised optical flow computing network based on SCV. The test results on the KITTI 2015 dataset could reach or outperform those of the common supervised training networks such as FlowNet and LightFlowNet. 

Key words: optical flow estimation, self-supervised learning, convolutional block attention module, spatial/channel attention, sparse correlation volume

CLC Number: