Abstract: Aiming at the problems of high false detection rate and poor edge continuity in current video shadow detection algorithms in mobile view, a real-time shadow detection algorithm based on track and detection framework was proposed. Firstly, the overlapped shadow parts of the two frames were tracked twice, the tracking points with larger error were filtered by forward and backward tracking, and the accuracy of the tracking edge was ensured by Canny edge confidence. Then, the new region to be detected was obtained by the region division method based on optical flow. Secondly, seven-dimensional feature vectors were constructed for texture edge error detection, soft shadow detection, and dark area error detection. Then the support vector machine (SVM) classifier was trained by extracting feature vectors from shadow edge, and the trained classifier was employed to detect the shadow in the new area. Finally, for the broken edges in the detection results, an algorithm based on RGB color space gradient direction consistency was proposed to optimize the connection of the broken edge. Experimental results show that the proposed algorithm exhibits the best comprehensive performance compared with the latest research results, and is superior to the existing methods in terms of detection accuracy and edge continuity. 

Key words: augmented reality, optical flow tracking, shadow detection, moving viewpoint, shadow interaction