Abstract: The depth image indicates the relative distance between the objects in the three-dimensional scene. According to the information expressed by the depth image, the position of the object in space and the relative distance between different objects can be accurately obtained, so that the depth image has a wide range of applications in areas such as stereo vision. However, due to the limitations of RGB-D sensor hardware, the acquired resolution of depth image is low, which cannot meet the requirements of some practical applications with high precision. Deep learning, especially the convolutional neural networks (CNN), has achieved great success in image processing in recent years. In this light, this paper proposes a super-resolution reconstruction method of depth image guided by color image based on CNN. First, the CNN are used to obtain the edge feature information of color images and the depth feature information of depth images, so as to obtain high-resolution depth images with clear edge texture. Then, the edge texture details of the depth image are further optimized by the convolution layer of the filter kernels of different sizes, so as to obtain the depth image with higher resolution. The experimental results show that the RMSE value of the method proposed is lower than that of other methods, and the reconstructed image shows clearer edge texture details.

Key words:  super-resolution, depth images, depth information, convolution neural networks, deep learning