欢迎访问《图学学报》 分享到：
图学学报

图学学报

• 图像处理与计算机视觉 • 上一篇    下一篇

基于 CNN 的彩色图像引导的深度图像超分辨率重建

  

  1. (1. 武汉科技大学信息科学与工程学院，湖北 武汉 430080； 
    2. 武汉科技大学冶金自动化与检测技术教育部工程研究中心，湖北 武汉 430080)
  • 出版日期:2020-04-30 发布日期:2020-05-15
  • 基金资助:
    国家自然科学基金项目(61702384，61502357)；湖北省自然科学基金项目(2015CFB365)

Super-resolution reconstruction of depth image guided by color image  based on CNN

  1. (1. College of Information Science and Engineering, Wuhan University of Science and Technology, Wuhan Hubei 430080, China; 
    2. Engineering Research Center of Ministry of Education of Metallurgical Automation and Testing Technology,  Wuhan University of Science and Technology, Wuhan Hubei 430080, China)
  • Online:2020-04-30 Published:2020-05-15

PDF (PC)

271

摘要： 深度图像表达了三维场景内物体之间的相对距离信息，根据深度图像表达的信息， 人们能够准确的获得物体在空间中的位置以及不同物体之间的相对距离，使得深度图像在立体 视觉等领域有着广泛的应用。然而受 RGB-D 传感器硬件条件的限制，获取的深度图像分辨率 低，无法满足一些具有高精度要求的实际应用的需求。近年来深度学习特别是卷积神经网络 (CNN)在图像处理方面获得了非常大的成功。为此提出了一种基于 CNN 的彩色图像引导的深度 图像超分辨率重建。首先，利用 CNN 学习彩色图像的边缘特征信息与深度图像的深度特征信 息，获得边缘纹理清晰的高分辨率深度图像；再通过不同大小尺寸滤波核的卷积层，进一步优 化深度图像的边缘纹理细节，获得更高质量的高分辨率深度图像。实验结果表明，相较于其他 方法，该方法 RMSE 值更低，重建的图像也能更好的恢复图像边缘纹理细节。

关键词: 超分辨率重建, 深度图像, 深度信息, 卷积神经网络, 深度学习

Abstract: The depth image indicates the relative distance between the objects in the three-dimensional scene. According to the information expressed by the depth image, the position of the object in space and the relative distance between different objects can be accurately obtained, so that the depth image has a wide range of applications in areas such as stereo vision. However, due to the limitations of RGB-D sensor hardware, the acquired resolution of depth image is low, which cannot meet the requirements of some practical applications with high precision. Deep learning, especially the convolutional neural networks (CNN), has achieved great success in image processing in recent years. In this light, this paper proposes a super-resolution reconstruction method of depth image guided by color image based on CNN. First, the CNN are used to obtain the edge feature information of color images and the depth feature information of depth images, so as to obtain high-resolution depth images with clear edge texture. Then, the edge texture details of the depth image are further optimized by the convolution layer of the filter kernels of different sizes, so as to obtain the depth image with higher resolution. The experimental results show that the RMSE value of the method proposed is lower than that of other methods, and the reconstructed image shows clearer edge texture details.

Key words:  super-resolution, depth images, depth information, convolution neural networks, deep learning