欢迎访问《图学学报》 分享到：
图学学报

图学学报 ›› 2022, Vol. 43 ›› Issue (1): 70-78.DOI: 10.11996/JG.j.2095-302X.2022010070

• 图像处理与计算机视觉 • 上一篇    下一篇

基于上下文门卷积的盲图像修复

  

  1. 1. 上海大学上海电影学院，上海 200072；  2. 上海电影特效工程技术研究中心，上海 200072
  • 出版日期:2022-02-28 发布日期:2022-02-16
  • 基金资助:
    国家自然科学基金项目(61303093，61402278)

Blind image inpainting based on context gated convolution

  1. 1. Shanghai Film Academy, Shanghai University, Shanghai 200072, China;  2. Shanghai Engineering Research Center of Motion Picture Special Effects, Shanghai 200072, China
  • Online:2022-02-28 Published:2022-02-16
  • Supported by:
    National Natural Science Foundation of China (61303093, 61402278) 

PDF (PC)

172

摘要： 目前基于深度学习的图像修复方法已经取得较大地进展，其方法均是基于输入的掩模对图像的 退化区域进行修复。基于此，提出了由掩模预测网络和图像修复网络组成的 2 阶段盲图像修复网络。整个修复 过程无需输入掩模，掩模预测网络可以根据输入图像自动检测图像退化区域并生成掩模，图像修复网络根据预 测掩模对输入图像的缺失部分进行修复。为了更好地利用全局上下文信息，基于上下文门卷积设计了一个上下 文门残差块(CGRB)模块来提取特征信息。另外，还提出了空间注意力残差块(SARB)对远距离图像像素的关系 进行建模，过滤了一些无关的细节信息。在 CelebA-HQ，FFHQ 和 PairsStreet 数据集上的大量实验结果表明， 该改进算法优于其他对比方法，且能生成令人信服的图像。

关键词: 图像修复, 盲图像修复, 上下文门卷积, 上下文门残差块, 空间注意力残差块

Abstract: Image inpainting methods based on deep learning have achieved great progress. At present, most of the image inpainting methods use the input mask to reconstruct the degraded areas of the image. Based on this observation, a two-stage blind image inpainting network was proposed, comprising a mask prediction network and an image inpainting network. The input of a mask was not required in the whole inpainting process. The mask prediction network could automatically detect the degraded area of the image and generate a mask according to the input image, and the image inpainting network could restore the missing part of the input image based on the prediction mask. In order to make better use of global context information, a context-gated residual block (CGRB) module was designed based on context-gated convolution to extract feature information. In addition, the spatial attention residual block (SARB) was proposed to model the relationship between pixels in the long-distance image, filtering some irrelevant details. A large number of experimental results on the CelebA-HQ, FFHQ, and PairsStreet datasets show that the improved algorithm is superior to other comparison methods and can generate convincing images. 

Key words: image inpainting, blind image inpainting, context-gated convolution, context-gated residual block, spatial attention residual block 

中图分类号: