欢迎访问《图学学报》 分享到:

图学学报 ›› 2023, Vol. 44 ›› Issue (1): 67-76.DOI: 10.11996/JG.j.2095-302X.2023010067

• 图像处理与计算机视觉 • 上一篇    下一篇

草图引导的选择循环推理式人脸图像修复网络

邵英杰1,2(), 尹辉1,2(), 谢颖1,2, 黄华1,2   

  1. 1.北京交通大学计算机与信息技术学院,北京 100044
    2.北京交通大学交通数据分析与挖掘北京市重点实验室,北京 100044
  • 收稿日期:2022-06-19 修回日期:2022-07-20 出版日期:2023-10-31 发布日期:2023-02-16
  • 通讯作者: 尹辉
  • 作者简介:邵英杰(1998-),男,硕士研究生。主要研究方向为计算机视觉、深度学习。E-mail:906612726@qq.com
  • 基金资助:
    北京市教育委员会科技重大项目(KJZD20191000402);国家自然科学基金项目(51827813);国家自然科学基金项目(61472029)

A sketch-guided facial image completion network via selective recurrent inference

SHAO Ying-jie1,2(), YIN Hui1,2(), XIE Ying1,2, HUANG Hua1,2   

  1. 1. School of Computer and Information Technology, Beijing Jiaotong University, Beijing 100044, China
    2. Beijing Key Laboratory of Traffic Data Analysis and Mining, Beijing Jiaotong University, Beijing 100044, China
  • Received:2022-06-19 Revised:2022-07-20 Online:2023-10-31 Published:2023-02-16
  • Contact: YIN Hui
  • About author:SHAO Ying-jie (1998-), master student. His main research interests cover computer vision and deep learning. E-mail:906612726@qq.com
  • Supported by:
    Major Science and Technology Project of Beijing Municipal Education Commission(KJZD20191000402);National Natural Science Foundation of China(51827813);National Natural Science Foundation of China(61472029)

摘要:

图像修复在修复老照片、消除人脸马赛克等应用中起到关键作用。针对现有深度学习人脸图像修复方法因受干扰信息影响,存在编解码器修复效果欠佳,修复结果因概率多样性出现偏离用户预期等问题。提出了一种草图引导的选择循环推理式人脸图像修复网络,通过设计一种选择循环推理策略,在循环网络中引入选择机制降低干扰信息对编解码的推理影响,并在编码器和解码器之间的跳跃连接中加入基于草图的结构信息修正模块,从而限制修复结果相对于用户期望的结构偏离。在CelebA-HQ数据集上的实验结果表明,该方法在评价指标和引导生成用户期望内容方面均优于其他经典网络。在人工手绘草图上的实验结果表明,可以通过简单的手绘方式生成用户指定的内容,具有一定的实际应用意义。

关键词: 人脸, 图像修复, 深度学习, 草图

Abstract:

Image inpainting plays a key role in applications such as inpainting old photos and removing face mosaics. There are many problems in the existing deep learning-based inpainting models, such as interference information erroneously affecting the encoder and decoder in generating the result, which weakens inpainting quality, and the probabilistic diversity leading to deviation from users' expectations. To address the problems, a sketch-guided facial image completion network was proposed via selective recurrent inference. A selective recurrent inferential strategy was designed at first. A selection mechanism was introduced to solve the influence of erroneous interference on the inference of the encoder and decoder. Then a sketch-based structural information correction module was added to the skip connection between the encoder and decoder, thereby limiting the deviation of the repair results from users' expected structure. Experimental results on the CelebA-HQ dataset show that the proposed method could outperform other classical network models in terms of evaluation indicators and guidance for generating user-expected content. The experimental results on the manually drawn sketches show that user-specified content could be generated by simple hand-drawn methods, exhibiting certain significance in practical application.

Key words: human face, image inpainting, deep learning, sketch image

中图分类号: