欢迎访问《图学学报》 分享到：
图学学报

图学学报 ›› 2021, Vol. 42 ›› Issue (2): 190-197.DOI: 10.11996/JG.j.2095-302X.2021020190

• 图像处理与计算机视觉 • 上一篇    下一篇

显著区域保留的图像风格迁移算法

  

  1. 1. 上海师范大学信息与机电工程学院，上海 200234； 2. 上海师范大学上海智能教育大数据工程技术研究中心，上海 200234； 3. 上海市中小学在线教育研究基地，上海 200234； 4. 上海理工大学光电信息与计算机工程学院，上海 200093； 5. 上海交通大学电子信息与电气工程学院，上海 200240
  • 出版日期:2021-04-30 发布日期:2021-04-30
  • 基金资助:
    国家自然科学基金项目(61775139，62072126，61772164，61872242) 

Style transfer algorithm for salient region preservation 

  1. 1. The College of Information, Mechanical and Electrical Engineering, Shanghai Normal University, Shanghai 200234, China; 2. Shanghai Engineering Research Center of Intelligent Education and Bigdata, Shanghai Normal University, Shanghai 200234, China; 3. The Research Base of Online Education for Shanghai Middle and Primary Schools, Shanghai 200234, China; 4. School of Optical-Electrical and Computer engineering, University of Shanghai for Science and Technology, Shanghai 200093, China; 5. School of Electronic Information and Electrical Engineering, Shanghai Jiao Tong University, Shanghai 200240, China
  • Online:2021-04-30 Published:2021-04-30
  • Supported by:
    National Natural Science Foundation of China (61775139，62072126，61772164，61872242)

PDF (PC)

275

摘要： 基于神经网络的风格迁移成为近年来学术界和工业界的热点研究问题之一。现有的方法可以将 不同风格作用在给定的内容图像上生成风格化图像，并且在视觉效果和转换效率上有了较大提升，而侧重学习 图像底层特征容易导致风格化图像丢失内容图像的语义信息。据此提出了使风格化图像与内容图像的显著区域 保持一致的改进方案。通过加入显著性检测网络生成合成图像和内容图像的显著图，在训练过程中计算两者的 损失，使合成图像保持与内容图像相一致的显著区域，这有助于提高风格化图像的质量。实验表明，该风格迁 移模型生成的风格化图像不仅具有更好的视觉效果，且保留了内容图像的语义信息。特别是对于显著区域突出 的内容图像，保证显著区域不被扭曲是生成视觉友好图像的重要前提。

关键词: 风格迁移, 图像变换, 显著区域保留, 卷积神经网络, 显著性检测

Abstract: Style transfer based on neural networks has become one of the hot research issues in academia and industry in recent years. Existing methods can apply different styles to a given content image to generate a stylized image and greatly enhance visual effects and conversion efficiency. However, these methods focus on learning the underlying features of the image, easily leading to the loss of content image semantic information of stylized images. Based on this, an improved scheme was proposed to match the salient area of the stylized image with that of the content image. By adding a saliency detection network to generate a saliency map of the composite image and the content image, the loss of the saliency map was calculated during the training process, so that the composite image could almost maintain a saliency area consistent with that of the content image, which is conducive to improving the stylized image. The experiment shows that the stylized image generated by the style transfer model can not only produce better visual effects, but also retains the semantic information of the content image. Ensuring the undistorted ness of salient areas is a significant prerequisite for generating visually friendly images, especially for the content image with prominent salient areas. 

Key words: style transfer, image transformation, salient region preservation, convolutional neural network, saliency detection

中图分类号: