Diversified generation of theatrical masks based on SASGAN

doi:10.11996/JG.j.2095-302X.2024010102

Abstract

Abstract:

To address the problem of low resolution and lack of realism in existing automatically generated theatrical masks, a stylized generative adversarial network (SASGAN) based on a self-attentive mechanism was proposed. Firstly, SASGAN introduced the self-attentive mechanism and vector quantization method based on StyleGAN, thereby enhancing the extraction of geometric structure features of mask patterns. Subsequently, the diversified differentiation generation (DDG) method was supplemented with a mask hue-assisted algorithm by expanding the data with DDG to build a theatrical mask dataset containing 12,599 images. The final training was performed on this dataset to generate mask images with both diversity and realism. The experimental results demonstrated significant improvement in data augmentation for theatrical masks using the DDG method compared to the traditional methods, while SASGAN enhanced the resolution and realism of theatrical masks, achieving the desired effect in subjective visualization.

Key words: theatrical masks, generation adversarial network, image generation, attention mechanism, vector quantization

CLC Number:

TP391

GU Tianjun, XIONG Suya, LIN Xiao. Diversified generation of theatrical masks based on SASGAN[J]. Journal of Graphics, 2024, 45(1): 102-111.

Figures/Tables 19

References 28

[1]	周大雯. 中国戏剧脸谱漫谈[J]. 戏剧之家, 2018(12): 35.
	ZHOU D W. Random talk on China’s traditional opera facial makeup[J]. Home Drama, 2018(12): 35 (in Chinese).
[2]	何婧慈. 脸谱的装饰构图特色及其艺术价值[J]. 艺术品鉴, 2017(2): 198-199.
	HE J C. Characteristics of decorative composition of Facebook and its artistic value[J]. Appreciation, 2017(2): 198-199 (in Chinese).
[3]	ZHAO W C, ZHANG Q, LI H H, et al. Low-altitude UAV detection method based on one-staged detection framework[C]// The 2nd International Conference on Advances in Computer Technology, Information Science and Communications. New York: IEEE Press, 2020: 112-117.
[4]	PUTTARUKSA C, TAEPRASARTSIT P. Color data augmentation through learning color-mapping parameters between cameras[C]// The 15th International Joint Conference on Computer Science and Software Engineering (JCSSE). New York: IEEE Press, 2018: 1-5.
[5]	LI B Y, CUI Y, LIN T-Y, et al. Single image texture translation for data augmentation[EB/OL]. [2023-05-17]. https://arxiv.org/abs/2106.13804v1.
[6]	ANTONIOU A, STORKEY A, EDWARDS H. Data augmentation generative adversarial networks[EB/OL]. [2023-05-17]. https://arxiv.org/abs/1711.04340v2.
[7]	CHEN M H, ZHAO S, LIU H F, et al. Adversarial-learned loss for domain adaptation[EB/OL]. [2023-05-21]. http://arxiv.org/abs/2001.01046.
[8]	JIANG X, LAO Q C, MATWIN S, et al. Implicit class- conditioned domain alignment for unsupervised domain adaptation[C]// The 37th International Conference on Machine Learning. New York: ACM, 2020: 4816-4827.
[9]	ZHANG X L, SONG H H, ZHANG K H, et al. Single image super-resolution with enhanced Laplacian pyramid network via conditional generative adversarial learning[J]. Neurocomputing, 2020, 398: 531-538. DOI URL
[10]	KIM H J, LEE D. Image denoising with conditional generative adversarial networks (CGAN) in low dose chest images[J]. Nuclear Instruments and Methods in Physics Research Section A: Accelerators, Spectrometers, Detectors and Associated Equipment, 2020, 954: 161914. DOI URL
[11]	JOSE A, FRANCIS A. Reversible colour density compression of images using cGANs[EB/OL]. [2023-04-21]. https://arxiv.org/abs/2106.10542.pdf.
[12]	黄凯奇, 赵鑫, 李乔哲, 等. 视觉图灵: 从人机对抗看计算机视觉下一步发展[J]. 图学学报, 2021, 42(3): 339-348.
	HUANG K Q, ZHAO X, LI Q Z, et al. Visual Turing: the next development of computer vision in the view of human- computer gaming[J]. Journal of Graphics, 2021, 42(3): 339-348 (in Chinese).
[13]	林晓, 屈时操, 黄伟, 等. 显著区域保留的图像风格迁移算法[J]. 图学学报, 2021, 42(2): 190-197.
	LIN X, QU S C, HUANG W, et al. Style transfer algorithm for salient region preservation[J]. Journal of Graphics, 2021, 42(2): 190-197 (in Chinese).
[14]	任好盼, 王文明, 危德健, 等. 基于高分辨率网络的人体姿态估计方法[J]. 图学学报, 2021, 42(3): 432-438.
	REN H P, WANG W M, WEI D J, et al. Human pose estimation based on high-resolution net[J]. Journal of Graphics, 2021, 42(3): 432-438 (in Chinese). DOI
[15]	李彬, 王平, 赵思逸. 基于双重注意力机制的图像超分辨重建算法[J]. 图学学报, 2021, 42(2): 206-215.
	LI B, WANG P, ZHAO S Y. Image super-resolution reconstruction based on dual attention mechanism[J]. Journal of Graphics, 2021, 42(2): 206-215 (in Chinese). DOI
[16]	GOODFELLOW I, POUGET-ABADIE J, MIRZA M, et al. Generative adversarial networks[J]. Communications of the ACM, 2020, 63(11): 139-144. DOI URL
[17]	KANG M G, PARK J. ContraGAN: contrastive learning for conditional image generation[EB/OL]. [2023-04-21]. https://arxiv.org/abs/2006.12681v3.
[18]	ARJOVSKY M, CHINTALA S, BOTTOU L. Wasserstein generative adversarial networks[C]// The 34th International Conference on Machine Learning - Volume 70. New York:ACM, 2017: 214-223.
[19]	KARRAS T, LAINE S, AILA T M. A style-based generator architecture for generative adversarial networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021, 43(12): 4217-4228. DOI URL
[20]	ZHANG H, GOODFELLOW I, METAXAS D, et al. Self-attention generative adversarial networks[EB/OL]. [2023-04-21]. https://arxiv.org/abs/1805.08318.pdf.
[21]	KARRAS T, LAINE S, AITTALA M, et al. Analyzing and improving the image quality of StyleGAN[C]// 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2020: 8107-8116.
[22]	KARRAS T, AILA T, LAINE S, et al. [EB/OL]. [2023-04-21]. https://arxiv.org/abs/1710.10196v2.
[23]	曹锦纲, 杨国田, 杨锡运. 基于注意力机制的深度学习路面裂缝检测[J]. 计算机辅助设计与图形学学报, 2020, 32(8): 1324-1333.
	CAO J G, YANG G T, YANG X Y. Pavement crack detection with deep learning based on attention mechanism[J]. Journal of Computer-Aided Design & Computer Graphics, 2020, 32(8): 1324-1333 (in Chinese).
[24]	王一鸣, 杜慧敏, 张霞, 等. 视觉注意力网络在工件表面缺陷检测中的应用[J]. 计算机辅助设计与图形学学报, 2019, 31(9): 1528-1534.
	WANG Y M, DU H M, ZHANG X, et al. Application of visual attention network in workpiece surface defect detection[J]. Journal of Computer-Aided Design & Computer Graphics, 2019, 31(9): 1528-1534 (in Chinese).
[25]	刘德志, 梁正友, 孙宇. 结合空间注意力机制与光流特征的微表情识别方法[J]. 计算机辅助设计与图形学学报, 2021, 33(10): 1541-1552.
	LIU D Z, LIANG Z Y, SUN Y. Micro-expression recognition method based on spatial attention mechanism and optical flow features[J]. Journal of Computer-Aided Design & Computer Graphics, 2021, 33(10): 1541-1552 (in Chinese).
[26]	GRAY R. Vector quantization[J]. IEEE ASSP Magazine, 1984, 1(2): 4-29.
[27]	GERSHO A, GRAY R. Vector quantization and signal compression[EB/OL]. [2023-04-21]. https://www.researchgate.net/publication/279368025_Vector_Quantization_and_Signal_Compression.
[28]	LINDE Y, BUZO A, GRAY R. An algorithm for vector quantizer design[J]. IEEE Transactions on Communications, 1980, 28(1): 84-95. DOI URL

网络	SSIM	NIMA
SASGAN无矢量量化	0.813 6	5.762±1.637
SASGAN+矢量量化	0.820 7	5.796±1.622

网络	SSIM	NIMA
SASGAN无矢量量化	0.813 6	5.762±1.637
SASGAN+矢量量化	0.820 7	5.796±1.622

网络	SSIM	NIMA
SASGAN	0.820 7	5.796±1.622
StyleGAN	0.796 8	5.568±1.714
GAN	0.619 2	4.996±1.983

网络	SSIM	NIMA
SASGAN	0.820 7	5.796±1.622
StyleGAN	0.796 8	5.568±1.714
GAN	0.619 2	4.996±1.983

数据增广方法	SSIM	NIMA
DDG	0.820 7	5.796±1.622
Traditional	0.476 7	3.362±1.994