Research on automatic generation and application of Ruyuan Yao embroidery based on self-attention mechanism

doi:10.11996/JG.j.2095-302X.2024051096

Abstract

Abstract:

To address the limitations of current style migration models in processing Ruyuan Yao embroidery images, especially in effectively handling abstract geometric transformations and the high noise in the generated images, a style migration model for Ruyuan Yao embroidery named SA-CycleGAN was proposed. By incorporating a self-attention mechanism and replacing the objective function for generating the adversarial loss with WGAN, the model significantly enhanced its ability to capture the style features of Ruyuan Yao embroidery, thereby optimizing the quality of style mapping. In terms of application, the proposed SA-CycleGAN model not only provided solid technical support for the automatic generation and online design system of Ruyuan Yao embroidery patterns, but also facilitated the construction of the corresponding database and digital sharing platform. Rigorous comparative experiments demonstrated that the optimized SA-CycleGAN model achieved excellent performance in the evaluation indexes for Ruyuan Yao embroidery pattern factors, its FID value was reduced by 16.1%, and the IS value was relatively improved by 13.2% compared with the original CycleGAN model, resulting in significantly improved image quality that was visually closer to the original Ruyuan Yao embroidery style. The establishment of the pattern design system of Ruyuan Yao embroidery greatly enhanced the design efficiency, injecting new vigor and value into the preservation and innovation of the ethnic group patterns.

Key words: self-attention mechanism, style transfer, Ruyuan Yao embroidery, pattern, generative adversarial network, interaction design

CLC Number:

LIU Zongming, HONG Wei, LONG Rui, ZHU Yue, ZHANG Xiaoyu. Research on automatic generation and application of Ruyuan Yao embroidery based on self-attention mechanism[J]. Journal of Graphics, 2024, 45(5): 1096-1105.

Figures/Tables 16

Table 1 The source of the Ruyuan Yao embroidery pattern

图案类型	图案名称	图案来源
人文类	男人纹	成年男性
人文类	女人纹	成年女性
自然类	石榴纹	石榴果
	鹿纹	鹿
	树形纹	树
几何类	豆腐纹	豆腐
	八角纹1	八角花
	八角纹2	八角花
象征类	盘王印章形纹	盘瓠传说
象征类	龙眼纹	龙眼睛

Table 2 Layout of Ruyuan Yao embroidery patterns

二方连续	四方连续	单独纹样	边缘纹样

Table 3 Layout of Ruyuan Yao embroidery color extraction

RGB值
R:209	G:39	B:46
R:232	G:126	B:42
R:248	G:216	B:91
R:75	G:123	B:70
R:255	G:255	B:255
R:13	G:13	B:13

Fig. 1 CycleGAN network structure

Fig. 2 Basic structure of the self-attention mechanism

Fig. 3 SA-Resnet_block residual block structure

Fig. 4 Improved generative network architecture for CycleGAN

Fig. 5 New and improved modules

Fig. 6 Partial datasets

Fig. 7 Total_loss changes

Fig. 8 Comparison of image generation results of SA-CycleGAN with different networks ((a) Input symbol; (b) CycleGAN; (c) DualGAN; (d) DiscoGAN; (e) SA-CycleGAN)

Fig. 9 Comparison of image results generated from ablation experiments ((a) Input symbol; (b) CycleGAN; (c) CycleGAN+U-Net; (d) CycleGAN+SA_Resnet_block; (e) CycleGAN+SA_Resnet_block+WGAN)

Table 4 Comparison results of SA-CycleGAN with different networks

图像生成方法	IS值	FID值
CycleGAN	2.456	180.381
CycleGAN+U-Net	2.621	181.701
CycleGAN+SA_Resent	2.558	179.389
SA-CycleGAN	2.779	151.408

Fig. 10 System operation process

Fig. 11 System interface display diagram

Fig. 12 System design patterns

References 16

[1]	李筱文. 五彩斑斓: 广东瑶绣[M]. 广州: 广东教育出版社, 2012: 34-162.
	LI X W. Colorful: Guangdong Yao embroidery[M]. Guangzhou: Guangdong Education Press, 2012: 34-162 (in Chinese).
[2]	黎洁仪. 细语梧桐: 释读瑶绣梧桐花形纹的装饰运用与文化蕴意[J]. 装饰, 2019(5): 108-111.
	LI J Y. Whisper to Wutong flowers: decorative application and cultural implication of the pattern “Wutong Flower” in Yao embroidery[J]. Art & Design, 2019(5): 108-111 (in Chinese).
[3]	黄伟. 基于形状文法的乳源瑶绣纹样创新设计与应用[J]. 包装工程, 2023, 44(14): 270-276.
	HUANG W. Innovative design and application of Yao embroidery patterns in Ruyuan based on shape grammar[J]. Packaging Engineering, 2023, 44(14): 270-276 (in Chinese).
[4]	胡小平, 王明仪, 周晓晗, 等. 图像学在粤北瑶族服饰纹样分析中的运用[J]. 丝绸, 2021, 58(1): 123-129.
	HU X P, WANG M Y, ZHOU X H, et al. The application of imageology in the analysis of Yao nationality’s costume patterns in north Guangdong[J]. Journal of Silk, 2021, 58(1): 123-129 (in Chinese).
[5]	刘宗明, 王亚雨. 基于Citespace的中国瑶族刺绣知识图谱可视化分析[J]. 家具与室内装饰, 2023, 30(3): 100-105.
	LIU Z M, WANG Y Y. Visual analysis of the Chinese Yao embroidery knowledge graph based on citespace[J]. Furniture & Interior Design, 2023, 30(3): 100-105 (in Chinese).
[6]	GOODFELLOW I, POUGET-ABADIE J, MIRZA M, et al. Generative adversarial nets[EB/OL]. [2023-11-12]. https://arxiv.org/pdf/1406.2661.
[7]	MIRZA M, OSINDERO S. Conditional generative adversarial nets[EB/OL]. [2023-11-12]. http://arxiv.org/pdf/1411.1784.
[8]	郑锐, 钱文华, 徐丹, 等. 基于卷积神经网络的刺绣风格数字合成[J]. 浙江大学学报: 理学版, 2019, 46(3): 270-278.
	ZHENG R, QIAN W H, XU D, et al. Synthesis of embroidery based on convolutional neural network[J]. Journal of Zhejiang University: Science Edition, 2019, 46(3): 270-278 (in Chinese).
[9]	吴正存, 吴通, 敦晓荣. 基于可拓语义的澄城刺绣风格迁移设计方法研究[J]. 图学学报, 2023, 44(5): 1041-1049. DOI
	WU Z C, WU T, DUN X R. Research on design method of Chengcheng embroidery style migration based on extension semantics[J]. Journal of Graphics, 2023, 44(5): 1041-1049 (in Chinese).
[10]	ZHU J Y, PARK T, ISOLA P, et al. Unpaired image-to-image translation using cycle-consistent adversarial networks[C]// 2017 IEEE International Conference on Computer Vision. New York: IEEE Press, 2017: 2242-2251.
[11]	古天骏, 熊苏雅, 林晓. 基于SASGAN的戏剧脸谱多样化生成[J]. 图学学报, 2024, 45(1): 102-111. DOI
	GU T J, XIONG S Y, LIN X. Diversified generation of theatrical masks based on SASGAN[J]. Journal of Graphics, 2024, 45(1): 102-111 (in Chinese). DOI
[12]	FANG Y K, DENG W H, DU J P, et al. Identity-aware CycleGAN for face photo-sketch synthesis and recognition[J]. Pattern Recognition, 2020, 102: 107249.
[13]	GULRAJANI I, AHMED F, ARJOVSKY , et al. Improved training of wasserstein GANs[C]// The 31st International Conference on Neural Information Processing Systems. Red Hook: Curran Associates Inc., 2017: 5769-5779.
[14]	XU Z G, ZHANG C M, WU Y P. Digital inpainting of mural images based on DC-CycleGAN[J]. Heritage Science, 2023, 11(1): 169.
[15]	周伟, 赵海英. 传统民族服饰数字化采集元数据构建[J]. 图学学报, 2018, 39(6): 1183-1191. DOI
	ZHOU W, ZHAO H Y. Metadata construction of digital collection of traditional costumes[J]. Journal of Graphics, 2018, 39(6): 1183-1191 (in Chinese). DOI
[16]	侯宇康, 吕健, 刘翔, 等. 基于神经风格迁移网络的民族图案创新方法[J]. 图学学报, 2020, 41(4): 606-613. DOI
	HOU Y K, LV J, LIU X, et al. Innovative method of ethnic pattern based on neural style transfer network[J]. Journal of Graphics, 2020, 41(4): 606-613 (in Chinese).