Learning attention for Dongba paintings emotion classification

doi:10.11996/JG.j.2095-302X.2023010059

Abstract

Abstract:

Rich in emotions and limited in samples constitute the artistic characteristic of Dongba paintings. The classification algorithm of learning attention could effectively assist the emotional classification of Dongba paintings, solving the problem of limited samples of Dongba paintings. Firstly, Dongba paintings were divided into 4 themes such as figures, ghosts, animals, and plants. According to the painting emotions, Dongba paintings were divided into 12 kinds of emotions, such as industriousness, simplicity, grace, and beauty. Secondly, the architecture of encoder and decoder was employed to extract their emotional features, while the pre-training model was used to improve the generalization performance of the classification model and accelerate the convergence of emotion classification for small samples of Dongba paintings. Finally, the blank attention was set in the decoder and the output sequence of the encoder was fused. Through the decoder, the semantics of Dongba paintings were learned, guiding the model to more accurately and reasonably improve the classification ability. Experiments show that the classification algorithm of learning attention could attain a classification accuracy of 80.7% higher than the existing methods, solving the problem of rich and difficult-to-distinguish emotions in Dongba paintings.

Key words: Dongba paintings, attention, emotion classification, encoder

CLC Number:

TP391

PAN Sen-lei, QIAN Wen-hua, CAO Jin-de, XU Dan. Learning attention for Dongba paintings emotion classification[J]. Journal of Graphics, 2023, 44(1): 59-66.

Figures/Tables 9

References 17

[1]	吴树生. 纳西族东巴文化艺术结晶: 东巴画[J]. 地方文化研究, 2019(6): 2.
	WU S S. The crystallization of Naxi Dongba culture and art: Dongba painting[J]. Local Culture Research, 2019(6): 2 (in Chinese).
[2]	黎克, 钱文华, 王成学, 等. 基于图神经网络的东巴画小样本分类[J]. 计算机辅助设计与图形学学报, 2021, 33(7): 1073-1083.
	LI K, QIAN W H, WANG C X, et al. Dongba painting few-shot classification based on graph neural network[J]. Journal of Computer-Aided Design & Computer Graphics, 2021, 33(7): 1073-1083 (in Chinese).
[3]	CARION N, MASSA F, SYNNAEVE G, et al. End-to-end object detection with transformers[C]//European Conference on Computer Vision. Cham: Springer International Publishin, 2020: 213-229.
[4]	MACHAJDIK J, HANBURY A. Affective image classification using features inspired by psychology and art theory[C]// MM’10 - Proceedings of the ACM Multimedia 2010 International Conference. New York: ACM, 2010: 83-92.
[5]	BORTH D, JI R R, CHEN T, et al. Large-scale visual sentiment ontology and detectors using adjective noun pairs[C]// MM 2013 - Proceedings of the 2013 ACM Multimedia Conference. New York: ACM, 2013: 223-232.
[6]	SIERSDORFER S, MINACK E, DENG F, et al. Analyzing and predicting sentiment of images on the social web[C]//MM'10 - Proceedings of the ACM Multimedia 2010 International Conference. New York: ACM, 2010: 715-718.
[7]	KRIZHEVSKY A, SUTSKEVER I. Imagenet classification with deep convolutional neural networks[C]//Advances in Neural Information Processing Systems. Lake Tahoe: NIPS, 2012: 25-26.
[8]	ZHAO S, YAO H, YANG Y, et al. Affective image retrieval via multi-graph learning[C]// The 22nd ACM International Conference on Multimedia. New York: ACM, 2014: 1025-1028.
[9]	盛家川, 陈雅琦, 韩亚洪. 深层网络特征聚合重标定的中国画情感分类算法[J]. 计算机辅助设计与图形学学报, 2020, 32(9): 1420-1429.
	SHENG J C, CHEN Y Q, HAN Y H. Sentiment classification of Chinese paintings via feature recalibration of deep network aggregation[J]. Journal of Computer-Aided Design ＆ Computer Graphics, 2020, 32(9): 1420-1429 (in Chinese).
[10]	VASWANI A, SHAZEER N, PARMAR N. Attention is all You need[C]// Advances in Neural Information Processing Systems. Lone Beach: NIPS, 2017: 5998-6008.
[11]	ALMOWALLAD A, SANCHEZ V. Human emotion distribution learning from face images using cnn and lbc features[C]//2020 8th International Workshop on Biometrics and Forensics. New York, IEEE Press, 2020: 1-6.
[12]	YANG J, SHE D, LAI Y, et al. Weakly supervised coupled networks for visual sentiment analysis[C]// The IEEE Conference on Computer Vision and Pattern Recognition. New York, IEEE Press, 2018: 7584-7592.
[13]	钱文华, 徐丹, 徐瑾, 等. 东巴画艺术风格绘制[J]. 系统仿真学报, 2020, 32(7): 1349-1359. DOI
	QIAN W H, XU D, XU J, et al. Simulation of dongba art style painting[J]. Journal of System Simulation, 2020, 32(7): 1349-1359 (in Chinese). DOI
[14]	HE K M, ZHANG X Y, REN S Q, et al. Deep residual learning for image recognition[C]//The IEEE Conference on Computer Vision and Pattern Recognition. Los Alamitos: IEEE Computer Society Press, 2016: 770-778.
[15]	LIU Z, LIN Y, CAO Y, et al. Swin transformer: hierarchical vision transformer using shifted windows[C]//2021 IEEE/CVF International Conference on Computer Vision. New York, IEEE Press, 2021: 10012-10022.
[16]	LOSHCHILOV I, HUTTER F. Decoupled weight decay regularization[EB/OL]. (2017-11-05) [2021-08-13].https://arxiv.org/pdf/1711.05101.
[17]	DOSOVITSKIY A, BEYER L, KOLESNIKV A, et al. An image is worth 16x16 words: transformers for image recognition at scale[EB/OL]. (2020-10-11) [2021-11-20]. https://arxiv.org/pdf/2010.11929.

情感	画作题材
情感	人物	鬼怪	动物	植物
消极
消极	怀有戒心	反感抵触	焦虑紧张	萎靡发蔫
中立
中立	和睦友善	中立素净	应而务实	不偏不倚
积极
积极	勤劳朴实	狂热积极	文雅温和	曼妙美好

情感	画作题材
情感	人物	鬼怪	动物	植物
消极
消极	怀有戒心	反感抵触	焦虑紧张	萎靡发蔫
中立
中立	和睦友善	中立素净	应而务实	不偏不倚
积极
积极	勤劳朴实	狂热积极	文雅温和	曼妙美好

扩充方式	原始样本	扩充样本
图像旋转
随机噪点
制造破损

扩充方式	原始样本	扩充样本
图像旋转
随机噪点
制造破损

模型	分类准确率(%)
ResNet-18	64.5
ResNet-34	68.3
ResNet-50^[14]	70.2
ResNet-101	71.3
Swin-T^[15]	69.9
Swin-S^[15]	72.6
Swin-B^[15]	73.1
Vit-T^[17]	72.6
Vit-B^[17]	76.7
本文方法-Tiny	74.2
本文方法-Base	80.7