基于注意力机制的东巴画情感分类

doi:10.11996/JG.j.2095-302X.2023010059

图学学报 ›› 2023, Vol. 44 ›› Issue (1): 59-66.DOI: 10.11996/JG.j.2095-302X.2023010059

• 图像处理与计算机视觉 • 上一篇下一篇

基于注意力机制的东巴画情感分类

潘森垒¹(), 钱文华¹(), 曹进德², 徐丹¹

1.云南大学信息学院计算机科学与工程系，云南昆明 650504
2.东南大学数学学院系统科学系，江苏南京 211189

收稿日期:2022-06-19 修回日期:2022-07-11 出版日期:2023-10-31 发布日期:2023-02-16
通讯作者: 钱文华
作者简介:潘森垒(1996-)，男，硕士研究生。主要研究方向为数字图像处理与计算机视觉。E-mail：313000234@qq.com
基金资助:
国家自然科学基金项目(62162065);云南省科技厅应用基础研究计划重点项目(2019FA044);云南省中青年学术技术带头人后备人才项目(2019HB121);云南大学研究生科研创新项目(2021Z078)

Learning attention for Dongba paintings emotion classification

PAN Sen-lei¹(), QIAN Wen-hua¹(), CAO Jin-de², XU Dan¹

1. Department of Computer Science and Engineering, School of Information Science and Engineering, Yunnan University, Kunming Yunan 650504, China
2. Department of System Science, School of Mathematics, Southeast University, Nanjing Jiangsu 211189, China

Received:2022-06-19 Revised:2022-07-11 Online:2023-10-31 Published:2023-02-16
Contact: QIAN Wen-hua
About author:PAN Sen-lei (1996-), master student. His main research interests cover digital image processing and computer vision. E-mail：313000234@qq.com
Supported by:
National Natural Science Foundation of China(62162065);Key Project of Applied Basic Research Plan of Yunnan Provincial Department of Science and Technology(2019FA044);Yunnan Young and Middle-Aged Academic and Technical Leaders Reserve Talents Project(2019HB121);Postgraduate Research and Innovation Foundation of Yunnan University(2021Z078)

摘要/Abstract

摘要：

东巴画情感多元、样本较少，基于注意力机制的分类算法能有效辅助东巴画情感分类，解决东巴画样本较少的问题。首先，按照绘画主题，将东巴画分为人物、鬼怪、动物、植物4类题材，按照绘画情感分为勤劳朴实、曼妙美好等12种情感；其次，采用编码器-解码器架构实现东巴画的情感特征提取，引入预训练模型提升分类模型的泛化性能，加速小样本东巴画情感分类收敛；最后，在解码器中设置空白注意力并融合编码器的输出序列，经由解码器的解码学习机制，学习东巴画图像语义以指导模型更准确、合理地提升对东巴画情感的学习能力。实验结果表明，基于注意力机制的分类算法相较现有方法分类准确率更高，获得了80.7%的分类准确率，解决了东巴画情感丰富、难以区分的问题。

关键词: 东巴画, 注意力机制, 情感分类, 编码器

Abstract:

Rich in emotions and limited in samples constitute the artistic characteristic of Dongba paintings. The classification algorithm of learning attention could effectively assist the emotional classification of Dongba paintings, solving the problem of limited samples of Dongba paintings. Firstly, Dongba paintings were divided into 4 themes such as figures, ghosts, animals, and plants. According to the painting emotions, Dongba paintings were divided into 12 kinds of emotions, such as industriousness, simplicity, grace, and beauty. Secondly, the architecture of encoder and decoder was employed to extract their emotional features, while the pre-training model was used to improve the generalization performance of the classification model and accelerate the convergence of emotion classification for small samples of Dongba paintings. Finally, the blank attention was set in the decoder and the output sequence of the encoder was fused. Through the decoder, the semantics of Dongba paintings were learned, guiding the model to more accurately and reasonably improve the classification ability. Experiments show that the classification algorithm of learning attention could attain a classification accuracy of 80.7% higher than the existing methods, solving the problem of rich and difficult-to-distinguish emotions in Dongba paintings.

Key words: Dongba paintings, attention, emotion classification, encoder

中图分类号:

TP391

潘森垒, 钱文华, 曹进德, 徐丹. 基于注意力机制的东巴画情感分类[J]. 图学学报, 2023, 44(1): 59-66.

PAN Sen-lei, QIAN Wen-hua, CAO Jin-de, XU Dan. Learning attention for Dongba paintings emotion classification[J]. Journal of Graphics, 2023, 44(1): 59-66.

图/表 9

图1 本文模型

Fig. 1 Proposed model

图2 情感分类

Fig. 2 Emotion classification

表1 东巴画数据集部分示例

Table 1 Sample of of Dongba paintings datasets

情感	画作题材
情感	人物	鬼怪	动物	植物
消极
消极	怀有戒心	反感抵触	焦虑紧张	萎靡发蔫
中立
中立	和睦友善	中立素净	应而务实	不偏不倚
积极
积极	勤劳朴实	狂热积极	文雅温和	曼妙美好

表2 东巴画数据集扩充示例

Table 2 Sample of of Dongba paintings datasets expansion

扩充方式	原始样本	扩充样本
图像旋转
随机噪点
制造破损

表3 东巴画情感分类实验

Table 3 Emotion classification experiment of Dongba paintings

模型	分类准确率(%)
ResNet-18	64.5
ResNet-34	68.3
ResNet-50^[14]	70.2
ResNet-101	71.3
Swin-T^[15]	69.9
Swin-S^[15]	72.6
Swin-B^[15]	73.1
Vit-T^[17]	72.6
Vit-B^[17]	76.7
本文方法-Tiny	74.2
本文方法-Base	80.7

表4 主干网络实验

Table 4 Experiment of bckbone

方法	主干网络	参数量(M)	准确率(%)
1	ResNet-18	28	72.4
2	ResNet-34	38	76.6
3	ResNet-50	41	80.7
4	ResNet-101	60	81.8

图3 编码层-解码层数量分析

Fig. 3 Analysis of encoder-decoder layers number

表5 消融实验

Table 5 Ablation experiment

方法	解码器输出	编码器输出	准确率(%)
1	×	√	74.2
2	√	×	77.8
3	√	√	80.7

图4 可视化结果展示((a)初始图像；(b)消除解码器；(c)消除编码器；(d)本文模型)

Fig. 4 Visual presentation (a) Origin; (b) No-decoder; (c) No-encoder; (d) Ours)

参考文献 17

[1]	吴树生. 纳西族东巴文化艺术结晶: 东巴画[J]. 地方文化研究, 2019(6): 2.
	WU S S. The crystallization of Naxi Dongba culture and art: Dongba painting[J]. Local Culture Research, 2019(6): 2 (in Chinese).
[2]	黎克, 钱文华, 王成学, 等. 基于图神经网络的东巴画小样本分类[J]. 计算机辅助设计与图形学学报, 2021, 33(7): 1073-1083.
	LI K, QIAN W H, WANG C X, et al. Dongba painting few-shot classification based on graph neural network[J]. Journal of Computer-Aided Design & Computer Graphics, 2021, 33(7): 1073-1083 (in Chinese).
[3]	CARION N, MASSA F, SYNNAEVE G, et al. End-to-end object detection with transformers[C]//European Conference on Computer Vision. Cham: Springer International Publishin, 2020: 213-229.
[4]	MACHAJDIK J, HANBURY A. Affective image classification using features inspired by psychology and art theory[C]// MM’10 - Proceedings of the ACM Multimedia 2010 International Conference. New York: ACM, 2010: 83-92.
[5]	BORTH D, JI R R, CHEN T, et al. Large-scale visual sentiment ontology and detectors using adjective noun pairs[C]// MM 2013 - Proceedings of the 2013 ACM Multimedia Conference. New York: ACM, 2013: 223-232.
[6]	SIERSDORFER S, MINACK E, DENG F, et al. Analyzing and predicting sentiment of images on the social web[C]//MM'10 - Proceedings of the ACM Multimedia 2010 International Conference. New York: ACM, 2010: 715-718.
[7]	KRIZHEVSKY A, SUTSKEVER I. Imagenet classification with deep convolutional neural networks[C]//Advances in Neural Information Processing Systems. Lake Tahoe: NIPS, 2012: 25-26.
[8]	ZHAO S, YAO H, YANG Y, et al. Affective image retrieval via multi-graph learning[C]// The 22nd ACM International Conference on Multimedia. New York: ACM, 2014: 1025-1028.
[9]	盛家川, 陈雅琦, 韩亚洪. 深层网络特征聚合重标定的中国画情感分类算法[J]. 计算机辅助设计与图形学学报, 2020, 32(9): 1420-1429.
	SHENG J C, CHEN Y Q, HAN Y H. Sentiment classification of Chinese paintings via feature recalibration of deep network aggregation[J]. Journal of Computer-Aided Design ＆ Computer Graphics, 2020, 32(9): 1420-1429 (in Chinese).
[10]	VASWANI A, SHAZEER N, PARMAR N. Attention is all You need[C]// Advances in Neural Information Processing Systems. Lone Beach: NIPS, 2017: 5998-6008.
[11]	ALMOWALLAD A, SANCHEZ V. Human emotion distribution learning from face images using cnn and lbc features[C]//2020 8th International Workshop on Biometrics and Forensics. New York, IEEE Press, 2020: 1-6.
[12]	YANG J, SHE D, LAI Y, et al. Weakly supervised coupled networks for visual sentiment analysis[C]// The IEEE Conference on Computer Vision and Pattern Recognition. New York, IEEE Press, 2018: 7584-7592.
[13]	钱文华, 徐丹, 徐瑾, 等. 东巴画艺术风格绘制[J]. 系统仿真学报, 2020, 32(7): 1349-1359. DOI
	QIAN W H, XU D, XU J, et al. Simulation of dongba art style painting[J]. Journal of System Simulation, 2020, 32(7): 1349-1359 (in Chinese). DOI
[14]	HE K M, ZHANG X Y, REN S Q, et al. Deep residual learning for image recognition[C]//The IEEE Conference on Computer Vision and Pattern Recognition. Los Alamitos: IEEE Computer Society Press, 2016: 770-778.
[15]	LIU Z, LIN Y, CAO Y, et al. Swin transformer: hierarchical vision transformer using shifted windows[C]//2021 IEEE/CVF International Conference on Computer Vision. New York, IEEE Press, 2021: 10012-10022.
[16]	LOSHCHILOV I, HUTTER F. Decoupled weight decay regularization[EB/OL]. (2017-11-05) [2021-08-13].https://arxiv.org/pdf/1711.05101.
[17]	DOSOVITSKIY A, BEYER L, KOLESNIKV A, et al. An image is worth 16x16 words: transformers for image recognition at scale[EB/OL]. (2020-10-11) [2021-11-20]. https://arxiv.org/pdf/2010.11929.

基于注意力机制的东巴画情感分类

Learning attention for Dongba paintings emotion classification

RichHTML

PDF (PC)

可视化

摘要/Abstract

引用本文

使用本文

图/表 9

参考文献 17

相关文章 15

编辑推荐

Metrics

本文评价

[1]	杨陈成, 董秀成, 侯兵, 张党成, 向贤明, 冯琪茗. 基于参考的Transformer纹理迁移深度图像超分辨率重建[J]. 图学学报, 2023, 44(5): 861-867.
[2]	宋焕生, 文雅, 孙士杰, 宋翔宇, 张朝阳, 李旭. 基于改进教师学生网络的隧道火灾检测[J]. 图学学报, 2023, 44(5): 978-987.
[3]	李利霞, 王鑫, 王军, 张又元. 基于特征融合与注意力机制的无人机图像小目标检测算法[J]. 图学学报, 2023, 44(4): 658-666.
[4]	李鑫, 普园媛, 赵征鹏, 徐丹, 钱文华. 内容语义和风格特征匹配一致的艺术风格迁移[J]. 图学学报, 2023, 44(4): 699-709.
[5]	余伟群, 刘佳涛, 张亚萍. 融合注意力的拉普拉斯金字塔单目深度估计[J]. 图学学报, 2023, 44(4): 728-738.
[6]	胡欣, 周运强, 肖剑, 杨杰. 基于改进YOLOv5的螺纹钢表面缺陷检测[J]. 图学学报, 2023, 44(3): 427-437.
[7]	郝鹏飞, 刘立群, 顾任远. YOLO-RD-Apple果园异源图像遮挡果实检测模型[J]. 图学学报, 2023, 44(3): 456-464.
[8]	罗文宇, 傅明月. 基于YoloX-ECA模型的非法野泳野钓现场监测技术[J]. 图学学报, 2023, 44(3): 465-472.
[9]	李雨, 闫甜甜, 周东生, 魏小鹏. 基于注意力机制与深度多尺度特征融合的自然场景文本检测[J]. 图学学报, 2023, 44(3): 473-481.
[10]	史彩娟, 石泽, 闫巾玮, 毕阳阳. 基于双语义双向对齐VAE的广义零样本学习[J]. 图学学报, 2023, 44(3): 521-530.
[11]	吴文欢, 张淏坤. 融合空间十字注意力与通道注意力的语义分割网络[J]. 图学学报, 2023, 44(3): 531-539.
[12]	谢国波, 贺笛轩, 何宇钦, 林志毅. 基于P-CenterNet的光学遥感图像烟囱检测[J]. 图学学报, 2023, 44(2): 233-240.
[13]	熊举举, 徐杨, 范润泽, 孙少聪. 基于轻量化视觉Transformer的花卉识别[J]. 图学学报, 2023, 44(2): 271-279.
[14]	陈静, 杨学志, 陈鲸, 刘雪南. 采用自注意力抗干扰网络的视频房颤检测[J]. 图学学报, 2023, 44(2): 313-323.
[15]	成浪, 敬超. 基于改进YOLOv7的X线图像旋转目标检测[J]. 图学学报, 2023, 44(2): 324-334.