面向不平衡数据的肺部疾病细粒度分类模型

doi:10.11996/JG.j.2095-302X.2023030513

图学学报 ›› 2023, Vol. 44 ›› Issue (3): 513-520.DOI: 10.11996/JG.j.2095-302X.2023030513

• 图像处理与计算机视觉 • 上一篇下一篇

面向不平衡数据的肺部疾病细粒度分类模型

刘冰¹^,²^,³(), 叶成绪¹^,²^,³()

1.青海师范大学计算机学院，青海西宁 810000
2.青海师范大学青海省物联网重点实验室，青海西宁 810000
3.藏语智能信息处理及应用国家重点实验室，青海西宁 810000

收稿日期:2022-09-26 接受日期:2022-11-24 出版日期:2023-06-30 发布日期:2023-06-30
通讯作者: 叶成绪(1970-)，男，教授，博士。主要研究方向为机器学习与应用及信息安全等。E-mail：149926237@qq.com
作者简介:
刘冰(1998-)，男，硕士研究生。主要研究方向为机器学习与医学图像处理。E-mail：252859670@qq.com
基金资助:
青海省物联网重点实验室项目(2022-ZJ-Y21)

Fine-grained classification model of lung disease for imbalanced data

LIU Bing¹^,²^,³(), YE Cheng-xu¹^,²^,³()

1. School of Computing, Qinghai Normal University, Xining Qinghai 810000, China
2. Qinghai Provincial Key Laboratory of IoT, Qinghai Normal University, Xining Qinghai 810000, China
3. The State Key Laboratory of Tibetan Intelligent Information Processing and Application, Xining Qinghai 810000, China

Received:2022-09-26 Accepted:2022-11-24 Online:2023-06-30 Published:2023-06-30
Contact: YE Cheng-xu (1970-), professor, Ph.D. His main research interests cover machine learning and applications, information security, etc. E-mail：149926237@qq.com
About author:
LIU Bing (1998-), master student. His main research interests cover machine learning and medical image processing. E-mail：252859670@qq.com
Supported by:
The Key Laboratory of IoT of Qinghai(2022-ZJ-Y21)

摘要/Abstract

摘要：

肺部疾病种类繁多，不同病症的影像学表现存在细微差别，且相关医学影像数据普遍存在类别不平衡的现象，使用一般的深度学习模型对其进行区分存在困难。针对上述问题，提出一种面向不平衡数据的肺部疾病细粒度分类模型，其具有双分支的特征提取结构，分别是EfficientNetB0和添加卷积块注意力模块(CBAM)的MobileNetV2，通过注意力机制来增强图像中重要特征的权重。在特征提取后基于多模双线性池化对特征进行融合，并使用Focal Loss损失函数来改善不平衡数据的分类效果，通过超参数自适应调整的策略进行模型训练，最终完成分类。使用Grad-CAM对模型的关注点可视化，以解决分类的可解释性问题。实验结果表明，该模型的分类准确率为0.985，Kappa系数为0.973，F1值为0.981，各评价指标均有显著提升，具有较好的分类性能，有助于肺部疾病的辅助诊断。

关键词: 细粒度分类, 不平衡数据, 特征提取, 特征融合, 肺部X-ray图像

Abstract:

There are various types of lung diseases, each having distinct imaging manifestations. However, medical image data related to these diseases often suffer from category imbalance, making it difficult to distinguish them using general deep learning models. To tackle these problems, a fine-grained classification model for lung diseases based on imbalanced data was proposed. The model had a two-branch feature extraction structure, namely EfficientNetB0 and MobileNetV2 with a convolutional block attention module (CBAM). The attention mechanism was utilized to enhance the weight of important features in the images. After feature extraction, the features were fused based on multi-mode bilinear pooling, and the Focal Loss function was used to improve the classification effect of imbalanced data. The model was trained using the strategy of hyperparameter adaptive adjustment, and finally the classification was completed. Grad-CAM was also employed to visualize the concerns of the model to address the interpretability of the classification. The experimental results demonstrated that the proposed model achieved a classification accuracy of 0.985, a Kappa coefficient of 0.973, and an F1 value of 0.981. All the evaluation indexes have been significantly improved, which has exhibited excellent classification performance and can act as a helpful tool for the auxiliary diagnosis of lung diseases.

Key words: fine grained classification, imbalanced data, feature extraction, feature fusion, X-ray images of the lungs

中图分类号:

TP391

刘冰, 叶成绪. 面向不平衡数据的肺部疾病细粒度分类模型[J]. 图学学报, 2023, 44(3): 513-520.

LIU Bing, YE Cheng-xu. Fine-grained classification model of lung disease for imbalanced data[J]. Journal of Graphics, 2023, 44(3): 513-520.

图/表 12

图1 肺部疾病细粒度分类模型

Fig. 1 Fine-grained classification model of lung diseases

图2 添加CBAM的Bottleneck residual block

Fig. 2 Bottleneck residual block with CBAM

图3 数据样本展示((a)正常；(b)普通肺炎；(c)新冠肺炎；(d)肺结核)

Fig. 3 Data sample display ((a) Normal; (b) Pneumonia; (c) COVID-19; (d) Tuberculosis)

图4 各类别数量与占总体比例

Fig. 4 The number and proportion of each category

图5 迭代次数与准确率的对应关系

Fig. 5 The corresponding relationship between iteration times and accuracy

图6 学习率的变化趋势

Fig. 6 Trends in learning rates

图7 超参数对比实验((a)批处理大小；(b)随即丢弃比率)

Fig. 7 Hyperparameter comparison experiment ((a) Batch size; (b) Random dropout ratio)

图8 测试产生的混淆矩阵

Fig. 8 The confusion matrix generated by the test

表1 各类别对应的评价指标

Table 1 Evaluation index corresponding to each category

类别	Precision	Recall	F1
NORMAL	0.981	0.978	0.979
PNEUMONIA	0.987	0.989	0.988
COVID-19	0.985	0.985	0.985
TUBERCULOSIS	0.977	0.970	0.973

表2 消融实验对比结果

Table 2 Comparative results of ablation experiments

模型	ACC	Kappa	Precision	Recall	F1
M1	0.968	0.945	0.952	0.943	0.947
M2	0.932	0.881	0.923	0.857	0.889
M3	0.943	0.900	0.925	0.876	0.900
M4	0.924	0.865	0.925	0.823	0.871
本文	0.985	0.973	0.983	0.981	0.981

表3 不同模型对比实验结果

Table 3 Comparative experimental results of different models

模型	ACC	Kappa	F1	Params(M)
VGG16+ResNet50	0.912	0.846	0.852	164
VGG16+DenseNet121	0.933	0.897	0.877	147
ResNet50+DenseNet121	0.941	0.896	0.901	54
CoviNet^[11]	0.958	0.936	0.937	116
PAM-DenseNet^[22]	0.929	0.886	0.923	24
CoroNet^[23]	0.920	0.893	0.921	33
BSGAN^[24]	0.954	0.929	0.945	48
MHA-CoroCapsule^[25]	0.972	0.957	0.962	20
EnsembleNet^[26]	0.949	0.911	0.957	142
本文	0.985	0.973	0.981	15

图9 Grad-CAM产生的结果展示((a)正常；(b)普通肺炎；(c)新冠肺炎；(d)肺结核)

Fig. 9 Display of results produced by Grad-CAM ((a) Normal; (b) Pneumonia; (c) COVID-19; (d) Tuberculosis)

参考文献 27

[1]	SONI M, GOMATHI S, KUMAR P, et al. Hybridizing convolutional neural network for classification of lung diseases[J]. International Journal of Swarm Intelligence Research, 2022, 13(2): 1-15.
[2]	FRIX A N, COUSIN F, REFAEE T, et al. Radiomics in lung diseases imaging: state-of-the-art for clinicians[J]. Journal of Personalized Medicine, 2021, 11(7): 602. DOI URL
[3]	楚阳, 徐文龙. 基于计算机辅助诊断技术的阿尔兹海默症早期分类研究综述[J]. 计算机工程与科学, 2022, 44(5): 879-893.
	CHU Y, XU W L. Review of early classification of Alzheimer’s disease based on computer-aided diagnosis technology[J]. Computer Engineering & Science, 2022, 44(5): 879-893. (in Chinese)
[4]	成科扬, 王宁, 师文喜, 等. 深度学习可解释性研究进展[J]. 计算机研究与发展, 2020, 57(6): 1208-1217.
	CHENG K Y, WANG N, SHI W X, et al. Research advances in the interpretability of deep learning[J]. Journal of Computer Research and Development, 2020, 57(6): 1208-1217. (in Chinese)
[5]	YU S X, FENG X X, WANG B, et al. Automatic classification of cervical cells using deep learning method[J]. IEEE Access, 2021, 9: 32559-32568. DOI URL
[6]	BHAN A, KAPOOR S, GULATI M. Diagnosing Parkinson's disease in early stages using image enhancement, ROI extraction and deep learning algorithms[C]// The 2nd International Conference on Intelligent Engineering and Management. New York: IEEE Press, 2021: 521-525.
[7]	SHI F, CHEN B J, CAO Q Q, et al. Semi-supervised deep transfer learning for benign-malignant diagnosis of pulmonary nodules in chest CT images[J]. IEEE Transactions on Medical Imaging, 2022, 41(4): 771-781. DOI URL
[8]	MIYOSHI H, SATO K, KABEYA Y, et al. Deep learning shows the capability of high-level computer-aided diagnosis in malignant lymphoma[J]. Laboratory Investigation, 2020, 100(10): 1300-1310. DOI PMID
[9]	SHI W Q, TONG L, ZHU Y D, et al. COVID-19 automatic diagnosis with radiographic imaging: explainable attention transfer deep neural networks[J]. IEEE Journal of Biomedical and Health Informatics, 2021, 25(7): 2376-2387. DOI URL
[10]	李朝林, 张荣芬, 刘宇红. 融入多尺度双线性注意力的轻量化眼底疾病多分类网络[J]. 计算机应用研究, 2022, 39(7): 2183-2189, 2195.
	LI C L, ZHANG R F, LIU Y H. Lightweight fundus disease multi-classification network with multi-scale bilinear attention[J]. Application Research of Computers, 2022, 39(7): 2183-2189, 2195. (in Chinese)
[11]	LAFRAXO S, EL ANSARI M. CoviNet: automated COVID-19 detection from X-rays using deep learning techniques[C]// The 6th IEEE Congress on Information Science and Technology. New York: IEEE Press, 2021: 489-494.
[12]	DING Y F, MA Z Y, WEN S G, et al. AP-CNN: weakly supervised attention pyramid convolutional neural network for fine-grained visual classification[J]. IEEE Transactions on Image Processing, 2021, 30: 2826-2836. DOI PMID
[13]	YANG L F, LI X, SONG R J, et al. Dynamic MLP for fine-grained image classification by leveraging geographical and temporal information[C]// 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2022: 10935-10944.
[14]	SANDLER M, HOWARD A, ZHU M L, et al. MobileNetV2: inverted residuals and linear bottlenecks[C]// 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2018: 4510-4520.
[15]	TAN M, LE Q V. EfficientNet: rethinking model scaling for convolutional neural networks[C]// 2019 International Conference on Machine Learning. New York: IEEE Press, 2019: 6105-6114.
[16]	HU J, SHEN L, ALBANIE S, et al. Squeeze-and-excitation networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020, 42(8): 2011-2023. DOI PMID
[17]	GAO Y, BEIJBOM O, ZHANG N, et al. Compact bilinear pooling[C]// 2016 IEEE Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2016: 317-326.
[18]	WOO S, PARK J, LEE J Y, et al. CBAM: convolutional block attention module[C]// 2018 European Conference on Computer Vision. Cham: Springer International Publishing, 2018: 3-19.
[19]	DONG Y N, LIU Q W, DU B, et al. Weighted feature fusion of convolutional neural network and graph attention network for hyperspectral image classification[J]. IEEE Transactions on Image Processing, 2022, 31: 1559-1572. DOI URL
[20]	柴文光, 李嘉怡. 重加权在多类别不平衡医学图像检测中的应用[J]. 计算机工程与应用, 2022, 58(8): 237-242. DOI
	CHAI W G, LI J Y. Application of re-weight method in multiple class-imbalance medical images detection[J]. Computer Engineering and Applications, 2022, 58(8): 237-242. (in Chinese) DOI
[21]	LIN T Y, GOYAL P, GIRSHICK R, et al. Focal loss for dense object detection[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020, 42(2): 318-327. DOI URL
[22]	XIAO B, YANG Z Y, QIU X M, et al. PAM-DenseNet: a deep convolutional neural network for computer-aided COVID-19 diagnosis[J]. IEEE Transactions on Cybernetics, 2022, 52(11): 12163-12174. DOI URL
[23]	KHAN A I, SHAH J L, BHAT M M. CoroNet: a deep neural network for detection and diagnosis of COVID-19 from chest X-ray images[J]. Computer Methods and Programs in Biomedicine, 2020, 196: 105581. DOI URL
[24]	PASTORINO J, BISWAS A K. Data adequacy bias impact in a data-blinded semi-supervised GAN for privacy-aware COVID-19 chest X-ray classification[C]// The 13th ACM International Conference on Bioinformatics, Computational Biology and Health Informatics. New York: ACM, 2022: 1-8.
[25]	LI F D, LU X Y, YUAN J J. MHA-CoroCapsule: multi-head attention routing-based capsule network for COVID-19 chest X-ray image classification[J]. IEEE Transactions on Medical Imaging, 2022, 41(5): 1208-1218. DOI URL
[26]	AL-MONSUR A, KABIR M R, AR-RAFI A M, et al. Covid-EnsembleNet: an ensemble based approach for detecting covid-19 by utilising chest X-ray images[C]// 2022 IEEE World AI IoT Congress. New York: IEEE Press, 2022: 351-356.
[27]	SELVARAJU R R, COGSWELL M, DAS A, et al. Grad-CAM: visual explanations from deep networks via gradient-based localization[J]. International Journal of Computer Vision, 2020, 128(2): 336-359. DOI

面向不平衡数据的肺部疾病细粒度分类模型

Fine-grained classification model of lung disease for imbalanced data

RichHTML

PDF (PC)

可视化

摘要/Abstract

引用本文

使用本文

图/表 12

参考文献 27

相关文章 15

编辑推荐

Metrics

本文评价

[1]	李利霞, 王鑫, 王军, 张又元 . 基于特征融合与注意力机制的无人机图像小目标检测算法 [J]. 图学学报, 2023, 44(4): 658-666.
[2]	李鑫, 普园媛, 赵征鹏, 徐丹, 钱文华 . 内容语义和风格特征匹配一致的艺术风格迁移 [J]. 图学学报, 2023, 44(4): 699-709.
[3]	李雨, 闫甜甜, 周东生, 魏小鹏. 基于注意力机制与深度多尺度特征融合的自然场景文本检测[J]. 图学学报, 2023, 44(3): 473-481.
[4]	史彩娟, 石泽, 闫巾玮, 毕阳阳. 基于双语义双向对齐VAE的广义零样本学习[J]. 图学学报, 2023, 44(3): 521-530.
[5]	陆秋, 邵铧泽, 张云磊 . 动态平衡多尺度特征融合的结直肠息肉分割[J]. 图学学报, 2023, 44(2): 225-232.
[6]	李小波 , 李阳贵 , 郭宁 , 范震 . 融合注意力机制的 YOLOv5 口罩检测算法[J]. 图学学报, 2023, 44(1): 16-25.
[7]	张倩, 王夏黎, 王炜昊, 武历展, 李超. 基于多尺度特征融合的细胞计数方法[J]. 图学学报, 2023, 44(1): 41-49.
[8]	谷雨, 赵军. 列车闸瓦钎及闸瓦故障图像检测算法研究 [J]. 图学学报, 2023, 44(1): 88-94.
[9]	董哲同, 蔺宏伟. 计算机辅助拓扑设计 ——持续同调在几何设计和处理中的应用[J]. 图学学报, 2022, 43(6): 957-966.
[10]	郭文, 李冬, 袁飞 . 多尺度注意力融合和抗噪声的轻量点云人脸识别模型[J]. 图学学报, 2022, 43(6): 1124-1133.
[11]	墨瀚林, 郝优, 郭锐, 郝宏翔, 张贺, 李琪, 李华. 图形图像积分与微分不变量的构造与应用[J]. 图学学报, 2022, 43(6): 1182-1192.
[12]	武历展, 王夏黎, 张倩, 王炜昊, 李超. 基于优化 YOLOv5s 的跌倒人物目标检测方法[J]. 图学学报, 2022, 43(5): 791-802.
[13]	王素琴, 任琪, 石敏, 朱登明. 基于异常检测的产品表面缺陷检测与分割[J]. 图学学报, 2022, 43(3): 377-386.
[14]	曹力, 吴垚, 徐宜科. 基于中轴表达的三维模型轮廓提取方法[J]. 图学学报, 2022, 43(3): 461-468.
[15]	李扬科, 宋全博, 周元峰. 用于手势识别的时空融合网络以及虚拟签名系统[J]. 图学学报, 2022, 43(3): 504-512.