Fine-grained classification model of lung disease for imbalanced data

doi:10.11996/JG.j.2095-302X.2023030513

Abstract

Abstract:

There are various types of lung diseases, each having distinct imaging manifestations. However, medical image data related to these diseases often suffer from category imbalance, making it difficult to distinguish them using general deep learning models. To tackle these problems, a fine-grained classification model for lung diseases based on imbalanced data was proposed. The model had a two-branch feature extraction structure, namely EfficientNetB0 and MobileNetV2 with a convolutional block attention module (CBAM). The attention mechanism was utilized to enhance the weight of important features in the images. After feature extraction, the features were fused based on multi-mode bilinear pooling, and the Focal Loss function was used to improve the classification effect of imbalanced data. The model was trained using the strategy of hyperparameter adaptive adjustment, and finally the classification was completed. Grad-CAM was also employed to visualize the concerns of the model to address the interpretability of the classification. The experimental results demonstrated that the proposed model achieved a classification accuracy of 0.985, a Kappa coefficient of 0.973, and an F1 value of 0.981. All the evaluation indexes have been significantly improved, which has exhibited excellent classification performance and can act as a helpful tool for the auxiliary diagnosis of lung diseases.

Key words: fine grained classification, imbalanced data, feature extraction, feature fusion, X-ray images of the lungs

CLC Number:

TP391

LIU Bing, YE Cheng-xu. Fine-grained classification model of lung disease for imbalanced data[J]. Journal of Graphics, 2023, 44(3): 513-520.

Figures/Tables 12

Fig. 1 Fine-grained classification model of lung diseases

Fig. 2 Bottleneck residual block with CBAM

Fig. 3 Data sample display ((a) Normal; (b) Pneumonia; (c) COVID-19; (d) Tuberculosis)

Fig. 4 The number and proportion of each category

Fig. 5 The corresponding relationship between iteration times and accuracy

Fig. 6 Trends in learning rates

Fig. 7 Hyperparameter comparison experiment ((a) Batch size; (b) Random dropout ratio)

Fig. 8 The confusion matrix generated by the test

Table 1 Evaluation index corresponding to each category

类别	Precision	Recall	F1
NORMAL	0.981	0.978	0.979
PNEUMONIA	0.987	0.989	0.988
COVID-19	0.985	0.985	0.985
TUBERCULOSIS	0.977	0.970	0.973

Table 2 Comparative results of ablation experiments

模型	ACC	Kappa	Precision	Recall	F1
M1	0.968	0.945	0.952	0.943	0.947
M2	0.932	0.881	0.923	0.857	0.889
M3	0.943	0.900	0.925	0.876	0.900
M4	0.924	0.865	0.925	0.823	0.871
本文	0.985	0.973	0.983	0.981	0.981

Table 3 Comparative experimental results of different models

模型	ACC	Kappa	F1	Params(M)
VGG16+ResNet50	0.912	0.846	0.852	164
VGG16+DenseNet121	0.933	0.897	0.877	147
ResNet50+DenseNet121	0.941	0.896	0.901	54
CoviNet^[11]	0.958	0.936	0.937	116
PAM-DenseNet^[22]	0.929	0.886	0.923	24
CoroNet^[23]	0.920	0.893	0.921	33
BSGAN^[24]	0.954	0.929	0.945	48
MHA-CoroCapsule^[25]	0.972	0.957	0.962	20
EnsembleNet^[26]	0.949	0.911	0.957	142
本文	0.985	0.973	0.981	15

Fig. 9 Display of results produced by Grad-CAM ((a) Normal; (b) Pneumonia; (c) COVID-19; (d) Tuberculosis)

References 27

[1]	SONI M, GOMATHI S, KUMAR P, et al. Hybridizing convolutional neural network for classification of lung diseases[J]. International Journal of Swarm Intelligence Research, 2022, 13(2): 1-15.
[2]	FRIX A N, COUSIN F, REFAEE T, et al. Radiomics in lung diseases imaging: state-of-the-art for clinicians[J]. Journal of Personalized Medicine, 2021, 11(7): 602. DOI URL
[3]	楚阳, 徐文龙. 基于计算机辅助诊断技术的阿尔兹海默症早期分类研究综述[J]. 计算机工程与科学, 2022, 44(5): 879-893.
	CHU Y, XU W L. Review of early classification of Alzheimer’s disease based on computer-aided diagnosis technology[J]. Computer Engineering & Science, 2022, 44(5): 879-893. (in Chinese)
[4]	成科扬, 王宁, 师文喜, 等. 深度学习可解释性研究进展[J]. 计算机研究与发展, 2020, 57(6): 1208-1217.
	CHENG K Y, WANG N, SHI W X, et al. Research advances in the interpretability of deep learning[J]. Journal of Computer Research and Development, 2020, 57(6): 1208-1217. (in Chinese)
[5]	YU S X, FENG X X, WANG B, et al. Automatic classification of cervical cells using deep learning method[J]. IEEE Access, 2021, 9: 32559-32568. DOI URL
[6]	BHAN A, KAPOOR S, GULATI M. Diagnosing Parkinson's disease in early stages using image enhancement, ROI extraction and deep learning algorithms[C]// The 2nd International Conference on Intelligent Engineering and Management. New York: IEEE Press, 2021: 521-525.
[7]	SHI F, CHEN B J, CAO Q Q, et al. Semi-supervised deep transfer learning for benign-malignant diagnosis of pulmonary nodules in chest CT images[J]. IEEE Transactions on Medical Imaging, 2022, 41(4): 771-781. DOI URL
[8]	MIYOSHI H, SATO K, KABEYA Y, et al. Deep learning shows the capability of high-level computer-aided diagnosis in malignant lymphoma[J]. Laboratory Investigation, 2020, 100(10): 1300-1310. DOI PMID
[9]	SHI W Q, TONG L, ZHU Y D, et al. COVID-19 automatic diagnosis with radiographic imaging: explainable attention transfer deep neural networks[J]. IEEE Journal of Biomedical and Health Informatics, 2021, 25(7): 2376-2387. DOI URL
[10]	李朝林, 张荣芬, 刘宇红. 融入多尺度双线性注意力的轻量化眼底疾病多分类网络[J]. 计算机应用研究, 2022, 39(7): 2183-2189, 2195.
	LI C L, ZHANG R F, LIU Y H. Lightweight fundus disease multi-classification network with multi-scale bilinear attention[J]. Application Research of Computers, 2022, 39(7): 2183-2189, 2195. (in Chinese)
[11]	LAFRAXO S, EL ANSARI M. CoviNet: automated COVID-19 detection from X-rays using deep learning techniques[C]// The 6th IEEE Congress on Information Science and Technology. New York: IEEE Press, 2021: 489-494.
[12]	DING Y F, MA Z Y, WEN S G, et al. AP-CNN: weakly supervised attention pyramid convolutional neural network for fine-grained visual classification[J]. IEEE Transactions on Image Processing, 2021, 30: 2826-2836. DOI PMID
[13]	YANG L F, LI X, SONG R J, et al. Dynamic MLP for fine-grained image classification by leveraging geographical and temporal information[C]// 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2022: 10935-10944.
[14]	SANDLER M, HOWARD A, ZHU M L, et al. MobileNetV2: inverted residuals and linear bottlenecks[C]// 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2018: 4510-4520.
[15]	TAN M, LE Q V. EfficientNet: rethinking model scaling for convolutional neural networks[C]// 2019 International Conference on Machine Learning. New York: IEEE Press, 2019: 6105-6114.
[16]	HU J, SHEN L, ALBANIE S, et al. Squeeze-and-excitation networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020, 42(8): 2011-2023. DOI PMID
[17]	GAO Y, BEIJBOM O, ZHANG N, et al. Compact bilinear pooling[C]// 2016 IEEE Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2016: 317-326.
[18]	WOO S, PARK J, LEE J Y, et al. CBAM: convolutional block attention module[C]// 2018 European Conference on Computer Vision. Cham: Springer International Publishing, 2018: 3-19.
[19]	DONG Y N, LIU Q W, DU B, et al. Weighted feature fusion of convolutional neural network and graph attention network for hyperspectral image classification[J]. IEEE Transactions on Image Processing, 2022, 31: 1559-1572. DOI URL
[20]	柴文光, 李嘉怡. 重加权在多类别不平衡医学图像检测中的应用[J]. 计算机工程与应用, 2022, 58(8): 237-242. DOI
	CHAI W G, LI J Y. Application of re-weight method in multiple class-imbalance medical images detection[J]. Computer Engineering and Applications, 2022, 58(8): 237-242. (in Chinese) DOI
[21]	LIN T Y, GOYAL P, GIRSHICK R, et al. Focal loss for dense object detection[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020, 42(2): 318-327. DOI URL
[22]	XIAO B, YANG Z Y, QIU X M, et al. PAM-DenseNet: a deep convolutional neural network for computer-aided COVID-19 diagnosis[J]. IEEE Transactions on Cybernetics, 2022, 52(11): 12163-12174. DOI URL
[23]	KHAN A I, SHAH J L, BHAT M M. CoroNet: a deep neural network for detection and diagnosis of COVID-19 from chest X-ray images[J]. Computer Methods and Programs in Biomedicine, 2020, 196: 105581. DOI URL
[24]	PASTORINO J, BISWAS A K. Data adequacy bias impact in a data-blinded semi-supervised GAN for privacy-aware COVID-19 chest X-ray classification[C]// The 13th ACM International Conference on Bioinformatics, Computational Biology and Health Informatics. New York: ACM, 2022: 1-8.
[25]	LI F D, LU X Y, YUAN J J. MHA-CoroCapsule: multi-head attention routing-based capsule network for COVID-19 chest X-ray image classification[J]. IEEE Transactions on Medical Imaging, 2022, 41(5): 1208-1218. DOI URL
[26]	AL-MONSUR A, KABIR M R, AR-RAFI A M, et al. Covid-EnsembleNet: an ensemble based approach for detecting covid-19 by utilising chest X-ray images[C]// 2022 IEEE World AI IoT Congress. New York: IEEE Press, 2022: 351-356.
[27]	SELVARAJU R R, COGSWELL M, DAS A, et al. Grad-CAM: visual explanations from deep networks via gradient-based localization[J]. International Journal of Computer Vision, 2020, 128(2): 336-359. DOI