Multi-scale convolutional neural network incorporating attention mechanism for intestinal polyp segmentation

doi:10.11996/JG.j.2095-302X.2023010050

Abstract

Abstract:

Intestinal polyp segmentation provides the location and morphology of polyps in colon, allowing doctors to infer the possibility of canceration according to the degree of structural deformation, which facilitates the early diagnosis and treatment of colon cancer. In view of the limited multi-scale features extracted by many existing convolutional neural networks (CNN), and the frequently caused redundant and interfering features, it is difficult to extract the complex and variable targets. To address this challenge, a multi-scale convolutional neural network incorporating attention mechanism was proposed for intestinal polyp segmentation. Specifically, the pyramid strategy based on different scales of pooling was designed to capture the rich multi-scale context information. Then a channel attention mechanism was incorporated into the network so that the model could adaptively select appropriate local and global contextual information for feature integration based on the region of interest. Following that, by combining the pyramid pooling strategy and the channel attention mechanism, a multi-scale effective semantic fusion decoder network was constructed to improve the model robustness for segmentation of intestinal polyps with complex and variable shapes and sizes. The experimental results show that the Dice coefficient, IoU, and sensitivity produced by the proposed model reach 90.6%, 84.4%, and 91.1% on the CVC-ClinicDB dataset, and 80.6%, 72.6%, and 79.0% on the ETIS-Larib dataset, indicating that the proposed model could accurately and effectively segments polyps in colonoscopy images.

Key words: polyp segmentation, colonoscopy images, convolutional neural network, multiscale semantic information

CLC Number:

TP391

SHAN Fang-mei, WANG Meng-wen, LI Min. Multi-scale convolutional neural network incorporating attention mechanism for intestinal polyp segmentation[J]. Journal of Graphics, 2023, 44(1): 50-58.

Figures/Tables 9

References 14

[1]	MAHMUD T, PAUL B, FATTAH S A. PolypSegNet: a modified encoder-decoder architecture for automated polyp segmentation from colonoscopy images[J]. Computers in Biology and Medicine, 2021, 128: 104119. DOI URL
[2]	JHA D, SMEDSRUD P H, JOHANSEN D, et al. A comprehensive study on colorectal polyp segmentation with ResUNet++, conditional random field and test-time augmentation[J]. IEEE Journal of Biomedical and Health Informatics, 2021, 25(6): 2029-2040. DOI PMID
[3]	LONG J, SHELHAMER E, DARRELL T. Fully convolutional networks for semantic segmentation[C]//2015 IEEE Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2015: 3431-3440.
[4]	RONNEBERGER O, FISCHER P, BROX T. U-net: convolutional networks for biomedical image segmentation[M]// Lecture Notes in Computer Science. Cham: Springer International Publishing, 2015: 234-241.
[5]	IBTEHAZ N, RAHMAN M S. MultiResUNet: rethinking the U-Net architecture for multimodal biomedical image segmentation[J]. Neural Networks, 2020, 121: 74-87. DOI URL
[6]	GUO Y B, BERNAL J, MATUSZEWSKI B J. Polyp segmentation with fully convolutional deep neural networks-extended evaluation study[J]. Journal of Imaging, 2020, 6(7): 69. DOI URL
[7]	XIANG T G, ZHANG C Y, LIU D N, et al. BiO-net: learning recurrent Bi-directional connections for encoder-decoder architecture[M]//Medical Image Computing and Computer Assisted Intervention - MICCAI 2020. Cham: Springer International Publishing, 2020: 74-84.
[8]	SCHLEMPER J, OKTAY O, SCHAAP M, et al. Attention gated networks: learning to leverage salient regions in medical images[J]. Medical Image Analysis, 2019, 53: 197-207. DOI PMID
[9]	HU J, SHEN L, SUN G. Squeeze-and-excitation networks[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2018: 7132-7141.
[10]	FAN D P, JI G P, ZHOU T, et al. PraNet: parallel reverse attention network for polyp segmentation[M]//Medical Image Computing and Computer Assisted Intervention - MICCAI 2020. Cham: Springer International Publishing, 2020: 263-273.
[11]	BERNAL J, SÁNCHEZ F J, FERNÁNDEZ-ESPARRACH G, et al. WM-DOVA maps for accurate polyp highlighting in colonoscopy: validation vs. saliency maps from physicians[J]. Computerized Medical Imaging and Graphics, 2015, 43: 99-111. DOI URL
[12]	SILVA J, HISTACE A, ROMAIN O, et al. Toward embedded detection of polyps in WCE images for early diagnosis of colorectal cancer[J]. International Journal of Computer Assisted Radiology and Surgery, 2014, 9(2): 283-293. DOI PMID
[13]	ZHOU Z W, SIDDIQUEE M M R, TAJBAKHSH N, et al. UNet++: redesigning skip connections to exploit multiscale features in image segmentation[J]. IEEE Transactions on Medical Imaging, 2020, 39(6): 1856-1867. DOI PMID
[14]	JHA D, RIEGLER M A, JOHANSEN D, et al. DoubleU-net: a deep convolutional neural network for medical image segmentation[C]// 2020 IEEE 33rd International Symposium on Computer-Based Medical Systems. New York: IEEE Press, 2020: 558-564.

方法	Dice	IoU	Sensitivity	Precision
FCN	80.9	75.3	79.4	81.4
BiONet	84.8	77.8	85.0	89.5
Attention UNet	87.2	80.2	86.9	91.5
UNet++	87.4	77.3	82.2	92.1
UNet	88.1	83.5	89.3	95.2
MultiResUNet	88.5	82.7	85.1	96.3
Ours	90.6	84.4	91.1	92.1

方法	Dice	IoU	Sensitivity	Precision
FCN	80.9	75.3	79.4	81.4
BiONet	84.8	77.8	85.0	89.5
Attention UNet	87.2	80.2	86.9	91.5
UNet++	87.4	77.3	82.2	92.1
UNet	88.1	83.5	89.3	95.2
MultiResUNet	88.5	82.7	85.1	96.3
Ours	90.6	84.4	91.1	92.1

方法	Dice	IoU	Sensitivity	Precision
FCN	59.7	53.9	61.3	68.2
UNet++	65.3	54.8	56.6	79.6
Attention UNet	69.4	61.6	66.9	88.2
UNet	73.3	66.9	71.2	82.1
Dil. ResFCN	75.5	71.1	71.9	87.7
Double UNet	76.2	72.1	73.3	83.9
Ours	80.6	72.6	79.0	88.0

方法	Dice	IoU	Sensitivity	Precision
FCN	59.7	53.9	61.3	68.2
UNet++	65.3	54.8	56.6	79.6
Attention UNet	69.4	61.6	66.9	88.2
UNet	73.3	66.9	71.2	82.1
Dil. ResFCN	75.5	71.1	71.9	87.7
Double UNet	76.2	72.1	73.3	83.9
Ours	80.6	72.6	79.0	88.0

方法	Dice (%)	IoU (%)	Sensitivity (%)	Precision (%)	FLOPs (M)	Params (M)
基准网络	88.1	83.5	89.3	95.2	57.9	28.9
基准网络+金字塔池化	89.3	82.6	90.7	90.8	60.6	30.3
基准网络+多尺度有效语义融合	90.6	84.4	91.1	92.1	64.1	32.1