Dynamic balanced multi-scale feature fusion for colorectal polyp segmentation

doi:10.11996/JG.j.2095-302X.2023020225

Abstract

Abstract:

Colorectal cancer is one of the most prevalent diseases, and accurate colorectal polyp segmentation can aid physicians in early prevention. However, in the process of segmentation, colorectal polyp images present several challenges, such as low contrast, varied shapes of lesions, and randomized location. Moreover, with large parameters, the Unet network does not yield high segmentation accuracy. Therefore, an improved Unet algorithm based on dynamic balanced multiscale feature fusion was proposed. This algorithm took Unet as the main body and combined the atrous spatial pyramid pooling module (ASPP). A channel shuffle inception (CSI) module and a group inception (GI) module were also put forth to improve the convolution block of the codec, reduce the amount of network parameters, and improve the model′s characterization ability. Additionally, a residual pyramid split attention module (RPSA) was presented for the skip connection of the codec, balancing the channel information in the skip connection, and improving the overall network split performance. Experimental results showed that this method could not only outperform other methods in terms of segmentation effect, but also significantly reduce the number of parameters, thereby proving its effectiveness.

Key words: dynamic balance, multi-scale feature fusion, Unet, colorectal polyps, segmentation algorithm

CLC Number:

TP391

LU Qiu, SHAO Hua-ze, ZHANG Yun-lei. Dynamic balanced multi-scale feature fusion for colorectal polyp segmentation[J]. Journal of Graphics, 2023, 44(2): 225-232.

Figures/Tables 9

Fig. 1 Total network architecture

Fig. 2 ASPP module

Fig. 3 Two multiscale feature fusion modules ((a) CSI; (b) GI)

Fig. 4 Residual pyramid split attention module structure ((a) RPSA; (b) Split+Concat; (c) SE weight)

Fig. 5 Data enhancement

Table 1 Experimental comparison of each model

模型	DSC (%)	IOU (%)	R (%)	P (%)	参数量 (M)
FCN	82.42	70.46	77.66	91.64	5.00
Unet++	87.07	77.37	85.52	93.39	36.15
AUnet	92.99	86.98	88.16	96.55	8.91
Unet	91.81	85.13	87.34	95.75	31.03
RU++	92.60	86.52	89.09	95.07	4.07
DU	93.62	88.13	89.40	96.58	29.29
本文	93.91	88.64	90.34	95.29	13.39

Fig. 6 Variation of loss by model

Fig. 7 Visualization of the segmentation effect of each model ((a) The original drawing; (b) True segmentation; (c) FCN; (d) Unet++; (e) AUnet; (f) Unet; (g) RU++; (h) DU; (i) Ours)

Table 2 Experimental comparison of each model

基准模型	模型配置	DSC (%)	IOU (%)	R (%)	P (%)	参数量(M)
M1	Unet	91.81	85.13	87.34	95.75	31.03
M2	M1+ASPP	92.36	86.21	86.63	96.66	18.48
M3	M2+(CSI+GI)	93.06	87.12	88.23	96.41	11.06
M4	M3+RPSA	93.91	88.64	90.34	95.29	13.39

References 25

[1]	赵龙, 杨长江, 林易霖, 等. 人工智能在结直肠癌病理研究中的应用进展[J]. 中华实验外科杂志, 2022, 39(3): 597-601.
	ZHAO L, YANG C J, LIN Y L, et al. Application and development of artificial intelligence in pathological study of colorectal cancer[J]. Chinese Journal of Experimental Surgery, 2022, 39(3): 597-601. (in Chinese)
[2]	王嫣然, 陈清亮, 吴俊君. 面向复杂环境的图像语义分割方法综述[J]. 计算机科学, 2019, 46(9): 36-46.
	WANG Y R, CHEN Q L, WU J J. Research on image semantic segmentation for complex environments[J]. Computer Science, 2019, 46(9): 36-46. (in Chinese) DOI
[3]	陈超, 齐峰. 卷积神经网络的发展及其在计算机视觉领域中的应用综述[J]. 计算机科学, 2019, 46(3): 63-73. DOI
	CHEN C, QI F. Review on development of convolutional neural network and its application in computer vision[J]. Computer Science, 2019, 46(3): 63-73. (in Chinese) DOI
[4]	LONG J, SHELHAMER E, DARRELL T. Fully convolutional networks for semantic segmentation[C]// 2015 IEEE Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2015: 3431-3440.
[5]	RONNEBERGER O, FISCHER P, BROX T. U-net: convolutional networks for biomedical image segmentation[M]// Lecture Notes in Computer Science. Cham: Springer International Publishing, 2015: 234-241.
[6]	李翠云, 白静, 郑凉. 融合边缘增强注意力机制和U-Net网络的医学图像分割[J]. 图学学报, 2022, 43(2): 273-278.
	LI C Y, BAI J, ZHENG L. A U-Net based contour enhanced attention for medical image segmentation[J]. Journal of Graphics, 2022, 43(2): 273-278. (in Chinese)
[7]	陈铭, 梅雪, 朱文俊, 等. 一种新型Mobile Unet网络的肺结节图像分割方法[J]. 南京工业大学学报: 自然科学版, 2022(1): 76-81, 91.
	CHEN M, MEI X, ZHU W J, et al. A novel pulmonarynodule segmentation method using Mobile Unet network[J]. Journal of Nanjing Tech University: Natural Science Edition, 2022(1): 76-81, 91. (in Chinese)
[8]	HOWARD A, SANDLER M, CHEN B, et al. Searching for MobileNetV3[C]// 2019 IEEE/CVF International Conference on Computer Vision. New York: IEEE Press, 2020: 1314-1324.
[9]	ZHOU Z W, SIDDIQUEE M M R, TAJBAKHSH N, et al. UNet: redesigning skip connections to exploit multiscale features in image segmentation[J]. IEEE Transactions on Medical Imaging, 2020, 39(6): 1856-1867. DOI URL
[10]	FAN D P, JI G P, ZHOU T, et al. PraNet: parallel reverse attention network for polyp segmentation[M]// Medical Image Computing and Computer Assisted Intervention. Cham: Springer International Publishing, 2020: 263-273.
[11]	JHA D, RIEGLER M A, JOHANSEN D, et al. DoubleU-net: a deep convolutional neural network for medical image segmentation[C]// 2020 IEEE 33rd International Symposium on Computer-Based Medical Systems. New York: IEEE Press, 2020: 558-564.
[12]	CHEN L C, ZHU Y K, PAPANDREOU G, et al. Encoder-decoder with atrous separable convolution for semantic image segmentation[M]//Computer Vision - ECCV 2018. Cham: Springer International Publishing, 2018: 833-851.
[13]	XU Q, MA Z C, HE N, et al. DCSAU-net: a deeper and more compact split-attention U-net for medical image segmentation[EB/OL]. [2022-09-24]. https://arxiv.org/abs/2202.00972.
[14]	JHA D, SMEDSRUD P H, RIEGLER M A, et al. ResUNet++: an advanced architecture for medical image segmentation[C]// 2019 IEEE International Symposium on Multimedia. New York: IEEE Press, 2020: 225-2255.
[15]	ZHANG Z X, LIU Q J, WANG Y H. Road extraction by deep residual U-net[J]. IEEE Geoscience and Remote Sensing Letters, 2018, 15(5): 749-753. DOI URL
[16]	HU J, SHEN L, SUN G. Squeeze-and-excitation networks[C]// 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2018: 7132-7141.
[17]	YU F, KOLTUN V. Multi-scale context aggregation by dilated convolutions[EB/OL]. [2022-04-18]. https://arxiv.org/abs/1511.07122.
[18]	曲长波, 姜思瑶, 吴德阳. 空洞卷积的多尺度语义分割网络[J]. 计算机工程与应用, 2019, 55(24): 91-95. DOI
	QU C B, JIANG S Y, WU D Y. Multiscale semantic segmentation network based on cavity convolution[J]. Computer Engineering and Applications, 2019, 55(24): 91-95. (in Chinese) DOI
[19]	王军, 冯孙铖, 程勇. 深度学习的轻量化神经网络结构研究综述[J]. 计算机工程, 2021, 47(8): 1-13. DOI
	WANG J, FENG S C, CHENG Y. Survey of research on lightweight neural network structures for deep learning[J]. Computer Engineering, 2021, 47(8): 1-13. (in Chinese) DOI
[20]	MA N N, ZHANG X Y, ZHENG H T, et al. ShuffleNet V2: practical guidelines for efficient CNN architecture design[M]// Computer Vision - ECCV 2018. Cham: Springer International Publishing, 2018: 122-138.
[21]	冯兴杰, 张天泽. 基于分组卷积进行特征融合的全景分割算法[J]. 计算机应用, 2021, 41(7): 2054-2061. DOI
	FENG X J, ZHANG T Z. Panoptic segmentation algorithm based on grouped convolution for feature fusion[J]. Journal of Computer Applications, 2021, 41(7): 2054-2061. (in Chinese) DOI
[22]	IBTEHAZ N, RAHMAN M S. MultiResUNet: Rethinking the U-Net architecture for multimodal biomedical image segmentation[J]. Neural Networks, 2020, 121: 74-87. DOI URL
[23]	ZHANG H, ZU K K, LU J, et al. EPSANet: an efficient pyramid split attention block on convolutional neural network[EB/OL]. [2022-07-22]. https://arxiv.org/abs/2105.14447.
[24]	BERNAL J, TAJKBAKSH N, SANCHEZ F J, et al. Comparative validation of polyp detection methods in video colonoscopy: results from the MICCAI 2015 endoscopic vision challenge[J]. IEEE Transactions on Medical Imaging, 2017, 36(6): 1231-1249. DOI PMID
[25]	OKTAY O, SCHLEMPER J, FOLGOC L L, et al. Attention U-net: learning where to look for the pancreas[EB/OL]. [2022-05-20]. https://arxiv.org/abs/1804.03999.