基于多尺度特征融合的细胞计数方法

doi:10.11996/JG.j.2095-302X.2023010041

摘要/Abstract

摘要：

细胞计数一直是医学影像分析中非常重要的一项工作，在生物医学实验和临床医学等领域起着十分关键的作用。针对细胞计数工作中存在的由细胞尺寸变化等因素造成的细胞计数精度低的问题，引入高度拥挤目标识别网络 CSRNet 并加以改进，构建了一种基于多尺度特征融合的细胞计数方法。首先，使用 VGG16 的前 10 层提取细胞特征，避免了由于网络过深造成的小目标信息丢失；其次，引入空间金字塔池化结构提取细胞的多尺度特征并进行特征融合，降低了因细胞形态各异、尺寸不一和细胞遮挡等问题带来的计数误差；然后，使用混合空洞卷积对特征图进行解码，得到密度图，解决了 CSRNet 在解码过程中像素遗漏的问题；最后对密度图逐像素进行回归得到细胞总数。另外，在训练过程中引入了一种新的组合损失函数以代替欧几里得损失函数，不仅考虑了 ground truth 密度图与预测密度图单个像素点之间的关系，还考虑了其全局和局部的密度水平。实验证明，优化后的 CSRNet 在 VGG cells 和 MBM cells 数据集上取得了较好的结果，有效改善了由细胞尺寸变化等因素造成的细胞计数精度低的问题。

关键词:

"> 细胞计数, 多尺度特征融合, 密度估计, 空间金字塔池化, 混合空洞卷积

Abstract:

To address the problem of low cell counting accuracy caused by factors such as cell size variation in cell counting work, the highly crowded target recognition network CSRNet was introduced and improved, and a cell counting method based on multi-scale feature fusion was constructed. First, the first 10 layers of VGG16 were employed to extract cell features, avoiding the loss of small target information due to the deep network. Then, the spatial pyramid pooling structure was introduced to extract the multi-scale features of cells and perform feature fusion, reducing the counting errors caused by different cell shapes, sizes, and cell occlusion. Then the feature map was decoded using the hybrid dilated convolution to obtain the density map, solving the problem of missing pixels in the decoding process of CSRNet. Finally, the density map was regressed pixel by pixel to obtain the total number of cells. In addition, a new combined loss function was introduced in the training process to replace the Euclidean loss function, which not only considered the relationship between the ground truth density map and the single pixel point of the predicted density map, but also considered the global and local density levels. Experiments show that the optimized CSRNet could yield better results on VGG cells and MBM cells datasets, effectively improving the low cell counting accuracy caused by factors such as cell size variation.

Key words:

"> cell counting, multi-scale feature fusion, density estimation, spatial pyramid pooling, hybrid dilated convolution

中图分类号:

TP 391

张倩, 王夏黎, 王炜昊, 武历展, 李超. 基于多尺度特征融合的细胞计数方法[J]. 图学学报, 2023, 44(1): 41-49.

ZHANG Qian, WANG Xia-li, WANG Wei-hao, WU Li-zhan, LI Chao. Cell counting method based on multi-scale feature fusion[J]. Journal of Graphics, 2023, 44(1): 41-49.

参考文献

[1] MAUSUMI M, RAHUL K G, MANALI M. Detection and counting of red blood cells in blood cell images using Hough transform[J]. International Journal of Computer Applications, 2012, 53(16): 13-17.

[2] FAUSTINO G M, GATTASS M, REHEN S, et al. Automatic embryonic stem cells detection and counting method in fluorescence microscopy images[C]//2009 IEEE International Symposium on Biomedical Imaging: from Nano to Macro. New York: IEEE Press, 2009: 799-802.

[3] SHARIF J M, MISWAN M F, NGADI M A, et al. Red blood cell segmentation using masking and watershed algorithm: a preliminary study[J]. 2012 International Conference on Biomedical Engineering, 2012, 2012: 258-262.

[4] XUE Y, RAY N, HUGH J, et al. Cell counting by regression using convolutional neural network[C]//European Conference on Computer Vision. Chma: Springer International Publishing, 2016: 274-290.

[5] COHEN J P, BOUCHER G, GLASTONBURY C A, et al. Count-ception: counting by fully convolutional redundant counting[C]//2017 IEEE International Conference on Computer Vision Workshops. New York: IEEE Press, 2018: 18-26.

[6] JIANG N, YU F H. A multi-column network for cell counting[J]. OSA Continuum, 2020, 3(7): 1834-1846.

[7] LI Y, ZHANG X, CHEN D. CSRNet: dilated convolutional neural networks for understanding the highly congested scenes[EB/OL]. (2018-04-11) [2021-12-08]. https://arxiv.org/ abs/1802.10062.

[8] ZHANG C, SUN C, SU R, et al. Segmentation of clustered nuclei based on curvature weighting[C]//The 27th Conference on Image Vision Computing. New York: ACM, 2012: 49.

[9] 刘树杰. 基于卷积神经网络的红细胞检测和计数方法[D]. 广州: 华南理工大学, 2017. LIU S J. Red blood cell detection and counting based on convolutional neural network[D]. Guangzhou: South China University of Technology, 2017 (in Chinese).

[10] 徐晓涛. 基于深度目标识别的细胞计数研究[D]. 合肥: 安徽大学, 2020. XU X T. Research on cell counting based on depth target recognition[D]. Hefei: Anhui University, 2020 (in Chinese).

[11] 洪羽萌. 外周血细胞分类与计数在深度学习中的应用[D]. 南京: 南京大学, 2019. HONG Y M. Application of peripheral blood cell classification and counting in deep learning[D]. Nanjing: Nanjing University, 2019 (in Chinese).

[12] AICH S, STAVNESS I. Improving object counting with heatmap regulation[EB/OL]. (2018-03-14) [2021-12-08]. https://www.semanticscholar.org/paper/Improving-Object-Cou nting-with-Heatmap-Regulation-Aich-Stavness/aabbc2880cdc 9af83322e2bac8e2f88d1105f601.

[13] MARSDEN M, MCGUINNESS K, LITTLE S, et al. People, penguins and petri dishes: adapting object counting models to new visual domains and object types without forgetting[EB/OL]. (2017-11-15) [2022-01-03]. https:// ieeexplore.ieee.org/document/8578940.

[14] LEMPITSKY V, ZISSERMAN A. Learning to count objects in images[EB/OL]. (2010-12-06) [2021-12-18]. https://dl.acm. org/doi/10.5555/2997189.2997337.

[15] FIASCHI L, KOETHE U, NAIR R, et al. Learning to count with regression forest and structured labels [EB/OL]. (2012-11-01) [2021-12-22]. https://ieeexplore.ieee.org/document/ 6460719.

[16] JIANG N, YU F. A cell counting framework based on random forest and density map[J]. Applied Sciences, 2020, 10(23): 8346.

[17] GUO Y, STEIN J, WU G, et al. SAU-Net: a universal deep network for cell counting[C]//The 10th ACM International Conference on Bioinformatics, Computational Biology and Health Informatics. New York: ACM, 2019: 299-306.

[18] XIE W, NOBLE J A, ZISSERMAN A. Microscopy cell counting and detection with fully, convolutional regression networks[J]. Computer Methods in Biomechanics and Bio, 2018, 6(3-4): 283-292.

[19] HE K, ZHANG X, REN S, et al. Spatial pyramid pooling in deep convolutional networks for visual recognition[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 37(9): 1904-1916.

[20] SIMONYAN K, ZISSERMAN A. Very deep convolutional networks for large-scale image recognition[EB/OL]. (2014-09-04) [2022-01-10]. https://arxiv.org/abs/1409.1556v6.

[21] LIU W, SALZMANN M, FUA P. Context-aware crowd counting[EB/OL]. (2019-04-15) [2021-12-13]. https://infoscience. epfl.ch/record/265758.

[22] WANG P, CHEN P, YUAN Y, et al. Understanding convolution for semantic segmentation[EB/OL]. (2018-06-01) [2021-12-28]. https://arxiv.org/abs/1702.08502v3.

[23] KAINZ P, URSCHLER M, SCHULTER S, et al. You should use regression to detect cells[C]//International Conference on Medical Image Computing and Computer Assisted Intervention-MICCAI. Heidelberg：Springer, 2015: 276-283.

[24] ZHANG Y, ZHOU D, CHEN S, et al. Single-image crowd counting via multi-column convolutional neural network[C]// 2016 IEEE Conf- erence on Computer Vision and Pattern Recognition. New York：IEEE Press, 2016: 589-597

[25] JIANG X, XIAO Z, ZHANG B, et al. Crowd counting and density estimation by trellis encoder-decoder networks[EB/OL]. [2021-12-28]. https://xueshu.baidu.com/ usercenter/paper/show?paperid=147606t0n15c0860jb0b0010hr 729346.

[26] DAI F, LIU H, MA Y, et al. Dense scale network for crowd counting[EB/OL]. (2021-09-01) [2022-12-28]. https://dl.acm. org/doi/10.1145/3460426.3463628.

[27] 蒋妮. 基于密度估计的细胞计数方法及其在显微图像中的应用研究[D]. 杭州: 浙江大学, 2021. JIANG N. Research on cell counting based on density estimation and its applications in microscopic image[D]. Hangzhou: Zhejiang University, 2021 (in Chinese).