基于P-CenterNet的光学遥感图像烟囱检测

doi:10.11996/JG.j.2095-302X.2023020233

图学学报 ›› 2023, Vol. 44 ›› Issue (2): 233-240.DOI: 10.11996/JG.j.2095-302X.2023020233

• 图像处理与计算机视觉 • 上一篇下一篇

基于P-CenterNet的光学遥感图像烟囱检测

谢国波(), 贺笛轩, 何宇钦, 林志毅()

广东工业大学计算机学院，广东广州 510006

收稿日期:2022-06-05 接受日期:2022-08-10 出版日期:2023-04-30 发布日期:2023-05-01
通讯作者: 林志毅(1979-)，男，讲师，博士。主要研究方向为人工智能、生物信息学等。E-mail：lzy291@gdut.edu.cn
作者简介:谢国波(1977-)，男，教授，博士。主要研究方向为计算智能及其在遥感影像处理应用、高光谱遥感、复杂疾病模式挖掘等。E-mail：xiegb@gdut.edu.cn
基金资助:
国家自然科学基金项目(61802072)

P-CenterNet for chimney detection in optical remote-sensing images

XIE Guo-bo(), HE Di-xuan, HE Yu-qin, LIN Zhi-yi()

School of Computer Science, Guangdong University of Technology, Guangzhou Guangdong 510006, China

Received:2022-06-05 Accepted:2022-08-10 Online:2023-04-30 Published:2023-05-01
Contact: LIN Zhi-yi (1979-), lecturer, Ph.D. His main research interests cover artificial intelligence, bioinformatics, etc. E-mail：lzy291@gdut.edu.cn
About author:XIE Guo-bo (1977-), professor, Ph.D. His main research interests cover computational intelligence and its application to remote sensing image processing, hyperspectral remote sensing, complex disease pattern mining, etc. E-mail：xiegb@gdut.edu.cn
Supported by:
National Natural Science Foundation of China(61802072)

摘要/Abstract

摘要：

工业烟囱排放是城市空气污染的主要因素之一，城市环境质量与烟囱数量成反比。因此，烟囱位置检测对城市环境检测和治理具有积极的影响。在烟囱检测任务中，针对光学遥感图像背景复杂、目标小，存在大量相似对象导致的检测精度低的问题，提出了一种基于CenterNet的检测器P-CenterNet。首先，为了获得更丰富的语义特征，P-CenterNet使用了金字塔卷积取代骨干网络中的普通卷积；其次，并行于骨干网络设计了一个多尺度上下文特征提取模块来保留有助于从背景区域中区分对象区域的低级特征信息；最后，增加了一个卷积块注意力模块进一步提取骨干网络的输出特征，提高检测器对小目标的表达能力。实验使用了DIOR这个大规模的公开数据集来验证模型的有效性，采用线上、线下2种增强手段对数据集进行扩充，增强模型的鲁棒性。结果表明，与Faster-RCNN和YOLOv3这类模型相比，P-CenterNet在检测时间成本相近的情况下，明显提高了检测精度，mAP达到了89.77%。

关键词: 烟囱检测, 光学遥感图像, CenterNet, 金字塔卷积, 注意力机制

Abstract:

Industrial chimney emissions are among the primary drivers of urban air pollution, with the urban environment quality inversely related to the quantity of chimneys therein. Therefore, the detection of chimney placement exerts a positive impact on urban environmental detection and governance. To address the problem of low detection accuracy caused by complex backgrounds and small targets in optical remote sensing images with numerous similar objects, P-CenterNet, a CenterNet-based detector, has been proposed for chimney detection tasks. Firstly, P-CenterNet employed pyramidal convolution in the backbone network to obtain richer linguistic features, instead of normal convolution in the backbone network. Secondly, a multi-scale contextual feature extraction module was designed in parallel with the backbone network to retain low-level feature information that helped distinguish object regions from background regions. Finally, a convolutional block attention module was added to further extract the output features of the backbone network to improve the detector′s representation of small targets. Furthermore, DIOR, a large-scale public dataset, was applied for the validation of the model in the experiments. The dataset was expanded and the robustness of the model was enhanced via both online and offline enhancements. The results indicated that P-CentreNet could significantly improve detection accuracy with a similar detection time cost, compared with other models such as Faster-RCNN and YOLOv3, with mAP reaching 89.77%.

Key words: chimney detection, optical remote sensing images, CenterNet, pyramidal convolution, attention mechanism

中图分类号:

TP391

谢国波, 贺笛轩, 何宇钦, 林志毅. 基于P-CenterNet的光学遥感图像烟囱检测[J]. 图学学报, 2023, 44(2): 233-240.

XIE Guo-bo, HE Di-xuan, HE Yu-qin, LIN Zhi-yi. P-CenterNet for chimney detection in optical remote-sensing images[J]. Journal of Graphics, 2023, 44(2): 233-240.

图/表 9

表1 DIOR数据集的细节

Table 1 Details of DIOR dataset

参数	值
种类	20
分辨率	0.5~30.0 m
图像数量	23 463
包含烟囱目标的图像数量	854
烟囱实例	1 696
像素	800×800 pixels
年份	2018

图1 DIOR 数据集部分样本((a)具有复杂背景且检测目标极小的图像；(b)检测目标被烟囱工作时产生的白烟所遮挡的图像；(c)同一图像中包含不同尺寸大小的检测目标的图像；(d)包含与检测目标相似的物体的图像)

Fig. 1 DIOR dataset samples ((a) Shows an image with a complex background; (b) Shows an image in which the detection object is obscured by the white smoke produced by its operation; (c) Shows an image containing detection objects of different sizes in the same image; (d) Shows an image containing an object similar to the detection object)

图2 烟囱检测模型的架构

Fig. 2 Architecture of chimney detection

图3 普通卷积和金字塔卷积((a)普通卷积；(b)金字塔卷积)

Fig. 3 Ordinary convolution and PyConv ((a) Ordinary convolution; (b) PyConv)

图4 多尺度上下文特征提取模块

Fig. 4 Workflow of MSCF module

图5 卷积块注意力模块的使用

Fig. 5 Application of CBAM

图6 部分检测结果

Fig. 6 Some detection results

表2 对比实验结果

Table 2 Experimental results

模型	Backbone	评价指标
模型	Backbone	P (%)	R (%)	mAP0.75 (%)	FPS
Faster-RCNN	ResNet50	66.81	77.11	72.30	8.64
Yolov3	DarkNet53	83.95	81.93	77.81	24.52
CenterNet (baseline)	ResNet50	88.42	80.96	75.99	49.92
CenterNet	PvConv-ResNet50	88.63	82.65	80.80	28.26
P-CenterNet	PvConv-ResNet50	93.91	89.16	89.77	28.21

表3 消融实验结果

Table 3 Experimental results for each component

PyConv	MSCF	CBAM	评价指标
PyConv	MSCF	CBAM	P (%)	R (%)	mAP0.75 (%)	FPS	Params size (MB)
N	N	N	88.42	80.96	75.99	49.92	124.61
Y	N	N	88.63	82.65	80.80	28.69	121.90
Y	Y	N	89.11	84.82	82.61	28.61	121.96
Y	N	Y	93.89	88.92	88.93	28.56	125.92
Y	Y	Y	93.91	89.16	89.77	28.21	125.97

参考文献 25

[1]	任喜洋, 邓锋, 高兵, 等. 推动能源资源结构向绿色低碳转型[J]. 中国国土资源经济, 2021, 34(12): 48-54, 76.
	REN X Y, DENG F, GAO B, et al. Promote the transformation of energy and resource structure to green and low-carbon development[J]. Natural Resource Economics of China, 2021, 34(12): 48-54, 76. (in Chinese)
[2]	DENG Q, ZHANG H P. Chimney and condensing tower detection based on FPN in high resolution remote sensing images[C]// SPIE Remote Sensing. Proc SPIE 11155, Image and Signal Processing for Remote Sensing XXV. Bellingham: SPIE, 2019: 668-674.
[3]	HAN C M, LI G F, DING Y X, et al. Chimney detection based on faster R-CNN and spatial analysis methods in high resolution remote sensing images[J]. Sensors: Basel, Switzerland, 2020, 20(16): 4353.
[4]	姜志国, 张浩鹏, 姚远, 等. 一种基于遥感图像的烟囱和冷凝塔检测方法: CN108229524A[P]. 2018-06-29.
	JIANG Z G, ZHANG H P, YAO Y, et al. Chimney and condensing tower detection method based on remote-sensing image: CN108229524A[P]. 2018-06-29. (in Chinese)
[5]	YANG J C, ZHU Y H, JIANG B, et al. Aircraft detection in remote sensing images based on a deep residual network and Super-Vector coding[J]. Remote Sensing Letters, 2018, 9(3): 228-236. DOI URL
[6]	ZHENG J B, XI Y, FENG M C, et al. Object detection based on BING in optical remote sensing images[C]// The 9th International Congress on Image and Signal Processing, BioMedical Engineering and Informatics. New York: IEEE Press, 2016: 504-509.
[7]	LI B Y, CUI X G, BAI J. A cascade structure of aircraft detection in high resolution remote sensing images[C]// 2016 IEEE International Geoscience and Remote Sensing Symposium. New York: IEEE Press, 2016: 653-656.
[8]	WANG T, CHEN Y, QIAO M N, et al. Object detection in remote sensing images based on one-class classification[C]// 2017 Chinese Automation Congress. New York: IEEE Press, 2017: 6583-6587.
[9]	ZHAO Z Q, ZHENG P, XU S T, et al. Object detection with deep learning: a review[J]. IEEE Transactions on Neural Networks and Learning Systems, 2019, 30(11): 3212-3232. DOI URL
[10]	DUAN K W, BAI S, XIE L X, et al. CenterNet: keypoint triplets for object detection[C]// 2019 IEEE/CVF International Conference on Computer Vision. New York: IEEE Press, 2019: 6568-6577.
[11]	CHENG G, SI Y J, HONG H L, et al. Cross-scale feature fusion for object detection in optical remote sensing images[J]. IEEE Geoscience and Remote Sensing Letters, 2021, 18(3): 431-435. DOI URL
[12]	GIRSHICK R, DONAHUE J, DARRELL T, et al. Rich feature hierarchies for accurate object detection and semantic segmentation[C]// 2014 IEEE Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2014: 580-587.
[13]	GIRSHICK R. Fast R-CNN[C]// 2015 IEEE International Conference on Computer Vision. New York: IEEE Press, 2015: 1440-1448.
[14]	REN S Q, HE K M, GIRSHICK R, et al. Faster R-CNN: towards real-time object detection with region proposal networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(6): 1137-1149. DOI PMID
[15]	HE K M, GKIOXARI G, DOLLÁR P, et al. Mask R-CNN[C]// 2017 IEEE International Conference on Computer Vision. New York: IEEE Press, 2017: 2980-2988.
[16]	YAO Y, JIANG Z G, ZHANG H P, et al. Chimney and condensing tower detection based on faster R-CNN in high resolution remote sensing images[C]// 2017 IEEE International Geoscience and Remote Sensing Symposium. New York: IEEE Press, 2017: 3329-3332.
[17]	ZHANG H P, DENG Q. Deep learning based fossil-fuel power plant monitoring in high resolution remote sensing images: a comparative study[J]. Remote Sensing, 2019, 11(9): 1117. DOI URL
[18]	REDMON J, DIVVALA S, GIRSHICK R, et al. You only look once: unified, real-time object detection[C]// 2016 IEEE Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2016: 779-788.
[19]	REDMON J, FARHADI A. YOLOv3: an incremental improvement[EB/OL]. (2018-04-08) [2022-05-27]. https://arxiv.org/abs/1804.02767.
[20]	LAW H, DENG J. CornerNet: detecting objects as paired keypoints[M]//Computer Vision - ECCV 2018. Cham: Springer International Publishing, 2018: 765-781.
[21]	DUTA I C, LIU L, ZHU F, et al. Pyramidal convolution: rethinking convolutional neural networks for visual recognition[EB/OL]. (2020-06-20) [2022-05-27]. https://arxiv.org/abs/2006.11538.
[22]	NIE J, ANWER R M, CHOLAKKAL H, et al. Enriched feature guided refinement network for object detection[C]// 2019 IEEE/CVF International Conference on Computer Vision. New York: IEEE Press, 2019: 9536-9545.
[23]	WOO S, PARK J, LEE J Y, et al. CBAM: convolutional block attention module[M]// Computer Vision-ECCV 2018. Cham: Springer International Publishing, 2018: 3-19.
[24]	LI K, WAN G, CHENG G, et al. Object detection in optical remote sensing images: a survey and a new benchmark[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2020, 159: 296-307. DOI URL
[25]	ZHANG G J, LU S J, ZHANG W. CAD-net: a context-aware detection network for objects in remote sensing imagery[J]. IEEE Transactions on Geoscience and Remote Sensing, 2019, 57(12): 10015-10024. DOI URL

基于P-CenterNet的光学遥感图像烟囱检测

P-CenterNet for chimney detection in optical remote-sensing images

RichHTML

PDF (PC)

可视化

摘要/Abstract

引用本文

使用本文

图/表 9

参考文献 25

相关文章 15

编辑推荐

Metrics

本文评价

[1]	李利霞, 王鑫, 王军, 张又元 . 基于特征融合与注意力机制的无人机图像小目标检测算法 [J]. 图学学报, 2023, 44(4): 658-666.
[2]	李鑫, 普园媛, 赵征鹏, 徐丹, 钱文华 . 内容语义和风格特征匹配一致的艺术风格迁移 [J]. 图学学报, 2023, 44(4): 699-709.
[3]	余伟群, 刘佳涛, 张亚萍. 融合注意力的拉普拉斯金字塔单目深度估计 [J]. 图学学报, 2023, 44(4): 728-738.
[4]	胡欣, 周运强, 肖剑, 杨杰. 基于改进YOLOv5的螺纹钢表面缺陷检测[J]. 图学学报, 2023, 44(3): 427-437.
[5]	郝鹏飞, 刘立群, 顾任远. YOLO-RD-Apple果园异源图像遮挡果实检测模型[J]. 图学学报, 2023, 44(3): 456-464.
[6]	罗文宇, 傅明月. 基于YoloX-ECA模型的非法野泳野钓现场监测技术[J]. 图学学报, 2023, 44(3): 465-472.
[7]	李雨, 闫甜甜, 周东生, 魏小鹏. 基于注意力机制与深度多尺度特征融合的自然场景文本检测[J]. 图学学报, 2023, 44(3): 473-481.
[8]	吴文欢, 张淏坤. 融合空间十字注意力与通道注意力的语义分割网络[J]. 图学学报, 2023, 44(3): 531-539.
[9]	熊举举, 徐杨, 范润泽, 孙少聪. 基于轻量化视觉Transformer的花卉识别[J]. 图学学报, 2023, 44(2): 271-279.
[10]	成浪, 敬超. 基于改进YOLOv7的X线图像旋转目标检测[J]. 图学学报, 2023, 44(2): 324-334.
[11]	曹义亲, 伍铭林, 徐露. 基于改进YOLOv5算法的钢材表面缺陷检测[J]. 图学学报, 2023, 44(2): 335-345.
[12]	张伟康, 孙浩, 陈鑫凯, 李叙兵, 姚立纲, 东辉. 基于改进YOLOv5的智能除草机器人蔬菜苗田杂草检测研究[J]. 图学学报, 2023, 44(2): 346-356.
[13]	李小波 , 李阳贵 , 郭宁 , 范震 . 融合注意力机制的 YOLOv5 口罩检测算法[J]. 图学学报, 2023, 44(1): 16-25.
[14]	邵文斌, 刘玉杰, 孙晓瑞, 李宗民. 基于残差增强注意力的跨模态行人重识别[J]. 图学学报, 2023, 44(1): 33-40.
[15]	单芳湄, 王梦文, 李敏. 融合注意力机制的肠道息肉分割多尺度卷积神经网络 [J]. 图学学报, 2023, 44(1): 50-58.