Object detection in remote sensing image based on two-stage anchor and class balanced loss

doi:10.11996/JG.j.2095-302X.2023020249

Abstract

Abstract:

Due to the large differences in the number of different categories of targets in the existing remote sensing image data collection, the distribution of categories in the dataset is unbalanced, affecting the detection accuracy of network models for a few categories. In light of the aforementioned challenges, a two-stage anchor frame and class-balanced loss target detection algorithm for remote sensing images was presented. The class balance labels of remote sensing datasets were generated by K-means clustering, subsequently utilized as the initial center for the second stage of K-means clustering. The resulting preset anchor frames were able to take into account a few class scales and improve the detection accuracy of a few class instances. At the same time, class equalization loss (CEQL) was constructed. Based on equalization loss (EQL), effective samples were used to construct auxiliary weights to improve the model′s attention to a few categories during training. The experimental results demonstrated that the improved model achieved an average accuracy of 76.13% and a few categories′ average accuracy of 76.51%, increasing by 1.56% and 1.75%, respectively, compared with the datum network. When evaluated on the DOIR and NWPU VHR-10 datasets, and compared with the main methods such as Faster-RCNN, RetinaNet, CenterNet, YOLOv4, YOLOX-L, YOLOv5, and YOLOv7, the experiment showed that the improved algorithm could effectively improve the detection accuracy of a few categories while maintaining the detection accuracy of most categories.

Key words: remote sensing detection, class imbalance, re-weighting, K-means, YOLOv4

CLC Number:

ZENG Lun-jie, CHU Jun, CHEN Zhao-jun. Object detection in remote sensing image based on two-stage anchor and class balanced loss[J]. Journal of Graphics, 2023, 44(2): 249-259.

Figures/Tables 14

References 33

[1]	REN S Q, HE K M, GIRSHICK R, et al. Faster R-CNN: towards real-time object detection with region proposal networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(6): 1137-1149. DOI PMID
[2]	REDMON J, DIVVALA S, GIRSHICK R, et al. You only look once: unified, real-time object detection[C]// 2016 IEEE Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2016: 779-788.
[3]	LIU W, ANGUELOV D, ERHAN D, et al. SSD: single shot MultiBox detector[M]//Computer Vision - ECCV 2016. Cham: Springer International Publishing, 2016: 21-37.
[4]	TIAN Z, SHEN C H, CHEN H, et al. FCOS: fully convolutional one-stage object detection[C]// 2019 IEEE/CVF International Conference on Computer Vision. New York: IEEE Press, 2019: 9626-9635.
[5]	DUAN K W, BAI S, XIE L X, et al. CenterNet: keypoint triplets for object detection[C]// 2019 IEEE/CVF International Conference on Computer Vision. New York: IEEE Press, 2019: 6568-6577.
[6]	胡俊, 顾晶晶, 王秋红. 基于遥感图像的多模态小目标检测[J]. 图学学报, 2022, 43(2): 197-204.
	HU J, GU J J, WANG Q H. Multimodal small target detection based on remote sensing image[J]. Journal of Graphics, 2022, 43(2): 197-204. (in Chinese)
[7]	张燕, 高鑫, 刘以, 等. 基于改进像素相关性模型的图像分割算法[J]. 图学学报, 2022, 43(2): 205-213.
	ZHANG Y, GAO X, LIU Y, et al. Image segmentation algorithm based on improved pixel correlation model[J]. Journal of Graphics, 2022, 43(2): 205-213. (in Chinese)
[8]	LIU M J, WANG X H, ZHOU A J, et al. UAV-YOLO: small object detection on unmanned aerial vehicle perspective[J]. Sensors: Basel, Switzerland, 2020, 20(8): 2238.
[9]	YANG X, SUN H, SUN X, et al. Position detection and direction prediction for arbitrary-oriented ships via multitask rotation region convolutional neural network[J]. IEEE Access, 2018, 6: 50839-50849. DOI URL
[10]	YANG X, YANG J R, YAN J C, et al. SCRDet: towards more robust detection for small, cluttered and rotated objects[C]// 2019 IEEE/CVF International Conference on Computer Vision. New York: IEEE Press, 2019: 8231-8240.
[11]	XIA G S, BAI X, DING J, et al. DOTA: a large-scale dataset for object detection in aerial images[C]// 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2018: 3974-3983.
[12]	LI K, WAN G, CHENG G, et al. Object detection in optical remote sensing images: a survey and a new benchmark[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2020, 159: 296-307. DOI URL
[13]	CHENG G, ZHOU P C, HAN J W. Learning rotation-invariant convolutional neural networks for object detection in VHR optical remote sensing images[J]. IEEE Transactions on Geoscience and Remote Sensing, 2016, 54(12): 7405-7415. DOI URL
[14]	BOTÍA J A, VANDROVCOVA J, FORABOSCO P, et al. An additional k-means clustering step improves the biological features of WGCNA gene co-expression networks[J]. BMC Systems Biology, 2017, 11(1): 47. DOI PMID
[15]	ZHANG Y F, KANG B Y, HOOI B, et al. Deep long-tailed learning: a survey[EB/OL]. [2022-07-09]. https://arxiv.org/abs/2110.04596.
[16]	KIM J, JEONG J, SHIN J. M2m: imbalanced classification via major-to-minor translation[C]// 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2020: 13893-13902.
[17]	TAN C Q, SUN F C, KONG T, et al. A survey on deep transfer learning[M]//Artificial Neural Networks and Machine Learning - ICANN 2018. Cham: Springer International Publishing, 2018: 270-279.
[18]	YIN X, YU X, SOHN K, et al. Feature transfer learning for face recognition with under-represented data[C]// 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2019: 5697-5706.
[19]	ZHOU B Y, CUI Q, WEI X S, et al. BBN: bilateral-branch network with cumulative learning for long-tailed visual recognition[C]// 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2020: 9716-9725.
[20]	CAI J R, WANG Y Z, HWANG J N. ACE: ally complementary experts for solving long-tailed recognition in one-shot[C]// 2021 IEEE/CVF International Conference on Computer Vision. New York: IEEE Press, 2021: 112-121.
[21]	HUANG C, LI Y N, LOY C C, et al. Learning deep representation for imbalanced classification[C]// 2016 IEEE Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2016: 5375-5384.
[22]	LIN T Y, GOYAL P, GIRSHICK R, et al. Focal loss for dense object detection[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020, 42(2): 318-327. DOI URL
[23]	TAN J R, WANG C B, LI B Y, et al. Equalization loss for long-tailed object recognition[C]// 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2020: 11659-11668.
[24]	CUI Y, JIA M L, LIN T Y, et al. Class-balanced loss based on effective number of samples[C]// 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2019: 9260-9269.
[25]	BOCHKOVSKIY A, WANG C Y, LIAO H Y M. YOLOv4: optimal speed and accuracy of object detection[EB/OL]. [2022-07-09]. https://arxiv.org/abs/2004.10934.
[26]	ZAIDI S S A, ANSARI M S, ASLAM A, et al. A survey of modern deep learning based object detection models[EB/OL]. [2022-07-09]. https://arxiv.org/abs/2104.11892.
[27]	REDMON J, FARHADI A. YOLO9000: better, faster, stronger[C]// 2017 IEEE Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2017: 6517-6525.
[28]	邓聪颖, 叶波, 苗建国, 等. 基于K-means++聚类与概率神经网络的数控机床变位姿动态特性模糊评估[J]. 仪器仪表学报, 2020, 41(12): 227-235.
	DENG C Y, YE B, MIAO J G, et al. Fuzzy evaluation of machine tool dynamic characteristics for changing machining position based on K-means + + clustering and probabilistic neural network[J]. Chinese Journal of Scientific Instrument, 2020, 41(12): 227-235. (in Chinese)
[29]	LI Y, WANG T, KANG B Y, et al. Overcoming classifier imbalance for long-tail object detection with balanced group softmax[C]// 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2020: 10988-10997.
[30]	GUPTA A, DOLLÁR P, GIRSHICK R. LVIS: a dataset for large vocabulary instance segmentation[C]// 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2019: 5351-5359.
[31]	GE Z, LIU S T, WANG F, et al. YOLOX: exceeding YOLO series in 2021[EB/OL]. [2022-07-09]. https://arxiv.org/abs/2107.08430.
[32]	ZHU X K, LYU S C, WANG X, et al. TPH-YOLOv5: improved YOLOv5 based on transformer prediction head for object detection on drone-captured scenarios[C]// 2021 IEEE/CVF International Conference on Computer Vision Workshops. New York: IEEE Press, 2021: 2778-2788.
[33]	WANG C Y, BOCHKOVSKIY A, LIAO H Y M. YOLOv7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors[EB/OL]. [2022-07-06]. https://arxiv.org/abs/2207.02696.

方法	AP_r	AP_c	AP_f	mAP	FPS
YOLOv4	74.76	83.17	67.85	74.57	45.73
+ K-means	73.92	81.75	66.45	73.58	46.39
+ K-means++	74.73	82.65	67.75	74.47	45.94
+TK-means (Ours)	75.65	84.62	68.09	75.41	45.67

方法	AP_r	AP_c	AP_f	mAP	FPS
YOLOv4	74.76	83.17	67.85	74.57	45.73
+ K-means	73.92	81.75	66.45	73.58	46.39
+ K-means++	74.73	82.65	67.75	74.47	45.94
+TK-means (Ours)	75.65	84.62	68.09	75.41	45.67

方法	AP_r	AP_c	AP_f	mAP	FPS
YOLOv4	74.76	83.17	67.86	74.57	45.73
+ Focal Loss	73.31	82.88	67.30	73.38	45.99
+ EQL	57.82	83.85	67.43	61.87	45.33
+ CB Loss	11.17	35.26	41.16	18.08	46.45
+ CEQL (Ours)	75.99	84.85	68.27	75.72	45.78

方法	AP_r	AP_c	AP_f	mAP	FPS
YOLOv4	74.76	83.17	67.86	74.57	45.73
+ Focal Loss	73.31	82.88	67.30	73.38	45.99
+ EQL	57.82	83.85	67.43	61.87	45.33
+ CB Loss	11.17	35.26	41.16	18.08	46.45
+ CEQL (Ours)	75.99	84.85	68.27	75.72	45.78

数据集	TK-means	CEQL	AP_r	AP_c	AP_f	mAP
DIOR	-	-	74.76	83.17	67.85	74.57
	√	-	75.65	84.62	68.09	75.41
	-	√	75.99	84.85	68.27	75.72
	√	√	76.51	84.53	68.62	76.13
NWPU VHR-10	-	-	82.75	89.19	94.55	89.94
	√	-	84.68	91.62	94.84	91.15
	-	√	85.06	92.11	95.12	91.50
	√	√	85.72	93.05	95.04	91.85