Small-target worker detection on construction sites

doi:10.11996/JG.j.2095-302X.2024051040

Abstract

Abstract:

The accurate detection of workers using construction site surveillance videos or images can support the intelligent management of construction safety. However, the long-distance surveillance causes workers to appear as small targets in images, and the complex and changing environment further complicates worker detection. To address this problem, a small-target worker detection method integrating an improved YOLO model with the frame difference method was proposed. On the one hand, the stationary workers were detected through the improved YOLOv5 model, which introduced slicing aided inference to obtain the clearer features of small-target workers, added small-target detection heads to ensure feature completeness, and employed the ECA mechanism to improve the detection performance. On the other hand, the moving workers with weak image features were detected using the frame difference method, to some extent compensating for the shortcomings of image detection. The proposed method was validated on a self-constructed dataset, and the results showed that the F1-Score and mAP50 of the improved YOLOv5 model improved by 11.3% and 12.5%, respectively. Additionally, the detection rate of small-target workers using the integrated method increased by 3.6% to 84.2%, with a FPS of 6 frames·per second. Thus, the proposed method can better satisfy the needs of worker detection on construction sites.

Key words: site worker detection, small target detection, YOLO, frame difference method

CLC Number:

TP391
TU714

LI Jianhua, HAN Yu, SHI Kaiming, ZHANG Kejia, GUO Hongling, FANG Dongping, CAO Jiaming. Small-target worker detection on construction sites[J]. Journal of Graphics, 2024, 45(5): 1040-1049.

Figures/Tables 19

Fig. 1 Slicing aided hyper inference flow

Fig. 2 Block overlapping inference diagram ((a) The cuts do not overlap; (b) The cuts overlap)

Fig. 3 Image segmentation effect ((a) The original image; (b) Segmented subgraph)

Table 1 Effect comparison of YOLOv5 and v8/%

模型	P	R	mAP50	mAP50:95
YOLOv5m	76.7	64.8	72.1	35.1
YOLOv8m	70.1	58.1	63.4	30.5

Fig. 4 YOLOv5 network structure

Fig. 5 The height and width of the label distribution diagram

Fig. 6 YOLO detect head diagram

Table 2 The size of the clustered anchor frame/Pixel

检测头	20×20	40×40	80×80	160×160
小	(116,90)	(30,61)	(10,13)	(4,5)
中	(156,198)	(62.45)	(16,30)	(8,10)
大	(373,326)	(59,119)	(33,23)	(22,18)

Fig. 7 Structure comparison between SE and ECA ((a) SE; (b) ECA)

Fig. 8 Improved YOLOv5 network structure

Fig. 9 Frame difference method flow chart

Fig. 10 Frame difference method effect display ((a) The original image; (b) The pixel-difference map using two-frame differences; (c) The motion region map after binarization and morphological processing; (d) Box marked motion area map)

Fig. 11 IOU distribution

Fig. 12 Dataset image example

Table 3 Performance of SAHI

模型	P/%	R/%	mAP50/ %	mAP50:95/ %	FPS/ 帧每秒
YOLO-v5m	76.7	64.8	72.1	35.1	54.0
SAHI-1280	78.5	68.4	74.3	35.9	19.0
SAHI-640	81.2	71.6	75.2	36.7	8.0
SAHI-320	83.9	73.9	78.1	37.4	3.1

Table 4 Comparison of model optimization effect

模型	F1/%	P/%	R/%	mAP50/%	mAP50:95/%	FPS/帧每秒
YOLOv5m	70.2	76.7	64.8	72.1	35.1	54
小目标检测头	76.4	76.8	76.0	77.6	36.2	27
YOLOv5m-SE	75.7	78.8	72.8	78.7	33.7	43
YOLOv5m-CBAM	74.0	71.4	76.9	77.1	32.8	44
YOLOv5m-ECA	76.8	77.9	75.7	78.0	33.3	40
YOLOv5m-CA	75.6	79.5	72.0	77.7	33.0	41
YOLOv5m-GAM	75.9	75.7	76.1	79.1	38.4	25

Table 5 Ablation experiment result

SAHI	检测头	ECA	F1/%	P/%	R/%	mAP50/%	FPS/帧每秒
-	-	-	70.2	76.7	64.8	72.1	54
√	-	-	76.1	81.2	71.6	75.2	8
-	√	-	76.4	76.8	76.0	77.6	27
-	-	√	76.8	77.9	75.7	78.0	40
√	√	-	81.1	80.5	81.8	84.9	5
√	-	√	81.5	82.8	80.2	84.6	6
-	√	√	77.3	75.4	79.3	80.2	13
√	√	√	78.2	80.0	76.4	82.6	5

Fig. 13 Comparison between Improved YOLOv5 and frame difference method for independent use ((a) Comparison results，figure (a1) is the original image frame, figure (a2) is the detection result of frame difference method, figure (a3) is the motion region detected by frame difference method, and figure (a4) is the detection result of the improved YOLOv5 model; (b) Local amplification)

Table 6 Fusion algorithm effect comparison

模型	R/%	T/ms
改进的YOLOv5	80.6	265
帧差法	53.1	37
融合算法	84.2	313

References 27

[1]	段锐, 邓晖, 邓逸川. ICT支持的塔吊安全管理框架——回顾与展望[J]. 图学学报, 2022, 43(1): 11-20.
	DUAN R, DENG H, DENG Y C. Information communications technology assisted tower crane safety management-review and prospect[J]. Journal of Graphics, 2022, 43(1): 11-20 (in Chinese).
[2]	HAN K, ZENG X D. Deep learning-based workers safety helmet wearing detection on construction sites using multi-scale features[J]. IEEE Access, 2022, 10: 718-729.
[3]	李华, 薛曦澄, 吴立舟, 等. 深度学习下吊装作业工人防护装备及吊钩检测方法[J]. 安全与环境学报, 2024, 24(3): 1027-1035.
	LI H, XUE X C, WU L Z, et al. Protective equipment and hook testing methods for workers in lifting operations under deep learning[J]. Journal of Safety and Environment, 2024, 24(3): 1027-1035 (in Chinese).
[4]	WU H, HAN Y, ZHANG M, et al. Identifying unsafe behavior of construction workers: a dynamic approach combining skeleton information and spatiotemporal features[J]. Journal of Construction Engineering and Management, 2023, 149(11): 04023115.
[5]	张知田, 王园园, 罗柱邦, 等. 塔吊与工人空间交互下危险场景自动检测[J]. 清华大学学报: 自然科学版, 2024, 64(2): 198-204.
	ZHANG Z T, WANG Y Y, LUO Z B, et al. Automatic detection of hazardous scenarios during spatial interaction between tower cranes and workers[J]. Journal of Tsinghua University: Science and Technology, 2024, 64(2): 198-204 (in Chinese).
[6]	GIRSHICK R, DONAHUE J, DARRELL T, et al. Region-based convolutional networks for accurate object detection and segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2016, 38(1): 142-158. DOI PMID
[7]	GIRSHICK R, DONAHUE J, DARRELL T, et al. Rich feature hierarchies for accurate object detection and semantic segmentation[C]// 2014 IEEE Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2014: 580-587.
[8]	REN S Q, HE K M, GIRSHICK R, et al. Faster R-CNN: towards real-time object detection with region proposal networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(6): 1137-1149. DOI PMID
[9]	HE K M, GKIOXARI G, DOLLÁR P, et al. Mask R-CNN[C]// 2017 IEEE International Conference on Computer Vision. New York: IEEE Press, 2017: 2980-2988.
[10]	REDMON J, DIVVALA S, GIRSHICK R, et al. You only look once: unified, real-time object detection[C]// 2016 IEEE Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2016: 779-788.
[11]	LIU W, ANGUELOV D, ERHAN D, et al. SSD: single shot MultiBox detector[C]// The 14th European Conference on Computer Vision. Cham: Springer, 2016: 21-37.
[12]	曹坤煜, 陈述, 陈云, 等. 混凝土坝面交叉作业安全风险智能识别方法[J]. 水力发电学报, 2023, 42(12): 146-158.
	CAO K Y, CHEN S, CHEN Y, et al. Intelligent identification method for safety risks in cross operation on concrete dam surface[J]. Journal of Hydroelectric Engineering, 2023, 42(12): 146-158 (in Chinese).
[13]	邢金昊, 饶颖露, 张恒, 等. 基于改进YOLO的建筑预埋件检测算法[J]. 计算机应用与软件, 2021, 38(11): 179-184.
	XING J H, RAO Y L, ZHANG H, et al. Pre-buried parts detections based on improved YOLO model[J]. Computer Applications and Software, 2021, 38(11): 179-184 (in Chinese).
[14]	LI H B, WU D C, ZHANG W M, et al. YOLO-PL: helmet wearing detection algorithm based on improved YOLOv4[J]. Digital Signal Processing, 2024, 144: 104283.
[15]	KOYUN O C, KESER R K, AKKAYA İ B, et al. Focus-and-Detect: a small object detection framework for aerial images[J]. Signal Processing: Image Communication, 2022, 104: 116675.
[16]	YANG C H Y, HUANG Z H, WANG N Y. QueryDet: cascaded sparse query for accelerating high-resolution small object detection[C]// 2022 IEEE CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2022: 13658-13667.
[17]	GUO M H, XU T X, LIU J J, et al. Attention mechanisms in computer vision: a survey[J]. Computational Visual Media, 2022, 8(3): 331-368.
[18]	MITTAL P, SHARMA A, SINGH R, et al. Dilated convolution based RCNN using feature fusion for low-altitude aerial objects[J]. Expert Systems with Applications, 2022, 199: 117106.
[19]	XUE Y J, JU Z Y, LI Y M, et al. MAF-YOLO: multi-modal attention fusion based YOLO for pedestrian detection[J]. Infrared Physics & Technology, 2021, 118: 103906.
[20]	李小波, 李阳贵, 郭宁, 等. 融合注意力机制的YOLOv5口罩检测算法[J]. 图学学报, 2023, 44(1): 16-25. DOI
	LI X B, LI Y G, GUO N, et al. Mask detection algorithm based on YOLOv5 integrating attention mechanism[J]. Journal of Graphics, 2023, 44(1): 16-25 (in Chinese).
[21]	AKYON F C, ALTINUC S O, TEMIZEL A. Slicing aided hyper inference and fine-tuning for small object detection[C]// 2022 IEEE International Conference on Image Processing. New York: IEEE Press, 2022: 966-970.
[22]	KIM S, HONG S H, KIM H, et al. Small object detection (SOD) system for comprehensive construction site safety monitoring[J]. Automation in Construction, 2023, 156: 105103.
[23]	HU J, SHEN L, SUN G. Squeeze-and-excitation networks[C]// 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2018: 7132-7141.
[24]	WANG Q L, WU B G, ZHU P F, et al. ECA-Net: efficient channel attention for deep convolutional neural networks[C]// 2020 IEEE CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2020: 11531-11539.
[25]	FAN Y, TIAN L W, YANG L. Moving object detection based on enhanced frame difference[C]// The 3rd International Conference on Computer Science and Communication Technology. Bellingham: SPIE, 2022: 125061C.
[26]	王玉萍, 曾毅. 结合帧差的核相关滤波弱小红外目标检测[J]. 红外技术, 2023, 45(7): 755-767.
	WANG Y P, ZENG Y. Weak and small infrared target detection combined with frame difference kernel correlation filtering[J]. Infrared Technology, 2023, 45(7): 755-767 (in Chinese).
[27]	HE H, MA S C, SUN L. Multi-moving target detection based on the combination of three frame difference algorithm and background difference algorithm[C]// 2018 WRC Symposium on Advanced Robotics and Automation. New York: IEEE Press, 2018: 141-146.