Research on efficient detection model of tunnel lining crack based on DCNv2 and Transformer Decoder

doi:10.11996/JG.j.2095-302X.2024051050

Abstract

Abstract:

To address the problems of low recognition accuracy, slow detection speed, and large parameter quantities caused by the random and dense distribution of cracks in tunnel linings and low resolution of annotation boxes in existing models, the YOLOv8 network framework was improved based on the Deformable Convolution Network version 2 (DCNv2) and end-to-end Transformer Decoder to propose a lining crack detection model DTD-YOLOv8. Firstly, DCNv2 was added to fuse the YOLOv8 backbone convolutional network C2f, enabling the model to accurately and quickly perceive crack deformation features. At the same time, the Transformer Decoder replaced the YOLOv8 detection head to achieve a complete object detection process within an end-to-end framework, thereby eliminating the computational consumption caused by the Anchor-free processing mode. A self-built crack dataset was used to compare and verify seven detection models, including SSD, Faster-RCNN, RT-DETR, YOLOv3, YOLOv5, YOLOv8, and DTD-YOLOv8. The results indicated that the F1 score and mAP@50 of DTD-YOLOv8 reached 87.05% and 89.58%, respectively. Compared to the other six models, the F1 score increased by 14.16%, 7.68%, 1.55%, 41.36%, 8.20%, and 7.40%, while the mAP@50 increased by 28.84%, 15.47%, 1.33%, 47.65%, 10.14%, and 10.84%. The parameter count of the new model was only one-third of RT-DETR, and the detection speed for a single image was 16.01 ms, with an FPS of 65.46 frames per second, demonstrating a speed improvement over other comparative model. The DTD-YOLOv8 model can demonstrate efficient performance in meeting the requirements of crack detection tasks in operational tunnels.

Key words: tunnel engineering, object detection, deformable convolution network v2, Transformer Decoder, lining crack

CLC Number:

SUN Jilong, LIU Yong, ZHOU Liwei, LU Xin, HOU Xiaolong, WANG Yaqiong, WANG Zhifeng. Research on efficient detection model of tunnel lining crack based on DCNv2 and Transformer Decoder[J]. Journal of Graphics, 2024, 45(5): 1050-1061.

Figures/Tables 15

Fig. 1 Deformable convolutional kernel sampling ((a) Standard convolution; (b) Deformable convolution)

Fig. 2 Convolutional visualization diagram of tunnel lining crack images

Fig. 3 RT-DETR model structure diagram

Fig. 4 YOLOv8 model structure diagram

Fig. 5 DTD-YOLOv8 model structure diagram

Fig. 6 Partial tunnel lining crack diagram display

Fig. 7 Diagram of image expansion operation

Table 1 Experimental environment configuration

工具	环境
操作系统	Ubuntu
GPU	NVIDIA GeForce RTX 3080 Ti(12 GB) * 1GPU
编译器	PyCharm
框架	Pytorch2.0.0+CUDA11.8
语言	Python3.8

Fig. 8 Backbone feature extraction network

Table 2 Experimental results of adding DCN v2 at different position/%

模型	P	R	mAP@50	mAP@50:95	F1
YOLOv8	86.32	76.38	80.82	63.22	80.05
YOLOv-DCNv2-①	88.17	78.24	83.61	66.41	82.91
YOLOv8-DCNv2-②	88.18	77.63	83.52	66.82	82.57
YOLOv8-DCNv2-③	87.82	78.01	84.03	66.22	82.62
YOLOv8-DCNv2-④	89.08	79.05	83.87	66.98	83.76
YOLOv-DCNv2-①+②	87.21	77.21	80.52	64.31	81.91
YOLOv-DCNv2-①+③	87.34	76.92	82.92	66.14	81.80
YOLOv-DCNv2-①+④	88.02	78.54	83.01	66.73	83.01
YOLOv-DCNv2-②+③	87.91	78.39	82.16	65.98	82.88
YOLOv-DCNv2-②+④	87.83	79.01	83.27	66.23	83.19
YOLOv-DCNv2-①+②+③	86.15	76.31	80.26	65.87	80.93
YOLOv-DCNv2-①+③+④	85.82	77.26	81.38	66.08	81.32
YOLOv-DCNv2-②+③+④	86.96	77.53	82.23	66.33	81.97
YOLOv-DCNv2-①+②+③+④	86.52	76.94	81.07	65.97	81.45

Fig. 9 Comparison of differences in visualization of heat maps among different models ((a) Original images; (b) RT-DETR; (c) YOLOv8; (d) YOLOv8-DCNv2-④)

Fig. 10 Comparison of loss function results from DTD-YOLOv8 model

Table 3 Comparison of ablation test results

方案	mAP@50/%	mAP@50:95/%	F1/%	FPS/帧每秒	Par/M	GFLOPS/G
YOLOv8	80.82	63.22	80.05	45.57	3.1	8.9
YOLOv8-Decoder	84.67	69.35	81.52	58.21	9.5	16.7
YOLOv8-DCNv2-④	83.87	66.98	83.76	52.48	3.0	8.7
DTD-YOLOv8	89.58	75.32	87.05	65.46	6.1	11.6

Fig. 11 Comparison of performance of different models ((a) Accuracy indicators; (b) Speed indicator)

Fig. 12 Comparison of visual inspection results for tunnel lining crack detection ((a) Original images; (b) SSD; (c) Faster RCNN; (d) RT-DETR; (e) YOLOv5n; (f) YOLOv8n; (g) DTD-YOLOv8)

References 34

[1]	ZHANG L X, SHEN J K, ZHU B J. A review of the research and application of deep learning-based computer vision in structural damage detection[J]. Earthquake Engineering and Engineering Vibration, 2022, 21(1): 1-21.
[2]	刘新根, 陈莹莹, 朱爱玺, 等. 基于深度学习的隧道裂缝识别方法[J]. 广西大学学报: 自然科学版, 2018, 43(6): 2243-2251.
	LIU X G, CHEN Y Y, ZHU A X, et al. Tunnel crack identification based on deep learning[J]. Journal of Guangxi University: Natural Science Edition, 2018, 43(6): 2243-2251 (in Chinese).
[3]	刘勇, 王志丰, 王亚琼, 等. 基于案例推理的隧道洞门智能设计方法研究[J]. 铁道标准设计, 2024, 68(6): 137-144.
	LIU Y, WANG Z F, WANG Y Q, et al. Investigation on intelligent design method of tunnel portal based on case-based reasoning[J]. Railway Standard Design, 2024, 68(6): 137-144 (in Chinese).
[4]	刘学增, 李振, 杨芝璐, 等. 隧道衬砌裂缝特征对承载力的影响分析[J]. 哈尔滨工业大学学报, 2024, 56(7): 37-45.
	LIU X Z, LI Z, YANG Z L, et al. Analysis of the influence of tunnel lining cracks on bearing capacity[J]. Journal of Harbin Institute of Technology, 2024, 56(7): 37-45 (in Chinese).
[5]	ZHANG A, WANG K C P, LI B X, et al. Automated pixel-level pavement crack detection on 3D asphalt surfaces using a deep-learning network[J]. Computer-Aided Civil and Infrastructure Engineering, 2017, 32(10): 805-819.
[6]	FENG D S, WANG X, ZHANG B. Specific evaluation of tunnel lining multi-defects by all-refined GPR simulation method using hybrid algorithm of FETD and FDTD[J]. Construction and Building Materials, 2018, 185: 220-229.
[7]	YU T T, ZHU A X, CHAN Y Y. Efficient crack detection method for tunnel lining surface cracks based on infrared images[J]. Journal of Computing in Civil Engineering, 2017, 31(3): 04016067.
[8]	ZHOU S L, SONG W. Deep learning-based roadway crack classification using laser-scanned range images: a comparative study on hyperparameter selection[J]. Automation in Construction, 2020, 114: 103171.
[9]	杨泽洲, 陈思榕, 高翠芸, 等. 基于深度学习的代码生成方法研究进展[J]. 软件学报, 2024, 35(2): 604-628.
	YANG Z Z, CHEN S R, GAO C Y, et al. Deep learning based code generation methods: literature review[J]. Journal of Software, 2024, 35(2): 604-628 (in Chinese).
[10]	窦慧, 张凌茗, 韩峰, 等. 卷积神经网络的可解释性研究综述[J]. 软件学报, 2024, 35(1): 159-184.
	DOU H, ZHANG L M, HAN F, et al. Survey on convolutional neural network interpretability[J]. Journal of Software, 2024, 35(1): 159-184 (in Chinese).
[11]	REN Y P, HUANG J S, HONG Z Y, et al. Image-based concrete crack detection in tunnels using deep fully convolutional networks[J]. Construction and Building Materials, 2020, 234: 117367.
[12]	朱鑫. 隧道结构表面病害特征快速检测研究[D]. 成都: 西南交通大学, 2014.
	ZHU X. The research on rapid detection of characteristics of tunnel lining surface diseases[D]. Chengdu: Southwest Jiaotong University, 2014 (in Chinese).
[13]	曹义亲, 周一纬, 徐露. 基于E-YOLOX的实时金属表面缺陷检测算法[J]. 图学学报, 2023, 44(4): 677-690. DOI
	CAO Y Q, ZHOU Y W, XU L. A real-time metallic surface defect detection algorithm based on E-YOLOX[J]. Journal of Graphics, 2023, 44(4): 677-690 (in Chinese).
[14]	杨柳, 吴晓群. 基于深度学习的三维形状补全研究综述[J]. 图学学报, 2023, 44(2): 201-215. DOI
	YANG L, WU X Q. 3D shape completion via deep learning: a method survey[J]. Journal of Graphics, 2023, 44(2): 201-215 (in Chinese). DOI
[15]	阎光伟, 刘润泽, 焦润海, 等. 基于改进Cascade RCNN的输电线路防振锤脱落检测方法[J]. 图学学报, 2023, 44(5): 849-860. DOI
	YAN G W, LIU R Z, JIAO R H, et al. Detection method of dropped anti-vibration hammer for transmission line based on improved cascade RCNN[J]. Journal of Graphics, 2023, 44(5): 849-860 (in Chinese).
[16]	曾伦杰, 储珺, 陈昭俊. 二阶段锚框和类均衡损失的遥感图像目标检测[J]. 图学学报, 2023, 44(2): 249-259. DOI
	ZENG L J, CHU J, CHEN Z J. Object detection in remote sensing image based on two-stage anchor and class balanced loss[J]. Journal of Graphics, 2023, 44(2): 249-259 (in Chinese).
[17]	REN S Q, HE K M, GIRSHICK R, et al. Faster R-CNN: towards real-time object detection with region proposal networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(6): 1137-1149. DOI PMID
[18]	胡欣, 胡帅, 马丽军, 等. 基于融合MBAM与YOLOv5的PCB缺陷检测方法[J]. 图学学报, 2024, 45(1): 47-55. DOI
	HU X, HU S, MA L J, et al. PCB defect detection method based on fusion of MBAM and YOLOv5[J]. Journal of Graphics, 2024, 45(1): 47-55 (in Chinese). DOI
[19]	LIU W, ANGUELOV D, ERHAN D, et al. SSD: single shot MultiBox detector[C]// The 14th European Conference on Computer Vision. Cham: Springer, 2016: 21-37.
[20]	CARION N, MASSA F, SYNNAEVE G, et al. End-to-end object detection with transformers[C]// The 16th European Conference on Computer Vision. Cham: Springer, 2020: 213-229.
[21]	ZHAO Y A, LV W Y, XU S L, et al. DETRs beat YOLOs on real-time object detection[EB/OL]. [2024-01-10]. https://arxiv.org/abs/2304.08069.
[22]	ZHANG H, LI F, LIU S L, et al. DINO: DETR with improved DeNoising anchor boxes for end-to-end object detection[EB/OL]. [2024-01-10]. https://arxiv.org/abs/2203.03605.
[23]	周中, 张俊杰, 鲁四平. 基于改进YOLOv4的隧道衬砌裂缝检测算法[J]. 铁道学报, 2023, 45(10): 162-170.
	ZHOU Z, ZHANG J J, LU S P. Tunnel lining crack detection algorithm based on improved YOLOv4[J]. Journal of the China Railway Society, 2023, 45(10): 162-170 (in Chinese).
[24]	王建锋, 刘文豪, 潘清云. 基于图像处理与深度学习的隧道衬砌裂缝检测[J]. 北京交通大学学报, 2022, 46(5): 19-29. DOI
	WANG J F, LIU W H, PAN Q Y. Crack detection for tunnel lining based on image processing and deep learning[J]. Journal of Beijing Jiaotong University, 2022, 46(5): 19-29 (in Chinese). DOI
[25]	何兆益, 常宝霞, 吴逸飞, 等. 基于YOLO v5-IBX网络模型的公路隧道衬砌裂缝检测方法研究[J]. 沈阳建筑大学学报: 自然科学版, 2023, 39(5): 888-898.
	HE Z Y, CHANG B X, WU Y F, et al. Research on crack detection method of highway tunnel lining based on YOLO v5-IBX network model[J]. Journal of Shenyang Jianzhu University: Natural Science, 2023, 39(5): 888-898 (in Chinese).
[26]	陈莹莹, 刘新根, 黄永亮, 等. 基于神经网络与边缘修正的隧道衬砌裂缝识别[J]. 现代隧道技术, 2022, 59(6): 24-34.
	CHEN Y Y, LIU X G, HUANG Y L. Crack identification of tunnel lining based on neural network and edge correction[J]. Modern Tunnelling Technology, 2022, 59(6): 24-34 (in Chinese).
[27]	刘新根, 陈莹莹, 王雨萱, 等. 基于级联神经网络的隧道衬砌裂缝识别算法研究[J]. 铁道学报, 2021, 43(10): 127-135.
	LIU X G, CHEN Y Y, WANG Y X, et al. Research on tunnel lining crack identification algorithm based on cascade neural network[J]. Journal of the China Railway Society, 2021, 43(10): 127-135 (in Chinese).
[28]	YU Z W, SHEN Y G, SHEN C K. A real-time detection approach for bridge cracks based on YOLOv4-FPM[J]. Automation in Construction, 2021, 122: 103514.
[29]	ZHOU Z, ZHANG J J, GONG C J. Automatic detection method of tunnel lining multi-defects via an enhanced you only look once network[J]. Computer-Aided Civil and Infrastructure Engineering, 2022, 37(6): 762-780.
[30]	任松, 朱倩雯, 涂歆玥, 等. 基于深度学习的公路隧道衬砌病害识别方法[J]. 浙江大学学报: 工学版, 2022, 56(1): 92-99.
	REN S, ZHU Q W, TU X Y, et al. Lining disease identification of highway tunnel based on deep learning[J]. Journal of Zhejiang University: Engineering Science, 2022, 56(1): 92-99 (in Chinese).
[31]	方广欣. 基于YOLOv5的地铁隧道裂缝检测技术研究[D]. 合肥: 合肥工业大学, 2021.
	FANG G X. Research on crack detection technology of subway tunnel based on YOLOv5[D]. Hefei: Hefei University of Technology, 2021 (in Chinese).
[32]	武兵, 田莹. 基于注意力机制的多尺度道路损伤检测算法研究[J/OL]. 图学学报, (2024-07-02) [2024-08-12]. http://kns.cnki.net/kcms/detail/10.1034.T.20240702.1112.002.html.
	WU B, TIAN Y. Research on multi-scale road damage detection algorithm based on attention mechanism[J/OL]. Journal of Graphics, (2024-07-02) [2024-08-12]. http://kns.cnki.net/kcms/detail/10.1034.T.20240702.1112.002.html (in Chinese).
[33]	牛为华, 郭迅. 基于改进YOLOv8的船舰遥感图像旋转目标检测算法[J/OL]. 图学学报,(2024-07-01) [2024-08-12]. http://kns.cnki.net/kcms/detail/10.1034.t.20240701.1201.004.html.
	NIU W H, GUO X. Rotating target detection algorithm in ship remote sensing images based on YOLOv8[J/OL]. Journal of Graphics, (2024-07-01) [2024-08-12]. http://kns.cnki.net/kcms/detail/10.1034.t.20240701.1201.004.html (in Chinese).
[34]	ZHU X Z, HU H, LIN S, et al. Deformable ConvNets V2: more deformable, better results[C]// 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2019: 9300-9308.