基于DCNv2和Transformer Decoder的隧道衬砌裂缝高效检测模型研究

doi:10.11996/JG.j.2095-302X.2024051050

图学学报 ›› 2024, Vol. 45 ›› Issue (5): 1050-1061.DOI: 10.11996/JG.j.2095-302X.2024051050

基于DCNv2和Transformer Decoder的隧道衬砌裂缝高效检测模型研究

孙己龙¹(), 刘勇², 周黎伟², 路鑫³^,⁴, 侯小龙², 王亚琼², 王志丰²()

1.陕西省交通运输工程质量监测鉴定站，陕西西安 710075
2.长安大学公路学院，陕西西安 710064
3.长安大学材料科学与工程学院，陕西西安 710061
4.西安公路研究院有限公司，陕西西安 710065

收稿日期:2024-04-24 修回日期:2024-08-13 出版日期:2024-10-31 发布日期:2024-10-31
通讯作者:王志丰(1986-)，男，教授，博士。主要研究方向为隧道及岩土工程。E-mail：zhifeng.wang@chd.edu.cn
第一作者:孙己龙(1971-)，男，高级工程师，硕士。主要研究方向为公路工程质量和安全监督工作。E-mail：956036513@qq.com
基金资助:
国家重点研发计划项目(2021YFB2600404);陕西省交通运输厅交通科技项目(22-09K);陕西省创新能力支撑计划项目(2023-CX-TD-35);陕西省重点研发计划项目(2023KXJ-159)

Research on efficient detection model of tunnel lining crack based on DCNv2 and Transformer Decoder

SUN Jilong¹(), LIU Yong², ZHOU Liwei², LU Xin³^,⁴, HOU Xiaolong², WANG Yaqiong², WANG Zhifeng²()

1. Shaanxi Provincial Transportation Engineering Quality Monitoring and Appraisal Station, Xi’an Shaanxi 710075, China
2. School of Highway, Chang’an University, Xi’an Shaanxi 710064, China
3. School of Materials Science and Engineering, Chang’an University, Xi’an Shaanxi 710061, China
4. Xi’an highway research institute Co., Ltd., Xi’an Shaanxi 710065, China

Received:2024-04-24 Revised:2024-08-13 Published:2024-10-31 Online:2024-10-31
Contact: WANG Zhifeng (1986-), professor, Ph.D. His main research interest covers tunnel and geotechnical engineering. E-mail：zhifeng.wang@chd.edu.cn
First author：SUN Jilong (1971-), senior engineer, master. His main research interest covers e quality and safety supervision of highway. E-mail：956036513@qq.com
Supported by:
National Key Research and Development Program of China(2021YFB2600404);Shaanxi Transportation Technology Project(22-09K);The Innovation Capability Support Program of Shaanxi(2023-CX-TD-35);The Key Research and Development Program of Shaanxi(2023KXJ-159)

摘要/Abstract

摘要：

为解决因衬砌裂缝性状随机、分布密集、标注框分辨率低所导致的现有模型识别精度低、检测速度慢及参数量庞大等问题，以第2版可变形卷积网络(DCNv2)和端到端变换器解码器(Transformer Decoder)为基础对YOLOv8网络框架进行改进，提出了面向衬砌裂缝的检测模型DTD-YOLOv8。首先，通过引入DCNv2对YOLOv8主干卷积网络C2f进行融合以实现模型对裂缝形变特征的准确快速感知，同时采用Transformer Decoder对YOLOv8检测头进行替换以实现端到端框架内完整目标检测流程，从而消除因Anchor-free处理模式所带来的计算消耗。采用自建裂缝数据集对SSD，Faster-RCNN，RT-DETR，YOLOv3，YOLOv5，YOLOv8和DTD-YOLOv8的7种检测模型进行对比验证。结果表明：改进模型F1分数和mAP@50值分别为87.05%和89.58%；其中F1分数相较其他6种模型分别提高了14.16%，7.68%，1.55%，41.36%，8.20%和7.40%；mAP@50分别提高了28.84%，15.47%，1.33%，47.65%，10.14%和10.84%。改进模型参数量仅为RT-DETR的三分之一，检测单张图片的速度为16.01 ms，FPS为65.46帧每秒，对比其他模型检测速度得到提升。该模型在面向运营隧道裂缝检测任务需求时能够表现出高效的性能。

关键词: 隧道工程, 目标检测, 第2版可变形卷积网络, Transformer Decoder, 衬砌裂缝

Abstract:

To address the problems of low recognition accuracy, slow detection speed, and large parameter quantities caused by the random and dense distribution of cracks in tunnel linings and low resolution of annotation boxes in existing models, the YOLOv8 network framework was improved based on the Deformable Convolution Network version 2 (DCNv2) and end-to-end Transformer Decoder to propose a lining crack detection model DTD-YOLOv8. Firstly, DCNv2 was added to fuse the YOLOv8 backbone convolutional network C2f, enabling the model to accurately and quickly perceive crack deformation features. At the same time, the Transformer Decoder replaced the YOLOv8 detection head to achieve a complete object detection process within an end-to-end framework, thereby eliminating the computational consumption caused by the Anchor-free processing mode. A self-built crack dataset was used to compare and verify seven detection models, including SSD, Faster-RCNN, RT-DETR, YOLOv3, YOLOv5, YOLOv8, and DTD-YOLOv8. The results indicated that the F1 score and mAP@50 of DTD-YOLOv8 reached 87.05% and 89.58%, respectively. Compared to the other six models, the F1 score increased by 14.16%, 7.68%, 1.55%, 41.36%, 8.20%, and 7.40%, while the mAP@50 increased by 28.84%, 15.47%, 1.33%, 47.65%, 10.14%, and 10.84%. The parameter count of the new model was only one-third of RT-DETR, and the detection speed for a single image was 16.01 ms, with an FPS of 65.46 frames per second, demonstrating a speed improvement over other comparative model. The DTD-YOLOv8 model can demonstrate efficient performance in meeting the requirements of crack detection tasks in operational tunnels.

Key words: tunnel engineering, object detection, deformable convolution network v2, Transformer Decoder, lining crack

中图分类号:

孙己龙, 刘勇, 周黎伟, 路鑫, 侯小龙, 王亚琼, 王志丰. 基于DCNv2和Transformer Decoder的隧道衬砌裂缝高效检测模型研究[J]. 图学学报, 2024, 45(5): 1050-1061.

SUN Jilong, LIU Yong, ZHOU Liwei, LU Xin, HOU Xiaolong, WANG Yaqiong, WANG Zhifeng. Research on efficient detection model of tunnel lining crack based on DCNv2 and Transformer Decoder[J]. Journal of Graphics, 2024, 45(5): 1050-1061.

图/表 15

图1 可变形卷积采样((a)标准卷积；(b)可变形卷积)

Fig. 1 Deformable convolutional kernel sampling ((a) Standard convolution; (b) Deformable convolution)

图2 隧道衬砌裂缝图片卷积可视化示意图

Fig. 2 Convolutional visualization diagram of tunnel lining crack images

图3 RT-DETR模型结构图

Fig. 3 RT-DETR model structure diagram

图4 YOLOv8模型结构图

Fig. 4 YOLOv8 model structure diagram

图5 DTD-YOLOv8网络结构图

Fig. 5 DTD-YOLOv8 model structure diagram

图6 部分隧道衬砌裂缝图展示

Fig. 6 Partial tunnel lining crack diagram display

图7 图像扩充操作示意图

Fig. 7 Diagram of image expansion operation

表1 实验配置环境

Table 1 Experimental environment configuration

工具	环境
操作系统	Ubuntu
GPU	NVIDIA GeForce RTX 3080 Ti(12 GB) * 1GPU
编译器	PyCharm
框架	Pytorch2.0.0+CUDA11.8
语言	Python3.8

图8 主干特征提取网络示意图

Fig. 8 Backbone feature extraction network

表2 不同位置添加DCNv2实验结果/%

Table 2 Experimental results of adding DCN v2 at different position/%

模型	P	R	mAP@50	mAP@50:95	F1
YOLOv8	86.32	76.38	80.82	63.22	80.05
YOLOv-DCNv2-①	88.17	78.24	83.61	66.41	82.91
YOLOv8-DCNv2-②	88.18	77.63	83.52	66.82	82.57
YOLOv8-DCNv2-③	87.82	78.01	84.03	66.22	82.62
YOLOv8-DCNv2-④	89.08	79.05	83.87	66.98	83.76
YOLOv-DCNv2-①+②	87.21	77.21	80.52	64.31	81.91
YOLOv-DCNv2-①+③	87.34	76.92	82.92	66.14	81.80
YOLOv-DCNv2-①+④	88.02	78.54	83.01	66.73	83.01
YOLOv-DCNv2-②+③	87.91	78.39	82.16	65.98	82.88
YOLOv-DCNv2-②+④	87.83	79.01	83.27	66.23	83.19
YOLOv-DCNv2-①+②+③	86.15	76.31	80.26	65.87	80.93
YOLOv-DCNv2-①+③+④	85.82	77.26	81.38	66.08	81.32
YOLOv-DCNv2-②+③+④	86.96	77.53	82.23	66.33	81.97
YOLOv-DCNv2-①+②+③+④	86.52	76.94	81.07	65.97	81.45

图9 不同模型热力图可视化差异对比

Fig. 9 Comparison of differences in visualization of heat maps among different models ((a) Original images; (b) RT-DETR; (c) YOLOv8; (d) YOLOv8-DCNv2-④)

图10 DTD-YOLOv8模型损失函数结果对比

Fig. 10 Comparison of loss function results from DTD-YOLOv8 model

表3 消融试验结果比较

Table 3 Comparison of ablation test results

方案	mAP@50/%	mAP@50:95/%	F1/%	FPS/帧每秒	Par/M	GFLOPS/G
YOLOv8	80.82	63.22	80.05	45.57	3.1	8.9
YOLOv8-Decoder	84.67	69.35	81.52	58.21	9.5	16.7
YOLOv8-DCNv2-④	83.87	66.98	83.76	52.48	3.0	8.7
DTD-YOLOv8	89.58	75.32	87.05	65.46	6.1	11.6

图11 不同模型性能对比((a)准确度指标；(b)速度指标)

Fig. 11 Comparison of performance of different models ((a) Accuracy indicators; (b) Speed indicator)

图12 隧道衬砌裂缝检测可视化检测结果对比

Fig. 12 Comparison of visual inspection results for tunnel lining crack detection ((a) Original images; (b) SSD; (c) Faster RCNN; (d) RT-DETR; (e) YOLOv5n; (f) YOLOv8n; (g) DTD-YOLOv8)

参考文献 34

[1]	ZHANG L X, SHEN J K, ZHU B J. A review of the research and application of deep learning-based computer vision in structural damage detection[J]. Earthquake Engineering and Engineering Vibration, 2022, 21(1): 1-21.
[2]	刘新根, 陈莹莹, 朱爱玺, 等. 基于深度学习的隧道裂缝识别方法[J]. 广西大学学报: 自然科学版, 2018, 43(6): 2243-2251.
	LIU X G, CHEN Y Y, ZHU A X, et al. Tunnel crack identification based on deep learning[J]. Journal of Guangxi University: Natural Science Edition, 2018, 43(6): 2243-2251 (in Chinese).
[3]	刘勇, 王志丰, 王亚琼, 等. 基于案例推理的隧道洞门智能设计方法研究[J]. 铁道标准设计, 2024, 68(6): 137-144.
	LIU Y, WANG Z F, WANG Y Q, et al. Investigation on intelligent design method of tunnel portal based on case-based reasoning[J]. Railway Standard Design, 2024, 68(6): 137-144 (in Chinese).
[4]	刘学增, 李振, 杨芝璐, 等. 隧道衬砌裂缝特征对承载力的影响分析[J]. 哈尔滨工业大学学报, 2024, 56(7): 37-45.
	LIU X Z, LI Z, YANG Z L, et al. Analysis of the influence of tunnel lining cracks on bearing capacity[J]. Journal of Harbin Institute of Technology, 2024, 56(7): 37-45 (in Chinese).
[5]	ZHANG A, WANG K C P, LI B X, et al. Automated pixel-level pavement crack detection on 3D asphalt surfaces using a deep-learning network[J]. Computer-Aided Civil and Infrastructure Engineering, 2017, 32(10): 805-819.
[6]	FENG D S, WANG X, ZHANG B. Specific evaluation of tunnel lining multi-defects by all-refined GPR simulation method using hybrid algorithm of FETD and FDTD[J]. Construction and Building Materials, 2018, 185: 220-229.
[7]	YU T T, ZHU A X, CHAN Y Y. Efficient crack detection method for tunnel lining surface cracks based on infrared images[J]. Journal of Computing in Civil Engineering, 2017, 31(3): 04016067.
[8]	ZHOU S L, SONG W. Deep learning-based roadway crack classification using laser-scanned range images: a comparative study on hyperparameter selection[J]. Automation in Construction, 2020, 114: 103171.
[9]	杨泽洲, 陈思榕, 高翠芸, 等. 基于深度学习的代码生成方法研究进展[J]. 软件学报, 2024, 35(2): 604-628.
	YANG Z Z, CHEN S R, GAO C Y, et al. Deep learning based code generation methods: literature review[J]. Journal of Software, 2024, 35(2): 604-628 (in Chinese).
[10]	窦慧, 张凌茗, 韩峰, 等. 卷积神经网络的可解释性研究综述[J]. 软件学报, 2024, 35(1): 159-184.
	DOU H, ZHANG L M, HAN F, et al. Survey on convolutional neural network interpretability[J]. Journal of Software, 2024, 35(1): 159-184 (in Chinese).
[11]	REN Y P, HUANG J S, HONG Z Y, et al. Image-based concrete crack detection in tunnels using deep fully convolutional networks[J]. Construction and Building Materials, 2020, 234: 117367.
[12]	朱鑫. 隧道结构表面病害特征快速检测研究[D]. 成都: 西南交通大学, 2014.
	ZHU X. The research on rapid detection of characteristics of tunnel lining surface diseases[D]. Chengdu: Southwest Jiaotong University, 2014 (in Chinese).
[13]	曹义亲, 周一纬, 徐露. 基于E-YOLOX的实时金属表面缺陷检测算法[J]. 图学学报, 2023, 44(4): 677-690. DOI
	CAO Y Q, ZHOU Y W, XU L. A real-time metallic surface defect detection algorithm based on E-YOLOX[J]. Journal of Graphics, 2023, 44(4): 677-690 (in Chinese).
[14]	杨柳, 吴晓群. 基于深度学习的三维形状补全研究综述[J]. 图学学报, 2023, 44(2): 201-215. DOI
	YANG L, WU X Q. 3D shape completion via deep learning: a method survey[J]. Journal of Graphics, 2023, 44(2): 201-215 (in Chinese). DOI
[15]	阎光伟, 刘润泽, 焦润海, 等. 基于改进Cascade RCNN的输电线路防振锤脱落检测方法[J]. 图学学报, 2023, 44(5): 849-860. DOI
	YAN G W, LIU R Z, JIAO R H, et al. Detection method of dropped anti-vibration hammer for transmission line based on improved cascade RCNN[J]. Journal of Graphics, 2023, 44(5): 849-860 (in Chinese).
[16]	曾伦杰, 储珺, 陈昭俊. 二阶段锚框和类均衡损失的遥感图像目标检测[J]. 图学学报, 2023, 44(2): 249-259. DOI
	ZENG L J, CHU J, CHEN Z J. Object detection in remote sensing image based on two-stage anchor and class balanced loss[J]. Journal of Graphics, 2023, 44(2): 249-259 (in Chinese).
[17]	REN S Q, HE K M, GIRSHICK R, et al. Faster R-CNN: towards real-time object detection with region proposal networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(6): 1137-1149. DOI PMID
[18]	胡欣, 胡帅, 马丽军, 等. 基于融合MBAM与YOLOv5的PCB缺陷检测方法[J]. 图学学报, 2024, 45(1): 47-55. DOI
	HU X, HU S, MA L J, et al. PCB defect detection method based on fusion of MBAM and YOLOv5[J]. Journal of Graphics, 2024, 45(1): 47-55 (in Chinese). DOI
[19]	LIU W, ANGUELOV D, ERHAN D, et al. SSD: single shot MultiBox detector[C]// The 14th European Conference on Computer Vision. Cham: Springer, 2016: 21-37.
[20]	CARION N, MASSA F, SYNNAEVE G, et al. End-to-end object detection with transformers[C]// The 16th European Conference on Computer Vision. Cham: Springer, 2020: 213-229.
[21]	ZHAO Y A, LV W Y, XU S L, et al. DETRs beat YOLOs on real-time object detection[EB/OL]. [2024-01-10]. https://arxiv.org/abs/2304.08069.
[22]	ZHANG H, LI F, LIU S L, et al. DINO: DETR with improved DeNoising anchor boxes for end-to-end object detection[EB/OL]. [2024-01-10]. https://arxiv.org/abs/2203.03605.
[23]	周中, 张俊杰, 鲁四平. 基于改进YOLOv4的隧道衬砌裂缝检测算法[J]. 铁道学报, 2023, 45(10): 162-170.
	ZHOU Z, ZHANG J J, LU S P. Tunnel lining crack detection algorithm based on improved YOLOv4[J]. Journal of the China Railway Society, 2023, 45(10): 162-170 (in Chinese).
[24]	王建锋, 刘文豪, 潘清云. 基于图像处理与深度学习的隧道衬砌裂缝检测[J]. 北京交通大学学报, 2022, 46(5): 19-29. DOI
	WANG J F, LIU W H, PAN Q Y. Crack detection for tunnel lining based on image processing and deep learning[J]. Journal of Beijing Jiaotong University, 2022, 46(5): 19-29 (in Chinese). DOI
[25]	何兆益, 常宝霞, 吴逸飞, 等. 基于YOLO v5-IBX网络模型的公路隧道衬砌裂缝检测方法研究[J]. 沈阳建筑大学学报: 自然科学版, 2023, 39(5): 888-898.
	HE Z Y, CHANG B X, WU Y F, et al. Research on crack detection method of highway tunnel lining based on YOLO v5-IBX network model[J]. Journal of Shenyang Jianzhu University: Natural Science, 2023, 39(5): 888-898 (in Chinese).
[26]	陈莹莹, 刘新根, 黄永亮, 等. 基于神经网络与边缘修正的隧道衬砌裂缝识别[J]. 现代隧道技术, 2022, 59(6): 24-34.
	CHEN Y Y, LIU X G, HUANG Y L. Crack identification of tunnel lining based on neural network and edge correction[J]. Modern Tunnelling Technology, 2022, 59(6): 24-34 (in Chinese).
[27]	刘新根, 陈莹莹, 王雨萱, 等. 基于级联神经网络的隧道衬砌裂缝识别算法研究[J]. 铁道学报, 2021, 43(10): 127-135.
	LIU X G, CHEN Y Y, WANG Y X, et al. Research on tunnel lining crack identification algorithm based on cascade neural network[J]. Journal of the China Railway Society, 2021, 43(10): 127-135 (in Chinese).
[28]	YU Z W, SHEN Y G, SHEN C K. A real-time detection approach for bridge cracks based on YOLOv4-FPM[J]. Automation in Construction, 2021, 122: 103514.
[29]	ZHOU Z, ZHANG J J, GONG C J. Automatic detection method of tunnel lining multi-defects via an enhanced you only look once network[J]. Computer-Aided Civil and Infrastructure Engineering, 2022, 37(6): 762-780.
[30]	任松, 朱倩雯, 涂歆玥, 等. 基于深度学习的公路隧道衬砌病害识别方法[J]. 浙江大学学报: 工学版, 2022, 56(1): 92-99.
	REN S, ZHU Q W, TU X Y, et al. Lining disease identification of highway tunnel based on deep learning[J]. Journal of Zhejiang University: Engineering Science, 2022, 56(1): 92-99 (in Chinese).
[31]	方广欣. 基于YOLOv5的地铁隧道裂缝检测技术研究[D]. 合肥: 合肥工业大学, 2021.
	FANG G X. Research on crack detection technology of subway tunnel based on YOLOv5[D]. Hefei: Hefei University of Technology, 2021 (in Chinese).
[32]	武兵, 田莹. 基于注意力机制的多尺度道路损伤检测算法研究[J/OL]. 图学学报, (2024-07-02) [2024-08-12]. http://kns.cnki.net/kcms/detail/10.1034.T.20240702.1112.002.html.
	WU B, TIAN Y. Research on multi-scale road damage detection algorithm based on attention mechanism[J/OL]. Journal of Graphics, (2024-07-02) [2024-08-12]. http://kns.cnki.net/kcms/detail/10.1034.T.20240702.1112.002.html (in Chinese).
[33]	牛为华, 郭迅. 基于改进YOLOv8的船舰遥感图像旋转目标检测算法[J/OL]. 图学学报,(2024-07-01) [2024-08-12]. http://kns.cnki.net/kcms/detail/10.1034.t.20240701.1201.004.html.
	NIU W H, GUO X. Rotating target detection algorithm in ship remote sensing images based on YOLOv8[J/OL]. Journal of Graphics, (2024-07-01) [2024-08-12]. http://kns.cnki.net/kcms/detail/10.1034.t.20240701.1201.004.html (in Chinese).
[34]	ZHU X Z, HU H, LIN S, et al. Deformable ConvNets V2: more deformable, better results[C]// 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2019: 9300-9308.

基于DCNv2和Transformer Decoder的隧道衬砌裂缝高效检测模型研究

Research on efficient detection model of tunnel lining crack based on DCNv2 and Transformer Decoder

RichHTML

PDF (PC)

可视化

摘要/Abstract

引用本文

使用本文

图/表 15

参考文献 34

相关文章 15

编辑推荐

Metrics

本文评价

[1]	李琼 , 考月英 , 张莹 , 徐沛 . 面向无人机航拍图像的目标检测研究综述[J]. 图学学报, 2024, 45(6): 1145-1164.
[2]	李珍峰, 符世琛, 徐乐, 孟博, 张昕, 秦建军. 基于MBI-YOLOv8的煤矸石目标检测算法研究[J]. 图学学报, 2024, 45(6): 1301-1312.
[3]	闫建红, 冉同霄. 基于YOLOv8的轻量化无人机图像目标检测算法[J]. 图学学报, 2024, 45(6): 1328-1337.
[4]	姜晓恒, 段金忠, 卢洋, 崔丽莎, 徐明亮. 融合先验知识推理的表面缺陷检测[J]. 图学学报, 2024, 45(5): 957-967.
[5]	章东平, 魏杨悦, 何数技, 徐云超, 胡海苗, 黄文君. 特征融合与层间传递：一种基于Anchor DETR改进的目标检测方法[J]. 图学学报, 2024, 45(5): 968-978.
[6]	李建华, 韩宇, 石开铭, 张可嘉, 郭红领, 方东平, 曹佳明. 施工现场小目标工人检测方法[J]. 图学学报, 2024, 45(5): 1040-1049.
[7]	李大湘, 吉展, 刘颖, 唐垚. 改进YOLOv7遥感图像目标检测算法[J]. 图学学报, 2024, 45(4): 650-658.
[8]	胡欣, 常娅姝, 秦皓, 肖剑, 程鸿亮. 基于改进YOLOv8和GMM图像点集匹配的双目测距方法[J]. 图学学报, 2024, 45(4): 714-725.
[9]	牛为华, 郭迅. 基于改进YOLOv8的船舰遥感图像旋转目标检测算法[J]. 图学学报, 2024, 45(4): 726-735.
[10]	曾志超, 徐玥, 王景玉, 叶元龙, 黄志开, 王欢. 基于SOE-YOLO轻量化的水面目标检测算法[J]. 图学学报, 2024, 45(4): 736-744.
[11]	宫永超, 沈旭昆. 一种用于互惠目标检测与实例分割的深层架构[J]. 图学学报, 2024, 45(4): 745-759.
[12]	张相胜, 杨骁. 基于改进YOLOv7-tiny的橡胶密封圈缺陷检测方法[J]. 图学学报, 2024, 45(3): 446-453.
[13]	胡欣, 胡帅, 马丽军, 司利云, 肖剑, 袁晔. 基于融合MBAM与YOLOv5的PCB缺陷检测方法[J]. 图学学报, 2024, 45(1): 47-55.
[14]	魏陈浩, 杨睿, 刘振丙, 蓝如师, 孙希延, 罗笑南. 具有双层路由注意力的YOLOv8道路场景目标检测方法[J]. 图学学报, 2023, 44(6): 1104-1111.
[15]	王大阜, 王静, 石宇凯, 邓志文, 贾志勇. 基于深度迁移学习的图像隐私目标检测研究[J]. 图学学报, 2023, 44(6): 1112-1120.