基于改进教师学生网络的隧道火灾检测

doi:10.11996/JG.j.2095-302X.2023050978

图学学报 ›› 2023, Vol. 44 ›› Issue (5): 978-987.DOI: 10.11996/JG.j.2095-302X.2023050978

• 图像处理与计算机视觉 • 上一篇下一篇

基于改进教师学生网络的隧道火灾检测

宋焕生¹(), 文雅¹, 孙士杰¹(), 宋翔宇², 张朝阳¹, 李旭¹

1.长安大学信息工程学院，陕西西安 710018
2.斯威本科技大学软件与电气工程学院，墨尔本 3000

收稿日期:2023-02-08 接受日期:2023-05-23 出版日期:2023-10-31 发布日期:2023-10-31
通讯作者: 孙士杰(1989-)，男，副教授，博士。主要研究方向为多目标检测跟踪、位姿估计。E-mail：shijieSun@chd.edu.cn
作者简介:宋焕生(1964-)，男，教授，博士。主要研究方向为图像/交通视频处理、智能交通。E-mail：hshsong@chd.edu.cn
基金资助:
国家自然科学基金项目(62006026);国家自然科学基金项目(62072053);国家自然科学基金项目(U21B2041)

Tunnel fire detection based on improved student-teacher network

SONG Huan-sheng¹(), WEN Ya¹, SUN Shi-jie¹(), SONG Xiang-yu², ZHANG Chao-yang¹, LI Xu¹

1. School of Information Engineering, Chang’an University, Xi’an Shaanxi 710018, China
2. School of Software and Electrical Engineering, Swinburne University of Technology, Melbourne 3000, Australia

Received:2023-02-08 Accepted:2023-05-23 Online:2023-10-31 Published:2023-10-31
Contact: SUN Shi-jie (1989-), associate professor, Ph.D. His main research interests cover multi object detection and tracking, pose estimation. E-mail：shijieSun@chd.edu.cn
About author:SONG Huan-sheng (1964-), professor, Ph.D. His main research interests cover image/traffic video processing, intelligent trans-portation. E-mail：hshsong@chd.edu.cn
Supported by:
National Natural Science Foundation of China(62006026);National Natural Science Foundation of China(62072053);National Natural Science Foundation of China(U21B2041)

摘要/Abstract

摘要：

隧道空间狭小，封闭性高，当发生火灾时，火势会迅速蔓延，导致救援难度增大，严重危害人们的生命财产安全。现有隧道火灾检测方法精度低且数据集匮乏，针对上述问题，提出一种基于改进教师学生网络的隧道火灾检测方法。首先，通过无监督学习对没有火灾的样本进行训练从而检测火灾，可以弥补隧道火灾数据集匮乏的问题，同时采用相同结构的学生网络和教师网络组成整体网络结构，在用于知识蒸馏的残差块中加入注意力机制以减少重要信息损失，过滤无关信息，其次用Mish激活函数代替Relu激活函数以提高网络性能，最后引入SPD-Conv模块代替跨步卷积层和池化层以提高较小火灾区域的检测精度。实验结果表明：改进的教师学生网络在自制隧道火灾数据集的像素级AUC-ROC和图像级AUC-ROC分别达到0.93和0.82，与现有隧道火灾检测算法相比，该模型检测精度均高于其他模型，验证了该模型的有效性。

关键词: 隧道火灾检测, 教师学生网络, 无监督学习, 注意力机制, Mish激活函数, SPD-Conv

Abstract:

Fire incidents in tunnels present serious hazards due to their rapid spread in confined spaces, endangering lives and property while making rescue operations challenging. Current tunnel fire detection methods suffer from inaccuracies and sufficient data. To address the above problems, a tunnel fire detection method based on an improved student-teacher network was proposed. Firstly, the proposed method trained unsupervised learning on fire-free samples to detect fires, compensating for the lack of tunnel fire datasets. At the same time, the student network and the teacher network with the same structure were adopted to form the whole network structure, and an attention mechanism was added to residual blocks for knowledge distillation to reduce the loss of important information and filter irrelevant information. Secondly, a Mish activation function was employed to replace a Relu activation function to enhance network performance. Finally, the SPD-Conv module replaced the strided convolution and pooling layer to improve the detection accuracy in smaller fire areas. The experimental results demonstrated that the pixel-level AUC-ROC and image-level AUC-ROC of the improved student-teacher network in the self-made tunnel fire dataset reached 0.93 and 0.82, respectively. Compared with the current tunnel fire detection algorithms, the detection accuracy of the improved model was higher than other models, substantiating its effectiveness.

Key words: tunnel fire detection, student-teacher network, unsupervised learning, attention mechanism, Mish activation function, SPD-Conv

中图分类号:

TP391

宋焕生, 文雅, 孙士杰, 宋翔宇, 张朝阳, 李旭. 基于改进教师学生网络的隧道火灾检测[J]. 图学学报, 2023, 44(5): 978-987.

SONG Huan-sheng, WEN Ya, SUN Shi-jie, SONG Xiang-yu, ZHANG Chao-yang, LI Xu. Tunnel fire detection based on improved student-teacher network[J]. Journal of Graphics, 2023, 44(5): 978-987.

图/表 17

图1 STPM网络结构

Fig. 1 STPM network structure

图2 STPM在不同分辨率检测结果

Fig. 2 STPM detection results at different resolutions

图3 SE注意力机制网络结构

Fig. 3 SE attention mechanism network structure

图4 SPD-Conv结构

Fig. 4 SPD-Conv structure

图5 知识蒸馏层结构

Fig. 5 Knowledge distillation layer structure

表1 实验环境

Table 1 Experimental environment

分类	设备	信息
硬件	处理器	Intel®Core-i9 10900K CPU @3.70 GHz
	内存	64 G
	GPU	NVIDIA GeForce RTX 2060SUPER
	显存	8 G
软件	系统	Ubuntu 20.04
	Cuda版本	CUDA 10.0
	平台	Pytorch 1.8.1

图6 训练集示例图

Fig. 6 Example diagrams of training dataset

图7 测试集示例图((a)~(d)隧道火灾场景；(e)高速公路火灾场景；(f)自制非车辆火灾场景；(g)高速公路车辆目标非火灾场景；(h) CoCo数据集中车辆目标非火灾场景)

Fig. 7 Example diagrams of test dataset ((a)~(d) Tunnel fire scenarios; (e) Highway fire scenarios; (f) Self-made non-vehicle fire scenarios; (g) Highway vehicle target fire-free scenarios; (h) Vehicle target fire-free scenarios in the CoCo dataset)

图8 ROC曲线((a)图像级ROC曲线图；(b)像素级ROC曲线图)

Fig. 8 ROC curves ((a) Image-level ROC curve; (b) Pixel-level ROC curve)

表2 注意力机制验证实验

Table 2 Attention mechanism verification experiment

算法模型	AR_I	AR_P	S	FPS
STPM	0.65	0.83	104.8	100
STPM+CA	0.72	0.84	110.4	95.6
STPM+ECA	0.75	0.86	108.0	88.4
STPM+ CBAM	0.78	0.87	115.7	80.8
STPM+SE	0.76	0.87	105.3	96.5

表3 激活函数验证实验

Table 3 Activate function verification experiment

方法	AR_I	AR_P	S	FPS
Relu	0.76	0.87	105.3	96.5
Mish	0.78	0.90	107.8	90.2

表4 SPD-Conv模块验证实验

Table 4 SPD-Conv module verification experiment

方法	AR_I	AR_P	S	FPS
STPM-SM	0.78	0.90	107.8	90.2
STPM-SMSC	0.82	0.93	117.9	80.2

表5 图像不同分辨率的AR

Table 5 AR at different image resolutions

指标	算法模型	64×64	32×32	16×16	结果
AR_P	STPM	0.75	0.79	0.82	0.83
AR_P	STPM-SMSC	0.86	0.87	0.92	0.93
AR_I	STPM	0.52	0.59	0.63	0.65
AR_I	STPM-SMSC	0.73	0.76	0.80	0.82

图9 与原始STPM网络结果对比图((a)场景a；(b)场景b；(c)场景c)

Fig. 9 Comparison with STPM network results ((a) Scene a; (b) Scene b; (c) Scene c)

图10 STPM-SMSC检测结果((a)高速公路火灾场景；(b)自制非车辆火灾场景；(c)~(g)隧道火灾场景)

Fig. 10 STPM-SMSC test results ((a) Highway fire scenarios; (b) Self-made non-vehicle fire scenarios; (c)~(g) Tunnel fire scenarios)

表6 与现有方法对比结果

Table 6 Comparison results with existing methods

算法模型	P	R	AR	S	FPS
YOLO v5	0.87	0.90	0.86	99.7	83.7
YOLO v4-Tiny	0.74	0.78	0.75	76.6	97.2
SSD	0.80	0.84	0.81	70.2	104.0
Faster R-CNN	0.86	0.83	0.85	127.5	70.6
Swin Transformer	0.88	0.89	0.89	105.6	75.8
STPM-SMSC	0.93	0.94	0.93	117.9	80.2

图11 部分检测结果对比((a)原图；(b)真值；(c) YOLO v5；(d) YOLO v4-Tiny；(e) SSD；(f) Faster R-CNN；(g) Swin Transformer；(h) STPM-SMSC)

Fig. 11 Test result comparison ((a) Original image; (b) Ground Truth; (c) YOLO v5; (d) YOLO v4-Tiny; (e) SSD; (f) Faster R-CNN; (g) Swin Transformer; (h) STPM-SMSC)

参考文献 33

[1]	郝瑀然. 2021年交通运输行业发展统计公报[EB/OL]. (2022-05-25) [2022-12-06]. https://xxgk.mot.gov.cn/2020/jigou/zhghs/202205/t20220524_3656659.html.
	HAO Y R. Statistical bulletin on the development of the transport industry in 2021[EB/OL]. (2022-05-25) [2022-12-06]. https://xxgk.mot.gov.cn/2020/jigou/zhghs/202205/t20220524_3656659.html. (in Chinese)
[2]	祝玉华, 司艺艺, 李智慧. 基于深度学习的烟雾与火灾检测算法综述[J]. 计算机工程与应用, 2022, 58(23): 1-11. DOI
	ZHU Y H, SI Y Y, LI Z H. Overview of smoke and fire detection algorithms based on deep learning[J]. Computer Engineering and Applications, 2022, 58(23): 1-11. (in Chinese) DOI
[3]	陈嘉卿, 张冰, 宋英磊. 基于RGB统计颜色模型的火焰识别[J]. 江苏科技大学学报: 自然科学版, 2017, 31(2): 178-184.
	CHEN J Q, ZHANG B, SONG Y L. Flame recognition based on statistical RGB color model[J]. Journal of Jiangsu University of Science and Technology: Natural Science Edition, 2017, 31(2): 178-184. (in Chinese)
[4]	SUN K L, ZHAO Q C, WANG X W. Using knowledge inference to suppress the lamp disturbance for fire detection[J]. Journal of Safety Science and Resilience, 2021, 2(3): 124-130. DOI URL
[5]	XIE Y K, ZHU J, CAO Y G, et al. Efficient video fire detection exploiting motion-flicker-based dynamic features and deep static features[J]. IEEE Access, 2020, 8: 81904-81917. DOI URL
[6]	WANG Y B, DANG L F, REN J Y. Forest fire image recognition based on convolutional neural network[J]. Journal of Algorithms & Computational Technology, 2019, 13: 174830261988768.
[7]	王文朋, 毛文涛, 何建樑, 等. 基于深度迁移学习的烟雾识别方法[J]. 计算机应用, 2017, 37(11): 3176-3181, 3193. DOI
	WANG W P, MAO W T, HE J L, et al. Smoke recognition based on deep transfer learning[J]. Journal of Computer Applications, 2017, 37(11): 3176-3181, 3193. (in Chinese) DOI
[8]	张倩, 周平平, 王公堂, 等. 基于合成图像的Faster R-CNN森林火灾烟雾检测[J]. 山东师范大学学报: 自然科学版, 2019, 34(2): 180-185.
	ZHANG Q, ZHOU P P, WANG G T, et al. Faster r-cnn forest fire smoke detection based on synthetic images[J]. Journal of Shandong Normal University: Natural Science, 2019, 34(2): 180-185. (in Chinese)
[9]	BERGMANN P, FAUSER M, SATTLEGGER D, et al. Uninformed students: student-teacher anomaly detection with discriminative latent embeddings[C]// 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2020: 4182-4191.
[10]	XIE E Z, DING J, WANG W H, et al. DetCo: unsupervised contrastive learning for object detection[C]// 2021 IEEE/CVF International Conference on Computer Vision. New York: IEEE Press, 2022: 8372-8381.
[11]	WANG L, YOON K J. Knowledge distillation and student-teacher learning for visual intelligence: a review and new outlooks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022, 44(6): 3048-3068. DOI URL
[12]	SHAW P, USZKOREIT J, VASWANI A. Self-attention with relative position representations[EB/OL]. [2022-12-06]. https://arxiv.org/abs/1803.02155.
[13]	MISRA D. Mish: a self regularized non-monotonic neural activation function[EB/OL]. (2019-08-23) [2022-12-06]. https://arxiv.org/abs/1908.08681.
[14]	SUNKARA R, LUO T. No more strided convolutions or pooling: a new CNN building block for low-resolution images and small objects[EB/OL]. [2022-12-06]. https://arxiv.org/abs/2208.03641.
[15]	WANG G D, HAN S M, DING E R, et al. Student-teacher feature pyramid matching for unsupervised anomaly detection[EB/OL]. [2022-12-06]. https://arxiv.org/abs/2103.04257v2.
[16]	COHEN N, HOSHEN Y. Sub-image anomaly detection with deep pyramid correspondences[EB/OL]. [2022-12-06]. https://arxiv.org/abs/2005.02357.
[17]	LIN T Y, DOLLÁR P, GIRSHICK R, et al. Feature pyramid networks for object detection[C]// 2017 IEEE Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2017: 936-944.
[18]	HE K M, GIRSHICK R, DOLLAR P. Rethinking ImageNet pre-training[C]// 2019 IEEE/CVF International Conference on Computer Vision. New York: IEEE Press, 2020: 4917-4926.
[19]	HE K M, ZHANG X Y, REN S Q, et al. Deep residual learning for image recognition[C]// 2016 IEEE Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2016: 770-778.
[20]	YAMADA S, HOTTA K. Reconstruction student with attention for student-teacher pyramid matching[EB/OL]. [2022-12-06]. https://arxiv.org/abs/2111.15376v2.
[21]	HU J, SHEN L, SUN G. Squeeze-and-excitation networks[C]// 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2018: 7132-7141.
[22]	田龙, 朱朋朋. 高速公路隧道突发事件应急管理与应急预案[J]. 黑龙江交通科技, 2022, 45(4): 145-148.
	TIAN L, ZHU P P. Contingency management and emergency plan for highway tunnel emergencies[J]. Communications Science and Technology Heilongjiang, 2022, 45(4): 145-148. (in Chinese)
[23]	LIN T Y, MAIRE M, BELONGIE S, et al. Microsoft COCO: common objects in context[C]// European Conference on Computer Vision. Cham: Springer, 2014: 740-755.
[24]	RUSSELL B C, TORRALBA A, MURPHY K P, et al. LabelMe: a database and web-based tool for image annotation[J]. International Journal of Computer Vision, 2008, 77(1-3): 157-173. DOI URL
[25]	NARKHEDE S. Understanding auc-roc curve[J]. Towards Data Science, 2018, 26(1): 220-227.
[26]	HOU Q B, ZHOU D Q, FENG J S. Coordinate attention for efficient mobile network design[C]// 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2021: 13708-13717.
[27]	WOO S, PARK J, LEE J Y, et al. CBAM: convolutional block attention module[C]// European Conference on Computer Vision. Cham: Springer, 2018: 3-19.
[28]	SHEN Z R, ZHANG M Y, ZHAO H Y, et al. Efficient attention: attention with linear complexities[C]// 2021 IEEE Winter Conference on Applications of Computer Vision. New York: IEEE Press, 2021: 3530-3538.
[29]	YAN B, FAN P, LEI X Y, et al. A real-time apple targets detection method for picking robot based on improved YOLOv5[J]. Remote Sensing, 2021, 13(9): 1619. DOI URL
[30]	缪伟志, 陆兆纳, 王俊龙, 等. 基于视觉的火灾检测研究[J]. 森林工程, 2022, 38(1): 86-92, 100.
	MIAO W Z, LU Z N, WANG J L, et al. Fire detection research based on vision[J]. Forest Engineering, 2022, 38(1): 86-92, 100. (in Chinese)
[31]	LIU W, ANGUELOV D, ERHAN D, et al. SSD: single shot MultiBox detector[C]// Computer Vision - ECCV 2016. Cham: Springer International Publishing, 2016: 21-37.
[32]	REN S Q, HE K M, GIRSHICK R, et al. Faster R-CNN: towards real-time object detection with region proposal networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(6): 1137-1149. DOI PMID
[33]	LIU Z, LIN Y T, CAO Y, et al. Swin transformer: hierarchical vision transformer using shifted windows[C]// 2021 IEEE/CVF International Conference on Computer Vision. New York: IEEE Press, 2022: 9992-10002.

基于改进教师学生网络的隧道火灾检测

Tunnel fire detection based on improved student-teacher network

RichHTML

PDF (PC)

可视化

摘要/Abstract

引用本文

使用本文

图/表 17

参考文献 33

相关文章 15

编辑推荐

Metrics

本文评价

[1]	魏陈浩 , 杨睿 , 刘振丙 , 蓝如师 , 孙希延 , 罗笑南 . 具有双层路由注意力的 YOLOv8 道路场景目标检测方法 [J]. 图学学报, 2023, 44(6): 1104-1111.
[2]	丁建川, 肖金桐, 赵可新, 贾冬青, 崔炳德, 杨鑫 . 基于脉冲神经网络的复杂场景导航避障算法 [J]. 图学学报, 2023, 44(6): 1121-1129.
[3]	张丽媛, 赵海蓉, 何巍, 唐雄风 . 融合全局-局部注意模块的 Mask R-CNN膝关节囊肿检测方法 [J]. 图学学报, 2023, 44(6): 1183-1190.
[4]	张驰, 张效娟, 赵洋, 杨帆. 基于调色板的半交互式低照度唐卡图像增强 [J]. 图学学报, 2023, 44(6): 1202-1211.
[5]	杨陈成, 董秀成, 侯兵, 张党成, 向贤明, 冯琪茗. 基于参考的Transformer纹理迁移深度图像超分辨率重建[J]. 图学学报, 2023, 44(5): 861-867.
[6]	李利霞, 王鑫, 王军, 张又元. 基于特征融合与注意力机制的无人机图像小目标检测算法[J]. 图学学报, 2023, 44(4): 658-666.
[7]	李鑫, 普园媛, 赵征鹏, 徐丹, 钱文华. 内容语义和风格特征匹配一致的艺术风格迁移[J]. 图学学报, 2023, 44(4): 699-709.
[8]	余伟群, 刘佳涛, 张亚萍. 融合注意力的拉普拉斯金字塔单目深度估计[J]. 图学学报, 2023, 44(4): 728-738.
[9]	胡欣, 周运强, 肖剑, 杨杰. 基于改进YOLOv5的螺纹钢表面缺陷检测[J]. 图学学报, 2023, 44(3): 427-437.
[10]	郝鹏飞, 刘立群, 顾任远. YOLO-RD-Apple果园异源图像遮挡果实检测模型[J]. 图学学报, 2023, 44(3): 456-464.
[11]	罗文宇, 傅明月. 基于YoloX-ECA模型的非法野泳野钓现场监测技术[J]. 图学学报, 2023, 44(3): 465-472.
[12]	李雨, 闫甜甜, 周东生, 魏小鹏. 基于注意力机制与深度多尺度特征融合的自然场景文本检测[J]. 图学学报, 2023, 44(3): 473-481.
[13]	吴文欢, 张淏坤. 融合空间十字注意力与通道注意力的语义分割网络[J]. 图学学报, 2023, 44(3): 531-539.
[14]	谢国波, 贺笛轩, 何宇钦, 林志毅. 基于P-CenterNet的光学遥感图像烟囱检测[J]. 图学学报, 2023, 44(2): 233-240.
[15]	熊举举, 徐杨, 范润泽, 孙少聪. 基于轻量化视觉Transformer的花卉识别[J]. 图学学报, 2023, 44(2): 271-279.