Tunnel fire detection based on improved student-teacher network

doi:10.11996/JG.j.2095-302X.2023050978

Abstract

Abstract:

Fire incidents in tunnels present serious hazards due to their rapid spread in confined spaces, endangering lives and property while making rescue operations challenging. Current tunnel fire detection methods suffer from inaccuracies and sufficient data. To address the above problems, a tunnel fire detection method based on an improved student-teacher network was proposed. Firstly, the proposed method trained unsupervised learning on fire-free samples to detect fires, compensating for the lack of tunnel fire datasets. At the same time, the student network and the teacher network with the same structure were adopted to form the whole network structure, and an attention mechanism was added to residual blocks for knowledge distillation to reduce the loss of important information and filter irrelevant information. Secondly, a Mish activation function was employed to replace a Relu activation function to enhance network performance. Finally, the SPD-Conv module replaced the strided convolution and pooling layer to improve the detection accuracy in smaller fire areas. The experimental results demonstrated that the pixel-level AUC-ROC and image-level AUC-ROC of the improved student-teacher network in the self-made tunnel fire dataset reached 0.93 and 0.82, respectively. Compared with the current tunnel fire detection algorithms, the detection accuracy of the improved model was higher than other models, substantiating its effectiveness.

Key words: tunnel fire detection, student-teacher network, unsupervised learning, attention mechanism, Mish activation function, SPD-Conv

CLC Number:

TP391

SONG Huan-sheng, WEN Ya, SUN Shi-jie, SONG Xiang-yu, ZHANG Chao-yang, LI Xu. Tunnel fire detection based on improved student-teacher network[J]. Journal of Graphics, 2023, 44(5): 978-987.

Figures/Tables 17

Fig. 1 STPM network structure

Fig. 2 STPM detection results at different resolutions

Fig. 3 SE attention mechanism network structure

Fig. 4 SPD-Conv structure

Fig. 5 Knowledge distillation layer structure

Table 1 Experimental environment

分类	设备	信息
硬件	处理器	Intel®Core-i9 10900K CPU @3.70 GHz
	内存	64 G
	GPU	NVIDIA GeForce RTX 2060SUPER
	显存	8 G
软件	系统	Ubuntu 20.04
	Cuda版本	CUDA 10.0
	平台	Pytorch 1.8.1

Fig. 6 Example diagrams of training dataset

Fig. 7 Example diagrams of test dataset ((a)~(d) Tunnel fire scenarios; (e) Highway fire scenarios; (f) Self-made non-vehicle fire scenarios; (g) Highway vehicle target fire-free scenarios; (h) Vehicle target fire-free scenarios in the CoCo dataset)

Fig. 8 ROC curves ((a) Image-level ROC curve; (b) Pixel-level ROC curve)

Table 2 Attention mechanism verification experiment

算法模型	AR_I	AR_P	S	FPS
STPM	0.65	0.83	104.8	100
STPM+CA	0.72	0.84	110.4	95.6
STPM+ECA	0.75	0.86	108.0	88.4
STPM+ CBAM	0.78	0.87	115.7	80.8
STPM+SE	0.76	0.87	105.3	96.5

Table 3 Activate function verification experiment

方法	AR_I	AR_P	S	FPS
Relu	0.76	0.87	105.3	96.5
Mish	0.78	0.90	107.8	90.2

Table 4 SPD-Conv module verification experiment

方法	AR_I	AR_P	S	FPS
STPM-SM	0.78	0.90	107.8	90.2
STPM-SMSC	0.82	0.93	117.9	80.2

Table 5 AR at different image resolutions

指标	算法模型	64×64	32×32	16×16	结果
AR_P	STPM	0.75	0.79	0.82	0.83
AR_P	STPM-SMSC	0.86	0.87	0.92	0.93
AR_I	STPM	0.52	0.59	0.63	0.65
AR_I	STPM-SMSC	0.73	0.76	0.80	0.82

Fig. 9 Comparison with STPM network results ((a) Scene a; (b) Scene b; (c) Scene c)

Fig. 10 STPM-SMSC test results ((a) Highway fire scenarios; (b) Self-made non-vehicle fire scenarios; (c)~(g) Tunnel fire scenarios)

Table 6 Comparison results with existing methods

算法模型	P	R	AR	S	FPS
YOLO v5	0.87	0.90	0.86	99.7	83.7
YOLO v4-Tiny	0.74	0.78	0.75	76.6	97.2
SSD	0.80	0.84	0.81	70.2	104.0
Faster R-CNN	0.86	0.83	0.85	127.5	70.6
Swin Transformer	0.88	0.89	0.89	105.6	75.8
STPM-SMSC	0.93	0.94	0.93	117.9	80.2

Fig. 11 Test result comparison ((a) Original image; (b) Ground Truth; (c) YOLO v5; (d) YOLO v4-Tiny; (e) SSD; (f) Faster R-CNN; (g) Swin Transformer; (h) STPM-SMSC)

References 33

[1]	郝瑀然. 2021年交通运输行业发展统计公报[EB/OL]. (2022-05-25) [2022-12-06]. https://xxgk.mot.gov.cn/2020/jigou/zhghs/202205/t20220524_3656659.html.
	HAO Y R. Statistical bulletin on the development of the transport industry in 2021[EB/OL]. (2022-05-25) [2022-12-06]. https://xxgk.mot.gov.cn/2020/jigou/zhghs/202205/t20220524_3656659.html. (in Chinese)
[2]	祝玉华, 司艺艺, 李智慧. 基于深度学习的烟雾与火灾检测算法综述[J]. 计算机工程与应用, 2022, 58(23): 1-11. DOI
	ZHU Y H, SI Y Y, LI Z H. Overview of smoke and fire detection algorithms based on deep learning[J]. Computer Engineering and Applications, 2022, 58(23): 1-11. (in Chinese) DOI
[3]	陈嘉卿, 张冰, 宋英磊. 基于RGB统计颜色模型的火焰识别[J]. 江苏科技大学学报: 自然科学版, 2017, 31(2): 178-184.
	CHEN J Q, ZHANG B, SONG Y L. Flame recognition based on statistical RGB color model[J]. Journal of Jiangsu University of Science and Technology: Natural Science Edition, 2017, 31(2): 178-184. (in Chinese)
[4]	SUN K L, ZHAO Q C, WANG X W. Using knowledge inference to suppress the lamp disturbance for fire detection[J]. Journal of Safety Science and Resilience, 2021, 2(3): 124-130. DOI URL
[5]	XIE Y K, ZHU J, CAO Y G, et al. Efficient video fire detection exploiting motion-flicker-based dynamic features and deep static features[J]. IEEE Access, 2020, 8: 81904-81917. DOI URL
[6]	WANG Y B, DANG L F, REN J Y. Forest fire image recognition based on convolutional neural network[J]. Journal of Algorithms & Computational Technology, 2019, 13: 174830261988768.
[7]	王文朋, 毛文涛, 何建樑, 等. 基于深度迁移学习的烟雾识别方法[J]. 计算机应用, 2017, 37(11): 3176-3181, 3193. DOI
	WANG W P, MAO W T, HE J L, et al. Smoke recognition based on deep transfer learning[J]. Journal of Computer Applications, 2017, 37(11): 3176-3181, 3193. (in Chinese) DOI
[8]	张倩, 周平平, 王公堂, 等. 基于合成图像的Faster R-CNN森林火灾烟雾检测[J]. 山东师范大学学报: 自然科学版, 2019, 34(2): 180-185.
	ZHANG Q, ZHOU P P, WANG G T, et al. Faster r-cnn forest fire smoke detection based on synthetic images[J]. Journal of Shandong Normal University: Natural Science, 2019, 34(2): 180-185. (in Chinese)
[9]	BERGMANN P, FAUSER M, SATTLEGGER D, et al. Uninformed students: student-teacher anomaly detection with discriminative latent embeddings[C]// 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2020: 4182-4191.
[10]	XIE E Z, DING J, WANG W H, et al. DetCo: unsupervised contrastive learning for object detection[C]// 2021 IEEE/CVF International Conference on Computer Vision. New York: IEEE Press, 2022: 8372-8381.
[11]	WANG L, YOON K J. Knowledge distillation and student-teacher learning for visual intelligence: a review and new outlooks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022, 44(6): 3048-3068. DOI URL
[12]	SHAW P, USZKOREIT J, VASWANI A. Self-attention with relative position representations[EB/OL]. [2022-12-06]. https://arxiv.org/abs/1803.02155.
[13]	MISRA D. Mish: a self regularized non-monotonic neural activation function[EB/OL]. (2019-08-23) [2022-12-06]. https://arxiv.org/abs/1908.08681.
[14]	SUNKARA R, LUO T. No more strided convolutions or pooling: a new CNN building block for low-resolution images and small objects[EB/OL]. [2022-12-06]. https://arxiv.org/abs/2208.03641.
[15]	WANG G D, HAN S M, DING E R, et al. Student-teacher feature pyramid matching for unsupervised anomaly detection[EB/OL]. [2022-12-06]. https://arxiv.org/abs/2103.04257v2.
[16]	COHEN N, HOSHEN Y. Sub-image anomaly detection with deep pyramid correspondences[EB/OL]. [2022-12-06]. https://arxiv.org/abs/2005.02357.
[17]	LIN T Y, DOLLÁR P, GIRSHICK R, et al. Feature pyramid networks for object detection[C]// 2017 IEEE Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2017: 936-944.
[18]	HE K M, GIRSHICK R, DOLLAR P. Rethinking ImageNet pre-training[C]// 2019 IEEE/CVF International Conference on Computer Vision. New York: IEEE Press, 2020: 4917-4926.
[19]	HE K M, ZHANG X Y, REN S Q, et al. Deep residual learning for image recognition[C]// 2016 IEEE Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2016: 770-778.
[20]	YAMADA S, HOTTA K. Reconstruction student with attention for student-teacher pyramid matching[EB/OL]. [2022-12-06]. https://arxiv.org/abs/2111.15376v2.
[21]	HU J, SHEN L, SUN G. Squeeze-and-excitation networks[C]// 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2018: 7132-7141.
[22]	田龙, 朱朋朋. 高速公路隧道突发事件应急管理与应急预案[J]. 黑龙江交通科技, 2022, 45(4): 145-148.
	TIAN L, ZHU P P. Contingency management and emergency plan for highway tunnel emergencies[J]. Communications Science and Technology Heilongjiang, 2022, 45(4): 145-148. (in Chinese)
[23]	LIN T Y, MAIRE M, BELONGIE S, et al. Microsoft COCO: common objects in context[C]// European Conference on Computer Vision. Cham: Springer, 2014: 740-755.
[24]	RUSSELL B C, TORRALBA A, MURPHY K P, et al. LabelMe: a database and web-based tool for image annotation[J]. International Journal of Computer Vision, 2008, 77(1-3): 157-173. DOI URL
[25]	NARKHEDE S. Understanding auc-roc curve[J]. Towards Data Science, 2018, 26(1): 220-227.
[26]	HOU Q B, ZHOU D Q, FENG J S. Coordinate attention for efficient mobile network design[C]// 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2021: 13708-13717.
[27]	WOO S, PARK J, LEE J Y, et al. CBAM: convolutional block attention module[C]// European Conference on Computer Vision. Cham: Springer, 2018: 3-19.
[28]	SHEN Z R, ZHANG M Y, ZHAO H Y, et al. Efficient attention: attention with linear complexities[C]// 2021 IEEE Winter Conference on Applications of Computer Vision. New York: IEEE Press, 2021: 3530-3538.
[29]	YAN B, FAN P, LEI X Y, et al. A real-time apple targets detection method for picking robot based on improved YOLOv5[J]. Remote Sensing, 2021, 13(9): 1619. DOI URL
[30]	缪伟志, 陆兆纳, 王俊龙, 等. 基于视觉的火灾检测研究[J]. 森林工程, 2022, 38(1): 86-92, 100.
	MIAO W Z, LU Z N, WANG J L, et al. Fire detection research based on vision[J]. Forest Engineering, 2022, 38(1): 86-92, 100. (in Chinese)
[31]	LIU W, ANGUELOV D, ERHAN D, et al. SSD: single shot MultiBox detector[C]// Computer Vision - ECCV 2016. Cham: Springer International Publishing, 2016: 21-37.
[32]	REN S Q, HE K M, GIRSHICK R, et al. Faster R-CNN: towards real-time object detection with region proposal networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(6): 1137-1149. DOI PMID
[33]	LIU Z, LIN Y T, CAO Y, et al. Swin transformer: hierarchical vision transformer using shifted windows[C]// 2021 IEEE/CVF International Conference on Computer Vision. New York: IEEE Press, 2022: 9992-10002.