Text detection method for electrical equipment nameplates based on deep learning

doi:10.11996/JG.j.2095-302X.2023040691

Abstract

Abstract:

The prompt detection of power equipment nameplates can help the complete transformer substations and power plants to efficiently comprehend device information and perform necessary maintenance, thus ensuring the proper functioning. This thesis addressed the problem of enhancing text detection efficiency while also taking into account the improvement of precision. To that end, we introduced the concept of convolutional block attention module (CBAM) into the DBNet, and improved the detection head. Multi-scale feature feature pyramid networks (FPN) structures were introduced into the backbone network, improving upon the original FPN. Meanwhile, in view of the absence of public data for power equipment nameplates and difficulties in obtaining it, we proposed a technique to enhance the data by cutting nameplate images into rectangles and then splicing them together into a new image. In this way, the data set could be effectively expanded. The experimental results showed that both the data enhancement method and the improved DBNet network structure proposed in this paper have played a role in improving the detection performance, surpassing most current text detection network structures on the market. The improved DBNet network structure combined with data enhancement method yielded a precision rate of 90.3% and a recalling rate of 79.7%. The rate of F-measure also increased to nearly 84.7%, a 3.3% improvement over the original model, indicating that the detection performance was greatly improved while the loss of detecting speed changes remained minimal.

Key words: text detection, DBNet, CBAM, data enhancement, electrical equipment nameplates

CLC Number:

TP391.1

WANG Dao-lei, KANG Bo, ZHU Rui. Text detection method for electrical equipment nameplates based on deep learning[J]. Journal of Graphics, 2023, 44(4): 691-698.

Figures/Tables 13

References 18

[1]	ZHOU X Y, YAO C, WEN H, et al. EAST: an efficient and accurate scene text detector[C]// 2017 IEEE Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2017: 2642-2651.
[2]	陈晓龙, 陈显龙, 袁建平, 等. 基于深度学习的电力设备铭牌识别[J]. 广西大学学报: 自然科学版, 2018, 43(6): 2216-2226.
	CHEN X L, CHEN X L, YUAN J P, et al. Electricity equipment nameplate recognition based on deep learning[J]. Journal of Guangxi University: Natural Science Edition, 2018, 43(6): 2216-2226 (in Chinese).
[3]	GUO C J, MA X L, SHU Y, et al. Text detection of power equipment nameplates based on deep learning[C]// 2019 IEEE 9th Annual International Conference on CYBER Technology in Automation, Control, and Intelligent Systems. New York: IEEE Press, 2020: 1635-1639.
[4]	LIAO M H, SHI B G, BAI X, et al. TextBoxes: a fast text detector with a single deep neural network[EB/OL]. [2022-05-10]. https://arxiv.org/abs/1611.06779v1.
[5]	ZHU Y X, DU J. Sliding line point regression for shape robust scene text detection[C]// The 24th International Conference on Pattern Recognition. New York: IEEE Press, 2018: 3735-3740.
[6]	SHI B G, BAI X, BELONGIE S. Detecting oriented text in natural images by linking segments[C]// 2017 IEEE Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2017: 3482-3490.
[7]	MA J Q, SHAO W Y, YE H, et al. Arbitrary-oriented scene text detection via rotation proposals[J]. IEEE Transactions on Multimedia, 2018, 20(11): 3111-3122. DOI URL
[8]	BAEK Y, LEE B, HAN D, et al. Character region awareness for text detection[C]// 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2020: 9357-9366.
[9]	XIE E Z, ZANG Y H, SHAO S, et al. Scene text detection with supervised pyramid context network[J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2019, 33(1): 9038-9045. DOI URL
[10]	LONG S B, RUAN J Q, ZHANG W J, et al. TextSnake: a flexible representation for detecting text of arbitrary shapes[M]//Computer Vision - ECCV 2018. Cham: Springer International Publishing, 2018: 19-35.
[11]	WANG W H, XIE E Z, LI X, et al. Shape robust text detection with progressive scale expansion network[C]// 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2020: 9328-9337.
[12]	LIU Y L, CHEN H, SHEN C H, et al. ABCNet: real-time scene text spotting with adaptive bezier-curve network[C]// 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2020: 9806-9815.
[13]	HUANG M X, LIU Y L, PENG Z H, et al. SwinTextSpotter: scene text spotting via better synergy between text detection and text recognition[C]// 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2022: 4583-4593.
[14]	LIAO M H, ZOU Z S, WAN Z Y, et al. Real-time scene text detection with differentiable binarization and adaptive scale fusion[C]// IEEE Transactions on Pattern Analysis and Machine Intelligence. New York: IEEE Press, 2022: 919-931.
[15]	WOO S, PARK J, LEE J Y, et al. CBAM: convolutional block attention module[M]//Computer Vision - ECCV 2018. Cham: Springer International Publishing, 2018: 3-19.
[16]	HE K M, ZHANG X Y, REN S Q, et al. Deep residual learning for image recognition[C]// 2016 IEEE Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2016: 770-778.
[17]	HOWARD A, SANDLER M, CHEN B, et al. Searching for MobileNetV3[C]// 2019 IEEE/CVF International Conference on Computer Vision. New York: IEEE Press, 2020: 1314-1324.
[18]	MA N N, ZHANG X Y, ZHENG H T, et al. ShuffleNet V2: practical guidelines for efficient CNN architecture design[C]// Computer Vision - ECCV 2018:15th European Conference. New York: ACM, 2018: 122-138.

网络模型	Datasets	P(%)	R(%)	F(%)	FPS
DBNet	ICDAR2013	96.3	90.6	93.4	27.1
DBNet	ICDAR2015	93.7	86.1	89.7	25.8
PSENet	ICDAR2013	94.6	89.4	91.9	17.3
PSENet	ICDAR2015	89.3	85.2	87.2	15.7
EAST	ICDAR2013	89.4	83.1	86.1	57.6
EAST	ICDAR2015	83.3	78.3	81.0	53.1
Ours	ICDAR2013	97.5	91.1	94.2	23.8
Ours	ICDAR2015	94.1	87.3	90.6	21.5

网络模型	Datasets	P(%)	R(%)	F(%)	FPS
DBNet	ICDAR2013	96.3	90.6	93.4	27.1
DBNet	ICDAR2015	93.7	86.1	89.7	25.8
PSENet	ICDAR2013	94.6	89.4	91.9	17.3
PSENet	ICDAR2015	89.3	85.2	87.2	15.7
EAST	ICDAR2013	89.4	83.1	86.1	57.6
EAST	ICDAR2015	83.3	78.3	81.0	53.1
Ours	ICDAR2013	97.5	91.1	94.2	23.8
Ours	ICDAR2015	94.1	87.3	90.6	21.5

网络模型	P(%)	R(%)	F(%)	FPS
DBNet	87.3	76.4	81.4	16.7
PSEnet	83.7	74.3	78.7	9.1
EAST	77.6	69.4	73.3	31.7
ABCNet	88.6	79.3	83.7	14.0
SwinTextSpotter	83.4	72.5	77.6	19.7
Ours	90.3	79.7	84.7	13.1

网络模型	P(%)	R(%)	F(%)	FPS
DBNet	87.3	76.4	81.4	16.7
PSEnet	83.7	74.3	78.7	9.1
EAST	77.6	69.4	73.3	31.7
ABCNet	88.6	79.3	83.7	14.0
SwinTextSpotter	83.4	72.5	77.6	19.7
Ours	90.3	79.7	84.7	13.1

数据增强方式	P	R	F
无数据增强	87.3	76.4	81.4
Ours	88.1	77.2	82.2