A real-time metallic surface defect detection algorithm based on E-YOLOX

doi:10.11996/JG.j.2095-302X.2023040677

Abstract

Abstract:

For metallic surface defect detection, a novel algorithm E-YOLOX was proposed to address the shortcomings of current methods, such as poor generalization ability and low detection speed. The algorithm utilized a new feature extraction network, ECMNet, which employed depth convolutions to reduce the parameters and computational cost of the network. The linear inverse bottleneck residual network was in use to enhance the feature extraction capability, while preserving more key features that were manifold distributed in low-dimensional subspaces within high-dimensional tensors during forward propagation. Additionally, the extended cross-stage partial network structure diversified the gradient flow paths of neural networks, making deep neural networks learn and converge more efficiently. Moreover, a new data augmentation method edge Cutout was proposed, which generated adaptive masks covering random regions of the image during the training process, enhancing the detection and generalization ability of the network. The experimental results demonstrated that E-YOLOX-l achieved 77.2% mAP in detection accuracy on the aluminum profile surface defect dataset AL6-DET and 36.8% mAP on steel surface defect dataset GC10-DET, which was 3.6% and 1.7% higher than the baseline algorithm YOLOX-l. At the same time, the number of parameters was reduced by 55% and the computational cost was reduced by 49%. The detection speed was 57 FPS, an increase of 21 FPS. Compared with other related algorithms, the new algorithm achieved a higher detection accuracy and a better balance between accuracy and speed.

Key words: metallic surface, defect detection, deep learning, YOLOX, lightweight network, data augmentation

CLC Number:

TP391

CAO Yi-qin, ZHOU Yi-wei, XU Lu. A real-time metallic surface defect detection algorithm based on E-YOLOX[J]. Journal of Graphics, 2023, 44(4): 677-690.

Figures/Tables 15

References 47

[1]	LIU W W, YAN Y H, LI J, et al. Automated on-line fast detection for surface defect of steel strip based on multivariate discriminant function[C]// 2008 Second International Symposium on Intelligent Information Technology Application. New York: IEEE Press, 2009: 493-497.
[2]	LUO Q W, SUN Y C, LI P C, et al. Generalized completed local binary patterns for time-efficient steel surface defect classification[J]. IEEE Transactions on Instrumentation and Measurement, 2019, 68(3): 667-679. DOI URL
[3]	CALEB P, STEUER M. Classification of surface defects on hot rolled steel using adaptive learning methods[C]// KES’2000. Fourth International Conference on Knowledge-Based Intelligent Engineering Systems and Allied Technologies. Proceedings (Cat. No.00TH8516). New York: IEEE Press, 2002: 103-108.
[4]	ASHOUR M W, KHALID F, HALIN A A, et al. Surface defects classification of hot-rolled steel strips using multi-directional shearlet features[J]. Arabian Journal for Science and Engineering, 2019, 44(4): 2925-2932. DOI
[5]	AI Y H, XU K. Surface detection of continuous casting slabs based on curvelet transform and kernel locality preserving projections[J]. Journal of Iron and Steel Research, International, 2013, 20(5): 80-86.
[6]	MEDINA R, GAYUBO F, GONZÁLEZ-RODRIGO L M, et al. Automated visual classification of frequent defects in flat steel coils[J]. The International Journal of Advanced Manufacturing Technology, 2011, 57(9): 1087-1097. DOI URL
[7]	GIRSHICK R, DONAHUE J, DARRELL T, et al. Rich feature hierarchies for accurate object detection and semantic segmentation[C]// 2014 IEEE Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2014: 580-587.
[8]	GIRSHICK R. Fast R-CNN[C]// 2015 IEEE International Conference on Computer Vision. New York: IEEE Press, 2016: 1440-1448.
[9]	REN S Q, HE K M, GIRSHICK R, et al. Faster R-CNN: towards real-time object detection with region proposal networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(6): 1137-1149. DOI PMID
[10]	CAI Z W, VASCONCELOS N. Cascade R-CNN: delving into high quality object detection[C]// 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2018: 6154-6162.
[11]	REDMON J, DIVVALA S, GIRSHICK R, et al. You only look once: unified, real-time object detection[C]// 2016 IEEE Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2016: 779-788.
[12]	REDMON J, FARHADI A. YOLO9000: better, faster, stronger[C]// 2017 IEEE Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2017: 6517-6525.
[13]	REDMON J, FARHADI A. YOLOv3: an incremental improvement[EB/OL]. (2018-04-08) [2022-06-04]. https://arxiv.org/abs/1804.02767.
[14]	BOCHKOVSKIY A, WANG C Y, LIAO H Y M. YOLOv4: optimal speed and accuracy of object detection[EB/OL]. (2020-04-23) [2022-06-04]. https://arxiv.org/abs/2004.10934.
[15]	WANG C Y, BOCHKOVSKIY A, LIAO H Y M. Scaled-YOLOv4: scaling cross stage partial network[C]// 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2021: 13024-13033.
[16]	GE Z, LIU S T, WANG F, et al. YOLOX: exceeding YOLO series in 2021[EB/OL]. (2020-08-06) [2022-06-04]. https://arxiv.org/abs/2107.08430.
[17]	LIU W, ANGUELOV D, ERHAN D, et al. SSD: single shot MultiBox detector[C]// European Conference on Computer Vision. Cham: Springer International Publishing, 2016: 21-37.
[18]	TIAN Z, SHEN C H, CHEN H, et al. FCOS: fully convolutional one-stage object detection[C]// 2019 IEEE/CVF International Conference on Computer Vision. New York: IEEE Press, 2020: 9626-9635.
[19]	ZHANG S F, CHI C, YAO Y Q, et al. Bridging the gap between anchor-based and anchor-free detection via adaptive training sample selection[C]// 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2020: 9756-9765.
[20]	黄凤荣, 李杨, 郭兰申, 等. 基于Faster R-CNN的零件表面缺陷检测算法[J]. 计算机辅助设计与图形学学报, 2020, 32(6): 883-893.
	HUANG F R, LI Y, GUO L S, et al. Method for detecting surface defects of engine parts based on faster R-CNN[J]. Journal of Computer-Aided Design & Computer Graphics, 2020, 32(6): 883-893 (in Chinese).
[21]	于海涛, 李健升, 刘亚姣, 等. 基于级联神经网络的型钢表面缺陷检测算法[J]. 计算机应用, 2023, 43(1): 232-241. DOI
	YU H T, LI J S, LIU Y J, et al. Detection algorithm of surface defects of section steel based on cascade neural network[J]. Journal of Computer Applications, 2023, 43(1): 232-241 (in Chinese).
[22]	MA Z X, LI Y B, HUANG M H, et al. A lightweight detector based on attention mechanism for aluminum strip surface defect detection[J]. Computers in Industry, 2022, 136: 103585. DOI URL
[23]	GUO Z X, WANG C S, YANG G, et al. MSFT-YOLO: improved YOLOv5 based on transformer for detecting defects of steel surface[J]. Sensors: Basel, Switzerland, 2022, 22(9): 3467.
[24]	孙琪翔, 何宁, 张敬尊, 等. 基于非局部高分辨率网络的轻量化人体姿态估计方法[J]. 计算机应用, 2022, 42(5): 1398-1406. DOI
	SUN Q X, HE N, ZHANG J Z, et al. Lightweight human pose estimation method based on non-local high-resolution network[J]. Journal of Computer Applications, 2022, 42(5): 1398-1406 (in Chinese). DOI
[25]	胡海涛, 杜昊晨, 王素琴, 等. 改进YOLOX的药品泡罩铝箔表面缺陷检测方法[J]. 图学学报, 2022, 43(5): 803-814.
	HU H T, DU H C, WANG S Q, et al. Improved YOLOX method for detecting surface defects of drug blister aluminum foil[J]. Journal of Graphics, 2022, 43(5): 803-814 (in Chinese).
[26]	LV X M, DUAN F J, JIANG J J, et al. Deep metallic surface defect detection: The new benchmark and detection network[J]. Sensors, 2020, 20(6): 1562. DOI URL
[27]	WANG C Y, MARK LIAO H Y, WU Y H, et al. CSPNet: a new backbone that can enhance learning capability of CNN[C]// 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. New York: IEEE Press, 2020: 1571-1580.
[28]	HOWARD A G, ZHU M L, CHEN B, et al. MobileNets: efficient convolutional neural networks for mobile vision applications[EB/OL]. (2017-04-17) [2022-06-04]. https://arxiv.org/abs/1704.04861.
[29]	SANDLER M, HOWARD A, ZHU M L, et al. MobileNetV2: inverted residuals and linear bottlenecks[C]// 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2018: 4510-4520.
[30]	HOWARD A, SANDLER M, CHEN B, et al. Searching for MobileNetV3[C]// 2019 IEEE/CVF International Conference on Computer Vision. New York: IEEE Press, 2020: 1314-1324.
[31]	ZHANG X Y, ZHOU X Y, LIN M X, et al. ShuffleNet: an extremely efficient convolutional neural network for mobile devices[C]// 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2018: 6848-6856.
[32]	MA N N, ZHANG X Y, ZHENG H T, et al. ShuffleNet V2: practical guidelines for efficient CNN architecture design[C]// European Conference on Computer Vision. Cham: Springer International Publishing, 2018: 122-138.
[33]	YU G H, CHANG Q Y, LV W Y, et al. PP-PicoDet: a better real-time object detector on mobile devices[EB/OL]. (2021-11-01) [2022-06-04]. https://arxiv.org/abs/2111.00902.
[34]	CHOLLET F. Xception: deep learning with depthwise separable convolutions[C]// 2017 IEEE Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2017: 1800-1807.
[35]	LIU S, QI L, QIN H F, et al. Path aggregation network for instance segmentation[C]// 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2018: 8759-8768.
[36]	LIN T Y, DOLLÁR P, GIRSHICK R, et al. Feature pyramid networks for object detection[C]// 2017 IEEE Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2017: 936-944.
[37]	HE K M, ZHANG X Y, REN S Q, et al. Deep residual learning for image recognition[C]// 2016 IEEE Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2016: 770-778.
[38]	GLOROT X, BORDES A, BENGIO Y. Deep sparse rectifier neural networks[EB/OL]. [2022-06-04]. http://proceedings.mlr.press/v15.
[39]	BREHMER J, CRANMER K. Flows for simultaneous manifold learning and density estimation[C]// Advances in Neural Information Processing Systems. New York: ACM Press, 2020: 442-453.
[40]	POPE P, ZHU C, ABDELKADER A, et al. The intrinsic dimension of images and its impact on learning[EB/OL]. (2021-04-18) [2022-06-04]. https://arxiv.org/abs/2104.08894.
[41]	LIU Z, MAO H Z, WU C Y, et al. A ConvNet for the 2020s[C]// 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2022: 11966-11976.
[42]	DEVRIES T, TAYLOR G W. Improved regularization of convolutional neural networks with cutout[EB/OL]. (2017-08-15) [2022-06-04]. https://arxiv.org/abs/1708.04552.
[43]	SUN P Z, ZHANG R F, JIANG Y, et al. Sparse R-CNN: end-to-end object detection with learnable proposals[C]// 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2021: 14449-14458.
[44]	FENG C J, ZHONG Y J, GAO Y, et al. TOOD: task-aligned one-stage object detection[C]// 2021 IEEE/CVF International Conference on Computer Vision. New York: IEEE Press, 2022: 3490-3499.
[45]	ZHANG H, WANG Y, DAYOUB F, et al. Varifocalnet: an iou-aware dense object detector[C]// 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2021:8514-8523.
[46]	WANG C Y, BOCHKOVSKIY A, LIAO H Y M. YOLOv7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors[EB/OL]. (2022-07-06) [2022-09-04]. https://arxiv.org/abs/2207.02696.
[47]	CHEN K, WANG J Q, PANG J M, et al. MMDetection: open MMLab detection toolbox and benchmark[EB/OL]. (2019-07-17) [2022-06-04]. https://arxiv.org/abs/1906.07155.

算法	特征提取网络	训练大小	AL6-DET	GC10-DET	参数量(M)	计算量(G)	FPS
Sparse RCNN	ResNet50	640	72.2	33.7	105.9	64.6	25
SSD	VGG16	512	67.5	31.4	24.4	88.4	38
ATSS	ResNet50	640	67.9	33.1	31.9	80.6	35
TOOD	ResNet50	640	68.8	31.5	31.8	72.3	32
Varifocal Net	ResNet50	640	67.9	32.6	32.5	75.7	31
YOLOv3	DarkNet53	608	67.2	30.7	61.6	70.0	39
YOLOv5-s	CSPDarkNet	640	72.1	33.5	7.0	15.8	62
YOLOv5-m	CSPDarkNet	640	73.8	33.7	20.9	47.9	51
YOLOv5-l	CSPDarkNet	640	75.5	34.3	46.2	108.4	43
YOLOX-s	CSPDarkNet	640	71.3	33.2	8.9	26.8	53
YOLOX-m	CSPDarkNet	640	72.5	33.9	25.3	73.8	42
YOLOX-l	CSPDarkNet	640	73.6	35.1	54.2	155.6	36
YOLOv7-tiny	E-ELAN	640	70.4	32.2	6.0	13.3	71
YOLOv7	E-ELAN	640	75.2	34.8	37.2	105.3	54
E-YOLOX-s	ECMNet	640	74.7	34.5	5.9	18.9	70
E-YOLOX-m	ECMNet	640	75.9	35.3	13.6	44.6	61
E-YOLOX-l	ECMNet	640	77.2	36.8	24.3	79.0	57

算法	特征提取网络	训练大小	AL6-DET	GC10-DET	参数量(M)	计算量(G)	FPS
Sparse RCNN	ResNet50	640	72.2	33.7	105.9	64.6	25
SSD	VGG16	512	67.5	31.4	24.4	88.4	38
ATSS	ResNet50	640	67.9	33.1	31.9	80.6	35
TOOD	ResNet50	640	68.8	31.5	31.8	72.3	32
Varifocal Net	ResNet50	640	67.9	32.6	32.5	75.7	31
YOLOv3	DarkNet53	608	67.2	30.7	61.6	70.0	39
YOLOv5-s	CSPDarkNet	640	72.1	33.5	7.0	15.8	62
YOLOv5-m	CSPDarkNet	640	73.8	33.7	20.9	47.9	51
YOLOv5-l	CSPDarkNet	640	75.5	34.3	46.2	108.4	43
YOLOX-s	CSPDarkNet	640	71.3	33.2	8.9	26.8	53
YOLOX-m	CSPDarkNet	640	72.5	33.9	25.3	73.8	42
YOLOX-l	CSPDarkNet	640	73.6	35.1	54.2	155.6	36
YOLOv7-tiny	E-ELAN	640	70.4	32.2	6.0	13.3	71
YOLOv7	E-ELAN	640	75.2	34.8	37.2	105.3	54
E-YOLOX-s	ECMNet	640	74.7	34.5	5.9	18.9	70
E-YOLOX-m	ECMNet	640	75.9	35.3	13.6	44.6	61
E-YOLOX-l	ECMNet	640	77.2	36.8	24.3	79.0	57

算法	mAP₅₀	不导电	桔皮	角位漏底	漏底	起坑	杂色
Sparse RCNN	88.7	59.7	100	85.4	86.8	100	100
SSD	91.7	72.7	100	88.7	89.7	99.1	100
ATSS	89.5	63.5	99.8	90.5	87.1	96.2	100
TOOD	89.2	60.9	100	92.9	85.4	97.7	98.4
Varifocal Net	88.6	65.8	99.9	89.3	84.8	93.9	97.8
YOLOv3	90.6	70.5	100	93.5	82.0	97.8	100
YOLOv5-s	91.0	62.4	100	93.8	89.8	100	100
YOLOv5-m	91.5	67.0	98.9	92.5	90.8	100	100
YOLOv5-l	92.2	67.4	100	94.5	91.2	100	100
YOLOX-s	90.2	65.1	100	85.7	91.1	99.5	100
YOLOX-m	91.3	66.6	100	93.4	87.9	100	100
YOLOX-l	91.6	67.7	100	92.6	89.3	100	100
YOLOv7-tiny	91.1	66.8	99.4	90.6	90.1	99.6	100
YOLOv7	92.5	69.7	100	93.7	91.6	100	100
E-YOLOX-s	92.3	74.6	99.8	91.4	87.7	100	100
E-YOLOX-m	92.6	77.0	98.5	92.0	88.3	100	100
E-YOLOX-l	93.4	77.6	100	92.4	90.6	100	100

算法	mAP₅₀	不导电	桔皮	角位漏底	漏底	起坑	杂色
Sparse RCNN	88.7	59.7	100	85.4	86.8	100	100
SSD	91.7	72.7	100	88.7	89.7	99.1	100
ATSS	89.5	63.5	99.8	90.5	87.1	96.2	100
TOOD	89.2	60.9	100	92.9	85.4	97.7	98.4
Varifocal Net	88.6	65.8	99.9	89.3	84.8	93.9	97.8
YOLOv3	90.6	70.5	100	93.5	82.0	97.8	100
YOLOv5-s	91.0	62.4	100	93.8	89.8	100	100
YOLOv5-m	91.5	67.0	98.9	92.5	90.8	100	100
YOLOv5-l	92.2	67.4	100	94.5	91.2	100	100
YOLOX-s	90.2	65.1	100	85.7	91.1	99.5	100
YOLOX-m	91.3	66.6	100	93.4	87.9	100	100
YOLOX-l	91.6	67.7	100	92.6	89.3	100	100
YOLOv7-tiny	91.1	66.8	99.4	90.6	90.1	99.6	100
YOLOv7	92.5	69.7	100	93.7	91.6	100	100
E-YOLOX-s	92.3	74.6	99.8	91.4	87.7	100	100
E-YOLOX-m	92.6	77.0	98.5	92.0	88.3	100	100
E-YOLOX-l	93.4	77.6	100	92.4	90.6	100	100

算法	mAP₅₀	冲孔	焊缝	月间隙	水斑	油斑	丝状斑	夹杂	压痕	折痕	腰折
Sparse RCNN	65.2	95.7	93.9	95.4	78.1	70.0	62.4	27.3	31.5	23.1	74.9
SSD	64.7	96.5	93.8	93.8	76.2	68.8	61.2	16.9	29.2	31.4	78.8
ATSS	67.7	91.8	86.4	96.5	79.6	73.7	67.6	28.1	47.3	20.5	85.7
TOOD	65.2	90.2	90.6	91.9	81.5	78.0	60.5	28.3	47.8	7.3	75.9
Varifocal Net	64.5	88.5	95.4	94.3	82.5	79.5	66.8	23.2	26.0	3.2	85.5
YOLOv3	65.6	93.3	91.0	98.0	77.4	70.1	62.6	38.4	36.3	26.2	62.4
YOLOv5-s	68.5	96.1	94.0	96.0	82.8	80.3	68.5	32.3	31.3	23.6	80.2
YOLOv5-m	69.3	97.2	95.4	96.6	76.9	76.0	68.2	33.6	28.3	33.5	86.9
YOLOv5-l	71.4	96.8	96.0	96.9	79.3	82.4	69.9	30.6	37.6	36.7	88.2
YOLOX-s	67.5	97.0	95.3	96.3	78.1	76.2	64.2	30.0	31.0	17.2	90.0
YOLOX-m	69.0	96.1	95.4	95.5	78.9	79.5	66.7	25.8	32.3	28.4	91.1
YOLOX-l	69.5	95.1	94.8	94.2	77.5	81.7	65.4	21.2	32.2	39.9	93.0
YOLOv7-tiny	66.5	96.4	80.9	97.5	82.3	74.7	68	30.3	23.8	30.2	80.8
YOLOv7	67.7	96.2	84.3	96.4	83.7	77.0	66.8	29.5	16.2	44.3	82.3
EDDN	65.2	90.0	88.5	84.8	55.8	62.2	65.0	25.6	36.4	52.1	91.9
E-YOLOX-s	68.3	97.3	93.3	96.4	79.5	78.2	67.0	33.3	31.4	18.4	88.5
E-YOLOX-m	70.4	97.0	94.6	96.9	78.4	79.6	68.8	24.8	36.4	35.2	92.3
E-YOLOX-l	73.3	97.2	95.7	98.7	80.5	76.5	67.3	32.1	48.2	41.7	94.6