Detection of apparent defects in a small sample of industrial products with category imbalance

doi:10.11996/JG.j.2095-302X.2025030568

Abstract

Abstract:

It has been demonstrated that generic target detection networks exhibit reduced overall detection accuracy when the number of defect samples is limited and the distribution of defect categories is uneven. Furthermore, the detection accuracy is markedly diminished for tail categories with particularly scarce defect samples. Based on these observations, an improved method for detecting apparent defects in industrial products using YOLOv8s was developed. Phantom convolution GSConv was employed in the Neck network to diminish the network complexity while simultaneously augmenting its nonlinearity, thus circumventing the potential issue of overfitting. Furthermore, the aggregation module VoV-GSCSP was employed to facilitate the extraction and fusion of features at varying levels, thereby enhancing the network’s capacity for feature extraction and fusion. A reweighted loss function was adopted to balance the training loss contributions across different categories of samples, increasing the loss contribution percentage for the tail category and thereby enhancing defect detection accuracy for the tail category. In comparison with the baseline model, the enhanced method achieved a mAP of 93.3% for the apparent defect detection accuracy in acupuncture needles, representing a 5.0% enhancement, and achieved a 9.1% improvement for broken needle defects. It should be noted that these improvements were achieved with the minimal number of samples. For medicinal plates, a mAP of apparent defect detection accuracy was achieved at 91.4%, representing a 2.6% improvement, and the improvement for dirty defects with the fewest samples was achieved at 3.2%. On the steel dataset, which featured a greater number of samples with uneven distribution, the overall defect detection accuracy improved by 2.6% in mAP. The experiments demonstrated that the enhanced methodology can markedly enhance the overall detection accuracy of apparent defects in industrial products under conditions of the limited number of defect samples and the imbalanced distribution of categories. Furthermore, it can markedly enhance the detection accuracy for categories with sparse samples, exhibiting excellent generalization capabilities.

Key words: apparent defect detection, small sample size, class imbalance, GSConv, re-weighting loss function

CLC Number:

WANG Suqin, DU Yujie, SHI Min, ZHU Dengming. Detection of apparent defects in a small sample of industrial products with category imbalance[J]. Journal of Graphics, 2025, 46(3): 568-577.

Figures/Tables 14

References 25

[1]	罗东亮, 蔡雨萱, 杨子豪, 等. 工业缺陷检测深度学习方法综述[J]. 中国科学: 信息科学, 2022, 52(6): 1002-1039.
	LUO D L, CAI Y X, YANG Z H, et al. Survey on industrial defect detection with deep learning[J]. SCIENTIA SINICA Informationis, 2022, 52(6): 1002-1039 (in Chinese).
[2]	疏义桂. 基于机器视觉的铝塑泡罩包装药品缺陷检测[D]. 武汉: 华中科技大学, 2013.
	SHU Y G. Research on detection of medicines in aluminum- plastic blister package based on machine vision[D]. Wuhan: Huazhong University of Science and Technology, 2013 (in Chinese).
[3]	谷紫颖. 铝塑泡罩药品缺陷检测技术的研究[D]. 济南: 山东大学, 2020.
	GU Z Y. Research on defect detection technology of aluminum- plastic blister medicine[D]. Jinan: Shandong University, 2020 (in Chinese).
[4]	邵延华, 张铎, 楚红雨, 等. 基于深度学习的YOLO目标检测综述[J]. 电子与信息学报, 2022, 44(10): 3697-3708.
	SHAO Y H, ZHANG D, CHU H Y, et al. A review of YOLO object detection based on deep learning[J]. Journal of Electronics & Information Technology, 2022, 44(10): 3697-3708 (in Chinese).
[5]	李炳臻, 姜文志, 顾佼佼, 等. 基于卷积神经网络的目标检测算法综述[J]. 计算机与数字工程, 2022, 50(5): 1010-1017.
	LI B Z, JIANG W Z, GU J J, et al. Review of target detection algorithms based on deep learning[J]. Computer & Digital Engineering, 2022, 50(5): 1010-1017 (in Chinese).
[6]	LI H L, LI J, WEI H B, et al. Slim-neck by GSConv: a lightweight-design for real-time detector architectures[J]. Journal of Real-Time Image Processing, 2024, 21(3): 63.
[7]	王素琴, 任琪, 石敏, 等. 基于异常检测的产品表面缺陷检测与分割[J]. 图学学报, 2022, 43(3): 377-386.
	WANG S Q, REN Q, SHI M, et al. Product surface defect detection and segmentation based on anomaly detection[J]. Journal of Graphics, 2022, 43(3): 377-386 (in Chinese). DOI
[8]	张玥, 陈锡伟, 陈梦丹, 等. 基于对比学习生成对抗网络的无监督工业品表面异常检测[J]. 电子测量与仪器学报, 2023, 37(10): 193-201.
	ZHANG Y, CHEN X W, CHEN M D, et al. Unsupervised surface anomaly detection of industrial products based on contrastive learning generative adversarial network[J]. Journal of Electronic Measurement and Instrumentation, 2023, 37(10): 193-201 (in Chinese).
[9]	杜娟, 杨钧植. 基于迁移学习的小样本连接器缺陷检测方法[J]. 自动化与信息工程, 2022, 43(5): 1-7.
	DU J, YANG J Z. Small-sample connector defect detection method based on transfer learning[J]. Automation & Information Engineering, 2022, 43(5): 1-7 (in Chinese).
[10]	翟永杰, 胡哲东, 白云山, 等. 融合迁移学习的绝缘子缺陷分级检测方法[J]. 电子测量技术, 2023, 46(6): 23-30.
	ZHAI Y J, HU Z D, BAI Y S, et al. Integrating transfer learning for insulator defect grading detection[J]. Electronic Measurement Technology, 2023, 46(6): 23-30 (in Chinese).
[11]	XIAO W W, SONG K C, LIU J, et al. Graph embedding and optimal transport for few-shot classification of metal surface defect[J]. IEEE Transactions on Instrumentation and Measurement, 2022, 71: 5010310.
[12]	丁鹏, 卢文壮, 刘杰, 等. 基于生成对抗网络的叶片表面缺陷图像数据增强[J]. 组合机床与自动化加工技术, 2022(7): 18-21. DOI
	DING P, LU W Z, LIU J, et al. Image data augmentation of blade surface defects based on generative adversarial network[J]. Modular Machine Tool & Automatic Manufacturing Technique, 2022(7): 18-21 (in Chinese).
[13]	李可, 祁阳, 宿磊, 等. 基于改进ACGAN的钢表面缺陷视觉检测方法[J]. 机械工程学报, 2022, 58(24): 32-40. DOI
	LI K, QI Y, SU L, et al. Visual inspection of steel surface defects based on improved auxiliary classification generation adversarial network[J]. Journal of Mechanical Engineering, 2022, 58(24): 32-40 (in Chinese). DOI
[14]	JOHNSON J M, KHOSHGOFTAAR T M. Survey on deep learning with class imbalance[J]. Journal of Big Data, 2019, 6: 27.
[15]	WANG T, LI Y, KANG B Y, et al. The devil is in classification: a simple framework for long-tail instance segmentation[C]// The 16th European Conference on Computer Vision. Cham: Springer, 2020: 728-744.
[16]	ZHANG Y H, HUANG C, LOY C C. FASA: feature augmentation and sampling adaptation for long-tailed instance segmentation[C]// 2021 IEEE/CVF International Conference on Computer Vision. New York: IEEE Press, 2021: 3437-3446.
[17]	CUI Y, JIA M L, LIN T Y, et al. Class-balanced loss based on effective number of samples[C]// 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2019: 9260-9269.
[18]	LI B, YAO Y Q, TAN J R, et al. Equalized focal loss for dense long-tailed object detection[C]// 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2022: 6980-6989.
[19]	胡海涛, 杜昊晨, 王素琴, 等. 改进YOLOX的药品泡罩铝箔表面缺陷检测方法[J]. 图学学报, 2022, 43(5): 803-814.
	HU H T, DU H C, WANG S Q, et al. Improved YOLOX method for detecting surface defects of drug blister aluminum foil[J]. Journal of Graphics, 2022, 43(5): 803-814 (in Chinese).
[20]	王健, 肖迪, 冯李航, 等. 基于改进YOLOv8s的PCB小目标缺陷检测模型[EB/OL]. (024-08-20) [2024-09-08]http://kns.cnki.net/kcms/detail/11.2127.tp.20240819.1152.016.html.
	WANG J, XIAO D, FENG L H. A PCB small object defect detection model based on improved YOLOv8s[EB/OL]. (2024-08-20) [2024-09-08]http://kns.cnki.net/kcms/detail/11.2127.tp.20240819.1152.016.html (in Chinese).
[21]	李文举, 苏攀, 崔柳. 基于随机扰动的过拟合抑制算法[J]. 计算机仿真, 2022, 39(5): 134-138.
	LI W J, SU P, CUI L. Over-fitting suppression algorithm based on random perturbation[J]. Computer Simulation, 2022, 39(5): 134-138 (in Chinese).
[22]	LI X, WANG W H, WU L J, et al. Generalized focal loss: learning qualified and distributed bounding boxes for dense object detection[C]// The 34th International Conference on Neural Information Processing Systems. New York: ACM, 2020: 1763.
[23]	TAN J R, LU X, ZHANG G, et al. Equalization loss v2: a new gradient balance approach for long-tailed object detection[C]// 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2020: 1685-1694.
[24]	WANG C Y, BOCHKOVSKIY A, LIAO H Y M. YOLOv7: trainable bag-of-freebies sets new state-of-the-art for real-time object detectors[C]// 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2023: 7464-7475.
[25]	WANG A, CHEN H, LIU L H, et al. YOLOv10:real-time end-to-end object detection[EB/OL]. (2024-10-30) [2024-12-10]https://arxiv.org/abs/2405.14458.

Model	AP/%						mAP/%	FPS
Model	空泡(169)	多针(69)	药板褶皱(54)	泡罩褶皱(37)	异常针(20)	断针(10)	mAP/%	FPS
Faster- RCNN	99.5	73.6	92.3	58.2	53.3	46.7	70.6	11.52
YOLOv5s	99.5	83.7	94.3	70.6	88.3	72.2	84.8	102.04
YOLOv5m	99.5	94.9	95.6	66.5	93.9	89.4	90.0	43.67
YOLOv5s+CBL	99.5	86.4	94.0	65.0	78.1	82.0	84.2	102.04
YOLOv7	99.5	76.0	81.9	47.6	54.2	68.0	71.2	46.08
YOLOv8s	99.5	92.4	91.5	72.8	92.4	81.1	88.3	98.04
YOLOv8m	99.5	94.9	95.8	80.6	90.6	84.5	91.0	43.10
YOLOv8s+EFL	99.5	92.7	95.8	73.5	94.6	84.0	90.0	96.15
YOLOv10s	99.4	83.1	91.6	69.2	90.1	75.9	84.0	86.96
Ours	99.5	96.8	95.6	82.7	94.8	90.2	93.3	99.01

Model	AP/%						mAP/%	FPS
Model	空泡(169)	多针(69)	药板褶皱(54)	泡罩褶皱(37)	异常针(20)	断针(10)	mAP/%	FPS
Faster- RCNN	99.5	73.6	92.3	58.2	53.3	46.7	70.6	11.52
YOLOv5s	99.5	83.7	94.3	70.6	88.3	72.2	84.8	102.04
YOLOv5m	99.5	94.9	95.6	66.5	93.9	89.4	90.0	43.67
YOLOv5s+CBL	99.5	86.4	94.0	65.0	78.1	82.0	84.2	102.04
YOLOv7	99.5	76.0	81.9	47.6	54.2	68.0	71.2	46.08
YOLOv8s	99.5	92.4	91.5	72.8	92.4	81.1	88.3	98.04
YOLOv8m	99.5	94.9	95.8	80.6	90.6	84.5	91.0	43.10
YOLOv8s+EFL	99.5	92.7	95.8	73.5	94.6	84.0	90.0	96.15
YOLOv10s	99.4	83.1	91.6	69.2	90.1	75.9	84.0	86.96
Ours	99.5	96.8	95.6	82.7	94.8	90.2	93.3	99.01

Model	AP/%				mAP/%	FPS
Model	破损(551)	铝箔压坏(202)	圆点(165)	脏污(56)	mAP/%	FPS
Faster-RCNN	91.7	94.5	83.2	82.9	88.1	11.15
YOLOv5s	86.3	94.7	90.7	84.0	89.0	108.70
YOLOv5m	91.2	94.6	90.0	84.5	90.1	42.74
YOLOv5s+CBL	90.6	95.6	90.2	80.4	89.2	106.38
YOLOv7	84.2	88.9	74.7	80.0	75.1	38.91
YOLOv8s	92.2	93.8	87.5	81.6	88.8	88.50
YOLOv8m	91.9	94.1	89.4	84.4	90.0	37.88
YOLOv8s+EFL	91.6	93.2	88.8	67.9	85.4	88.50
YOLOv10s	93.1	94.1	84.6	78.1	87.4	79.37
Ours	93.7	94.6	92.4	84.8	91.4	90.09

Model	AP/%				mAP/%	FPS
Model	破损(551)	铝箔压坏(202)	圆点(165)	脏污(56)	mAP/%	FPS
Faster-RCNN	91.7	94.5	83.2	82.9	88.1	11.15
YOLOv5s	86.3	94.7	90.7	84.0	89.0	108.70
YOLOv5m	91.2	94.6	90.0	84.5	90.1	42.74
YOLOv5s+CBL	90.6	95.6	90.2	80.4	89.2	106.38
YOLOv7	84.2	88.9	74.7	80.0	75.1	38.91
YOLOv8s	92.2	93.8	87.5	81.6	88.8	88.50
YOLOv8m	91.9	94.1	89.4	84.4	90.0	37.88
YOLOv8s+EFL	91.6	93.2	88.8	67.9	85.4	88.50
YOLOv10s	93.1	94.1	84.6	78.1	87.4	79.37
Ours	93.7	94.6	92.4	84.8	91.4	90.09

Model	Neck	Loss	AP/%						mAP/%
Model	Neck	Loss	空泡	多针	药板褶皱	泡罩褶皱	异常针	断针	mAP/%
YOLOv8s	×	×	99.5	92.4	91.5	72.8	92.4	81.1	88.3
+Neck	√	×	99.5	93.2	95.4	78.5	93.4	85.6	90.9
+Loss	×	√	99.5	96.6	95.4	72.9	92.1	86.9	90.6
Ours	√	√	99.5	96.8	95.6	82.7	94.8	90.2	93.3