类别不均衡的少样本工业产品表观缺陷检测

doi:10.11996/JG.j.2095-302X.2025030568

图学学报 ›› 2025, Vol. 46 ›› Issue (3): 568-577.DOI: 10.11996/JG.j.2095-302X.2025030568

• 图像处理与计算机视觉 • 上一篇下一篇

类别不均衡的少样本工业产品表观缺陷检测

王素琴¹(), 杜雨洁¹, 石敏¹(), 朱登明²^,³

1.华北电力大学控制与计算机工程学院，北京 102206
2.中国科学院计算技术研究所，北京 100190
3.太仓中科信息技术研究院，江苏太仓 215400

收稿日期:2024-06-20 接受日期:2024-12-22 出版日期:2025-06-30 发布日期:2025-06-13
通讯作者:石敏(1975-)，女，副教授，博士。主要研究方向为虚拟现实与人工智能、计算机视觉。E-mail：shi_min@ncepu.edu.cn
第一作者:王素琴(1970-)，女，副教授，硕士。主要研究方向为计算机视觉、智能软件技术。E-mail：wsq@ncepu.edu.cn
基金资助:
苏州市科技计划前沿技术研究项目(SYG202327)

Detection of apparent defects in a small sample of industrial products with category imbalance

WANG Suqin¹(), DU Yujie¹, SHI Min¹(), ZHU Dengming²^,³

1. College of Control and Computer Engineering, North China Electric Power University, Beijing 102206, China
2. Institute of Computing Technology, Chinese Academy of Sciences, Beijing 100190, China
3. Taicang Institute of Information Technology, Taicang Jiangsu 215400, China

Received:2024-06-20 Accepted:2024-12-22 Published:2025-06-30 Online:2025-06-13
Contact: SHI Min (1975-), associate professor, Ph.D. Her main research interests cover virtual reality and artificial intelligence, computer vision. E-mail：shi_min@ncepu.edu.cn
First author：WANG Suqin (1970-), associate professor, master. Her main research interests cover computer vision, intelligent software technology. E-mail：wsq@ncepu.edu.cn
Supported by:
Frontier Technology Research Project of Suzhou Science and Technology Programme(SYG202327)

摘要/Abstract

摘要：

通用的目标检测网络在缺陷样本数量较少、缺陷类别分布不均衡时，总体检测精度偏低，在缺陷样本稀少的尾部类别上检测精度更低。为此，提出了一种基于改进YOLOv8s的工业产品表观缺陷检测方法。通过在Neck网络使用幻影卷积(GSConv)，降低网络复杂度的同时增强网络非线性能力，以避免过拟合风险。利用聚合模块VoV-GSCSP进一步提取与融合不同层次特征，提升网络特征提取与融合能力。通过采用重加权损失函数以平衡不同类别样本的训练损失贡献，加大尾部类别样本的损失贡献占比，从而提高尾部类别缺陷的检测精度。相比基线模型，改进方法对针灸针表观缺陷检测精度mAP为93.3%，提高5.0%，样本最少的断针缺陷提升9.1%；药板表观缺陷检测精度mAP为91.4%，提高2.6%，样本最少的脏污缺陷提升3.2%。在样本较多且分布不均衡的钢材数据集上，整体缺陷检测精度mAP提高2.6%。实验表明，该改进方法在缺陷样本少且类别分布不均衡时，可有效提升工业产品表观缺陷总体检测精度，对样本稀少的尾部类别检测精度改善明显，泛化性良好。

关键词: 表观缺陷检测, 少样本, 类别不均衡, GSConv, 重加权损失函数

Abstract:

It has been demonstrated that generic target detection networks exhibit reduced overall detection accuracy when the number of defect samples is limited and the distribution of defect categories is uneven. Furthermore, the detection accuracy is markedly diminished for tail categories with particularly scarce defect samples. Based on these observations, an improved method for detecting apparent defects in industrial products using YOLOv8s was developed. Phantom convolution GSConv was employed in the Neck network to diminish the network complexity while simultaneously augmenting its nonlinearity, thus circumventing the potential issue of overfitting. Furthermore, the aggregation module VoV-GSCSP was employed to facilitate the extraction and fusion of features at varying levels, thereby enhancing the network’s capacity for feature extraction and fusion. A reweighted loss function was adopted to balance the training loss contributions across different categories of samples, increasing the loss contribution percentage for the tail category and thereby enhancing defect detection accuracy for the tail category. In comparison with the baseline model, the enhanced method achieved a mAP of 93.3% for the apparent defect detection accuracy in acupuncture needles, representing a 5.0% enhancement, and achieved a 9.1% improvement for broken needle defects. It should be noted that these improvements were achieved with the minimal number of samples. For medicinal plates, a mAP of apparent defect detection accuracy was achieved at 91.4%, representing a 2.6% improvement, and the improvement for dirty defects with the fewest samples was achieved at 3.2%. On the steel dataset, which featured a greater number of samples with uneven distribution, the overall defect detection accuracy improved by 2.6% in mAP. The experiments demonstrated that the enhanced methodology can markedly enhance the overall detection accuracy of apparent defects in industrial products under conditions of the limited number of defect samples and the imbalanced distribution of categories. Furthermore, it can markedly enhance the detection accuracy for categories with sparse samples, exhibiting excellent generalization capabilities.

Key words: apparent defect detection, small sample size, class imbalance, GSConv, re-weighting loss function

中图分类号:

王素琴, 杜雨洁, 石敏, 朱登明. 类别不均衡的少样本工业产品表观缺陷检测[J]. 图学学报, 2025, 46(3): 568-577.

WANG Suqin, DU Yujie, SHI Min, ZHU Dengming. Detection of apparent defects in a small sample of industrial products with category imbalance[J]. Journal of Graphics, 2025, 46(3): 568-577.

图/表 14

图1 网络结构图

Fig. 1 Network structure diagram

图2 模块结构图

Fig. 2 Module structure diagram ((a) GSConv; (b) VoV-GSCSP; (c) GS bottleneck)

图3 针灸针表观缺陷示例图((a)空泡；(b)多针；(c)药板褶皱；(d)泡罩褶皱；(e)异常针；(f)断针)

Fig. 3 Examples of apparent defects in aluminium blister packaging for acupuncture needles ((a) Empty-blister; (b) Multi-pin; (c) Unusual-pin; (d) Plate-fold; (e) Blister-fold; (f) Break-pin)

图4 针灸针数据集缺陷样本数量分布图

Fig. 4 Distribution of the number of defects in the acupuncture needle dataset

图5 药板表观缺陷示例图((a)破损；(b)铝箔压坏；(c)圆点；(d)脏污)

Fig. 5 Example chart of defective samples from the drug plate dataset ((a) Damage; (b) Crush; (c) Dot; (d) Dirt)

图6 药板数据集缺陷样本数量分布图

Fig. 6 Distribution of the number of defects in the drug plate dataset

图7 钢材表观缺陷示例图((a)夹杂；(b)斑块；(c)开裂；(d)轧制氧化皮；(e)划痕；(f)点蚀表面)

Fig. 7 Example diagram of steel dataset defects ((a) Inclusion; (b) Patches; (c) Crazing; (d) Rolled-in_scale; (e) Scratches; (f) Pitted_surface)

图8 钢材数据集缺陷样本数量分布图

Fig. 8 Distribution of the number of defects in the steel dataset

表1 不同方法在针灸针数据集上的缺陷检测结果

Table 1 Defect detection results of different methods on acupuncture needle dataset

Model	AP/%						mAP/%	FPS
Model	空泡(169)	多针(69)	药板褶皱(54)	泡罩褶皱(37)	异常针(20)	断针(10)	mAP/%	FPS
Faster- RCNN	99.5	73.6	92.3	58.2	53.3	46.7	70.6	11.52
YOLOv5s	99.5	83.7	94.3	70.6	88.3	72.2	84.8	102.04
YOLOv5m	99.5	94.9	95.6	66.5	93.9	89.4	90.0	43.67
YOLOv5s+CBL	99.5	86.4	94.0	65.0	78.1	82.0	84.2	102.04
YOLOv7	99.5	76.0	81.9	47.6	54.2	68.0	71.2	46.08
YOLOv8s	99.5	92.4	91.5	72.8	92.4	81.1	88.3	98.04
YOLOv8m	99.5	94.9	95.8	80.6	90.6	84.5	91.0	43.10
YOLOv8s+EFL	99.5	92.7	95.8	73.5	94.6	84.0	90.0	96.15
YOLOv10s	99.4	83.1	91.6	69.2	90.1	75.9	84.0	86.96
Ours	99.5	96.8	95.6	82.7	94.8	90.2	93.3	99.01

图9 针灸针数据集检测效果对比图((a)真实值；(b) YOLOv8s；(c)本文方法)

Fig. 9 Comparison chart of the detection effect of acupuncture needle dataset ((a) Ground truth; (b) YOLOv8s; (c) Ours)

表2 不同方法在药板数据集上的缺陷检测结果

Table 2 Defect detection results of different methods on the pharmaceutical plate dataset

Model	AP/%				mAP/%	FPS
Model	破损(551)	铝箔压坏(202)	圆点(165)	脏污(56)	mAP/%	FPS
Faster-RCNN	91.7	94.5	83.2	82.9	88.1	11.15
YOLOv5s	86.3	94.7	90.7	84.0	89.0	108.70
YOLOv5m	91.2	94.6	90.0	84.5	90.1	42.74
YOLOv5s+CBL	90.6	95.6	90.2	80.4	89.2	106.38
YOLOv7	84.2	88.9	74.7	80.0	75.1	38.91
YOLOv8s	92.2	93.8	87.5	81.6	88.8	88.50
YOLOv8m	91.9	94.1	89.4	84.4	90.0	37.88
YOLOv8s+EFL	91.6	93.2	88.8	67.9	85.4	88.50
YOLOv10s	93.1	94.1	84.6	78.1	87.4	79.37
Ours	93.7	94.6	92.4	84.8	91.4	90.09

图10 药板数据集检测效果对比图((a)真实值；(b) YOLOv8s；(c)本文方法)

Fig. 10 Comparison of the detection effect of the drug plate dataset ((a) Ground truth; (b) YOLOv8s; (c) Ours)

表3 消融实验结果

Table 3 Results of ablation experiments

Model	Neck	Loss	AP/%						mAP/%
Model	Neck	Loss	空泡	多针	药板褶皱	泡罩褶皱	异常针	断针	mAP/%
YOLOv8s	×	×	99.5	92.4	91.5	72.8	92.4	81.1	88.3
+Neck	√	×	99.5	93.2	95.4	78.5	93.4	85.6	90.9
+Loss	×	√	99.5	96.6	95.4	72.9	92.1	86.9	90.6
Ours	√	√	99.5	96.8	95.6	82.7	94.8	90.2	93.3

表4 钢材数据集不同方法的检测结果

Table 4 Results of different methods for the steel dataset

Model	AP/%						mAP/%	FPS
Model	夹杂(824)	斑块(689)	开裂(527)	轧制氧化皮(456)	划痕(390)	点蚀表面(295)	mAP/%	FPS
Faster- RCNN	80.4	87.9	49.8	66.1	94.1	72.6	75.2	12.08
YOLOv5s	73.8	87.5	53.1	58.7	91.2	78.8	73.9	114.94
YOLOv5m	83.6	87.9	52.9	68.4	91.8	78.0	77.1	43.67
YOLOv5s+CBL	81.6	86.2	45.4	59.5	91.1	73.3	72.9	114.94
YOLOv7	82.8	86.9	49.8	66.0	93.7	79.2	76.4	42.37
YOLOv8s	81.3	91.2	49.7	65.5	89.4	79.4	76.1	89.29
YOLOv8m	85.8	92.9	45.6	64.5	93.8	80.4	77.2	38.17
YOLOv8s+EFL	77.7	86.2	44.7	52.3	89.3	79.4	71.8	90.09
YOLOv10s	80.2	88.3	57.8	59.6	93.0	80.5	76.6	82.64
Ours	84.0	86.4	59.3	68.8	92.3	81.5	78.7	94.34

参考文献 25

[1]	罗东亮, 蔡雨萱, 杨子豪, 等. 工业缺陷检测深度学习方法综述[J]. 中国科学: 信息科学, 2022, 52(6): 1002-1039.
	LUO D L, CAI Y X, YANG Z H, et al. Survey on industrial defect detection with deep learning[J]. SCIENTIA SINICA Informationis, 2022, 52(6): 1002-1039 (in Chinese).
[2]	疏义桂. 基于机器视觉的铝塑泡罩包装药品缺陷检测[D]. 武汉: 华中科技大学, 2013.
	SHU Y G. Research on detection of medicines in aluminum- plastic blister package based on machine vision[D]. Wuhan: Huazhong University of Science and Technology, 2013 (in Chinese).
[3]	谷紫颖. 铝塑泡罩药品缺陷检测技术的研究[D]. 济南: 山东大学, 2020.
	GU Z Y. Research on defect detection technology of aluminum- plastic blister medicine[D]. Jinan: Shandong University, 2020 (in Chinese).
[4]	邵延华, 张铎, 楚红雨, 等. 基于深度学习的YOLO目标检测综述[J]. 电子与信息学报, 2022, 44(10): 3697-3708.
	SHAO Y H, ZHANG D, CHU H Y, et al. A review of YOLO object detection based on deep learning[J]. Journal of Electronics & Information Technology, 2022, 44(10): 3697-3708 (in Chinese).
[5]	李炳臻, 姜文志, 顾佼佼, 等. 基于卷积神经网络的目标检测算法综述[J]. 计算机与数字工程, 2022, 50(5): 1010-1017.
	LI B Z, JIANG W Z, GU J J, et al. Review of target detection algorithms based on deep learning[J]. Computer & Digital Engineering, 2022, 50(5): 1010-1017 (in Chinese).
[6]	LI H L, LI J, WEI H B, et al. Slim-neck by GSConv: a lightweight-design for real-time detector architectures[J]. Journal of Real-Time Image Processing, 2024, 21(3): 63.
[7]	王素琴, 任琪, 石敏, 等. 基于异常检测的产品表面缺陷检测与分割[J]. 图学学报, 2022, 43(3): 377-386.
	WANG S Q, REN Q, SHI M, et al. Product surface defect detection and segmentation based on anomaly detection[J]. Journal of Graphics, 2022, 43(3): 377-386 (in Chinese). DOI
[8]	张玥, 陈锡伟, 陈梦丹, 等. 基于对比学习生成对抗网络的无监督工业品表面异常检测[J]. 电子测量与仪器学报, 2023, 37(10): 193-201.
	ZHANG Y, CHEN X W, CHEN M D, et al. Unsupervised surface anomaly detection of industrial products based on contrastive learning generative adversarial network[J]. Journal of Electronic Measurement and Instrumentation, 2023, 37(10): 193-201 (in Chinese).
[9]	杜娟, 杨钧植. 基于迁移学习的小样本连接器缺陷检测方法[J]. 自动化与信息工程, 2022, 43(5): 1-7.
	DU J, YANG J Z. Small-sample connector defect detection method based on transfer learning[J]. Automation & Information Engineering, 2022, 43(5): 1-7 (in Chinese).
[10]	翟永杰, 胡哲东, 白云山, 等. 融合迁移学习的绝缘子缺陷分级检测方法[J]. 电子测量技术, 2023, 46(6): 23-30.
	ZHAI Y J, HU Z D, BAI Y S, et al. Integrating transfer learning for insulator defect grading detection[J]. Electronic Measurement Technology, 2023, 46(6): 23-30 (in Chinese).
[11]	XIAO W W, SONG K C, LIU J, et al. Graph embedding and optimal transport for few-shot classification of metal surface defect[J]. IEEE Transactions on Instrumentation and Measurement, 2022, 71: 5010310.
[12]	丁鹏, 卢文壮, 刘杰, 等. 基于生成对抗网络的叶片表面缺陷图像数据增强[J]. 组合机床与自动化加工技术, 2022(7): 18-21. DOI
	DING P, LU W Z, LIU J, et al. Image data augmentation of blade surface defects based on generative adversarial network[J]. Modular Machine Tool & Automatic Manufacturing Technique, 2022(7): 18-21 (in Chinese).
[13]	李可, 祁阳, 宿磊, 等. 基于改进ACGAN的钢表面缺陷视觉检测方法[J]. 机械工程学报, 2022, 58(24): 32-40. DOI
	LI K, QI Y, SU L, et al. Visual inspection of steel surface defects based on improved auxiliary classification generation adversarial network[J]. Journal of Mechanical Engineering, 2022, 58(24): 32-40 (in Chinese). DOI
[14]	JOHNSON J M, KHOSHGOFTAAR T M. Survey on deep learning with class imbalance[J]. Journal of Big Data, 2019, 6: 27.
[15]	WANG T, LI Y, KANG B Y, et al. The devil is in classification: a simple framework for long-tail instance segmentation[C]// The 16th European Conference on Computer Vision. Cham: Springer, 2020: 728-744.
[16]	ZHANG Y H, HUANG C, LOY C C. FASA: feature augmentation and sampling adaptation for long-tailed instance segmentation[C]// 2021 IEEE/CVF International Conference on Computer Vision. New York: IEEE Press, 2021: 3437-3446.
[17]	CUI Y, JIA M L, LIN T Y, et al. Class-balanced loss based on effective number of samples[C]// 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2019: 9260-9269.
[18]	LI B, YAO Y Q, TAN J R, et al. Equalized focal loss for dense long-tailed object detection[C]// 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2022: 6980-6989.
[19]	胡海涛, 杜昊晨, 王素琴, 等. 改进YOLOX的药品泡罩铝箔表面缺陷检测方法[J]. 图学学报, 2022, 43(5): 803-814.
	HU H T, DU H C, WANG S Q, et al. Improved YOLOX method for detecting surface defects of drug blister aluminum foil[J]. Journal of Graphics, 2022, 43(5): 803-814 (in Chinese).
[20]	王健, 肖迪, 冯李航, 等. 基于改进YOLOv8s的PCB小目标缺陷检测模型[EB/OL]. (024-08-20) [2024-09-08]http://kns.cnki.net/kcms/detail/11.2127.tp.20240819.1152.016.html.
	WANG J, XIAO D, FENG L H. A PCB small object defect detection model based on improved YOLOv8s[EB/OL]. (2024-08-20) [2024-09-08]http://kns.cnki.net/kcms/detail/11.2127.tp.20240819.1152.016.html (in Chinese).
[21]	李文举, 苏攀, 崔柳. 基于随机扰动的过拟合抑制算法[J]. 计算机仿真, 2022, 39(5): 134-138.
	LI W J, SU P, CUI L. Over-fitting suppression algorithm based on random perturbation[J]. Computer Simulation, 2022, 39(5): 134-138 (in Chinese).
[22]	LI X, WANG W H, WU L J, et al. Generalized focal loss: learning qualified and distributed bounding boxes for dense object detection[C]// The 34th International Conference on Neural Information Processing Systems. New York: ACM, 2020: 1763.
[23]	TAN J R, LU X, ZHANG G, et al. Equalization loss v2: a new gradient balance approach for long-tailed object detection[C]// 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2020: 1685-1694.
[24]	WANG C Y, BOCHKOVSKIY A, LIAO H Y M. YOLOv7: trainable bag-of-freebies sets new state-of-the-art for real-time object detectors[C]// 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2023: 7464-7475.
[25]	WANG A, CHEN H, LIU L H, et al. YOLOv10:real-time end-to-end object detection[EB/OL]. (2024-10-30) [2024-12-10]https://arxiv.org/abs/2405.14458.

类别不均衡的少样本工业产品表观缺陷检测

Detection of apparent defects in a small sample of industrial products with category imbalance

RichHTML

PDF (PC)

可视化

摘要/Abstract

引用本文

使用本文

图/表 14

参考文献 25

相关文章 0

编辑推荐

Metrics

本文评价