基于残差和特征分块注意力的激光打码字符分割

doi:10.11996/JG.j.2095-302X.2023030482

图学学报 ›› 2023, Vol. 44 ›› Issue (3): 482-491.DOI: 10.11996/JG.j.2095-302X.2023030482

• 图像处理与计算机视觉 • 上一篇下一篇

基于残差和特征分块注意力的激光打码字符分割

肖天行¹^,²(), 吴静静¹^,²()

1.江南大学机械工程学院，江苏无锡 214122
2.江苏省食品先进制造装备技术重点实验室，江苏无锡 214122

收稿日期:2022-09-09 接受日期:2022-11-28 出版日期:2023-06-30 发布日期:2023-06-30
通讯作者: 吴静静(1982-)，女，副教授，博士。主要研究方向为图像处理与模式识别等。E-mail：wjjlady720@jiangnan.edu.cn
作者简介:
肖天行(1998-)，男，硕士研究生。主要研究方向为图像处理、机器学习与模式识别。E-mail：6200810103@stu.jiangnan.edu.cn
基金资助:
国家自然科学基金项目(62072416);国家自然科学基金项目(61873246)

Segmentation of laser coding characters based on residual and feature-grouped attention

XIAO Tian-xing¹^,²(), WU Jing-jing¹^,²()

1. School of Mechanical Engineering, Jiangnan University, Wuxi Jiangsu 214122, China
2. Jiangsu Key Laboratory of Advanced Food Manufacturing Equipment and Technology, Wuxi Jiangsu 214122, China

Received:2022-09-09 Accepted:2022-11-28 Online:2023-06-30 Published:2023-06-30
Contact: WU Jing-jing (1982-), associate professor, Ph.D. Her main research interests cover image processing, pattern recognition, etc. E-mail：wjjlady720@jiangnan.edu.cn
About author:
XIAO Tian-xing (1998-), master student. His main research interests cover image processing, machine learning and pattern recognition. E-mail：6200810103@stu.jiangnan.edu.cn
Supported by:
National Natural Science Foundation of China(62072416);National Natural Science Foundation of China(61873246)

摘要/Abstract

摘要：

金属表面激光打码工艺易造成周围金属变性，产生灼伤等形式的大量噪声，导致字符区域背景复杂，字符对比度低及模糊问题，给后续字符识别带来困难。因此提出一种基于残差和特征分块注意力的激光打码字符特征增强与精细分割模型Res18-UNet，以突出字符信息，提高信噪比，从而有效分割目标。首先设计了注意力-残差特征提取单元，减少网络参数的同时避免网络退化，提高通道和空间的特征选择能力。其次提出特征分块注意力机制，加入了改进的特征分块空间注意力，增强微弱字符特征。此外，在上采样阶段设计了融合改进损失函数的多重监督模块，改善网络收敛能力，提高分割精度。在激光打码易拉罐罐底图像数据集上实验得到的mIoU系数、Dice系数和F1分数均优于原UNet，分别达到了0.801 0，0.889 5和0.903 5，预测速度是原UNet的2.6倍，为12.24张/秒。实验说明，该算法能够有效地对低对比度激光打码字符进行特征增强和高精度分割，且具有在嵌入式平台上部署运行的可行性与应用前景。

关键词: 激光打码字符, 图像分割, 空间注意力, 残差神经网络, 特征分块策略

Abstract:

Laser coding on metal surface can lead to denaturation of surrounding metal and generate a significant amount of noise in the form of burns. This results in complex backgrounds in the character region, low contrast, and ambiguity of characters, which can make subsequent character recognition challenging. In response, Res18-UNet, a novel laser coding character feature enhancement and fine segmentation model, was proposed. The proposed model was based on residual and feature-grouped attention to highlight character information and improve signal-to-noise ratio, thus effectively segmenting the target. Firstly, the A-R unit was designed to reduce network parameters, effectively avoid network degradation, and improve the feature selection ability in channels and spaces. Secondly, the feature-grouped attention mechanism was proposed, and the improved spatial attention was added to enhance weak character features. In addition, a deep supervision module integrating the improved loss function was designed in the upsampling stage to improve network convergence and enhance segmentation precision. According to the experiment on the image dataset of the can bottoms with laser coding, the proposed model outperformed the original UNet model in terms of mIoU, Dice coefficient, and F1 score. Specifically, the proposed model achieved 0.801 0, 0.889 5, and 0.903 5, respectively, and attained the prediction speed 2.6 times that of the original UNet at 12.24 images/s. Experiments have proven that this algorithm can effectively enhance the features of low contrast laser coded characters and segment them with high precision, and that it has the feasibility and application prospect of deployment and operation on embedded platforms.

Key words: laser coding characters, image segmentation, spatial attention mechanism, residual neural network, feature group strategy

中图分类号:

TP391

肖天行, 吴静静. 基于残差和特征分块注意力的激光打码字符分割[J]. 图学学报, 2023, 44(3): 482-491.

XIAO Tian-xing, WU Jing-jing. Segmentation of laser coding characters based on residual and feature-grouped attention[J]. Journal of Graphics, 2023, 44(3): 482-491.

图/表 14

图1 激光打码字符图像及现有分割结果((a)原图；(b)字符细节；(c)大津法分割结果；(d)原UNet分割结果)

Fig. 1 Laser coding character images and current segmentation results ((a) Original image; (b) Details; (c) OTSU result; (d) UNet result)

图2 本文算法流程框图

Fig. 2 Diagram of proposed algorithms

图3 Res18-UNet模型整体框架

Fig. 3 Overall structure of Res18-UNet model

图4 注意力-残差特征提取单元

Fig. 4 A-R unit

图5 特征分块注意力机制结构图

Fig. 5 Structure of feature-grouped attention

图6 多重监督模块

Fig. 6 DS module

表1 不同主干网络对性能影响

Table 1 Effect of different backbone on network

主干网络	mIoU	Dice	F1 Score	FPS
ResNet14	0.788 0	0.881 5	0.894 8	13.70
ResNet18	0.801 0	0.889 5	0.903 5	12.24
ResNet34	0.792 8	0.883 8	0.897 3	8.80

图7 Grad-CAM注意力分布热力图((a)原图；(b)无注意力；(c) CBAM；(d)特征分块注意力机制)

Fig. 7 Thermal map of attention distribution from Grad-CAM ((a) Origin; (b) No attention module; (c) CBAM; (d) Feature-grouped attention)

表2 注意力模块对网络性能的影响

Table 2 Effect of attention module on network

注意力模块	mIoU	Dice	F1 Score
None	0.785 0	0.878 6	0.889 5
CBAM	0.790 6	0.880 9	0.892 8
Proposed	0.801 0	0.889 5	0.903 5

表3 损失函数及多重监督对网络性能的影响

Table 3 Effect of loss function and deep supervision

方法	mIoU	Dice	F1 Score	收敛轮数
BCE Loss	0.795 9	0.884 7	0.897 4	-
BID Loss (1:1:1)	0.786 7	0.881 2	0.894 4	-
BID Loss (2:1:1)	0.800 4	0.890 1	0.903 2	102
BID Loss (2:1:1) +DS	0.801 0	0.889 5	0.903 5	56

图8 采用BID Loss (2:1:1)在有DS和无DS时的训练表现

Fig. 8 Train performance with/without DS using BIDLoss (2:1:1)

表4 不同分割方法性能对比

Table 4 Performance comparison of different segmentation algorithm

应用类型	方法	分割精度			算法效率
应用类型	方法	mIoU	Dice	F1 Score	Params(M)	FLOPs(G)	FPS
打码字符分割	大津双阈值法	0.058 3	0.101 5	0.113 8	-	-	48.01
	自适应阈值分割+连通域去噪^[3]	0.103 5	0.178 5	0.200 2	-	-	0.57
	高斯差分尺度空间+最大熵法^[4]	0.066 3	0.107 3	0.120 9	-	-	2.12
医学分割	UNet^[12]	0.779 9	0.873 3	0.886 2	17.27	751.54	4.70
	UNet++^[21]	0.756 4	0.850 0	0.861 9	9.16	653.20	5.19
	MSR^[9] + UNet	0.781 0	0.876 2	0.888 0	17.27	751.54	2.33
	RA-UNet^[19]	0.782 7	0.876 3	0.890 1	11.62	306.88	11.00
	改进型UNet^[22]	0.798 9	0.885 3	0.899 7	34.87	1570.91	2.99
	RV-GAN^[16]	0.799 8	0.884 1	0.898 3	14.34	289.29	11.12
	Res18-UNet (proposed)	0.801 0	0.889 5	0.903 5	13.38	100.80	12.24

图9 本文方法与其他分割算法结果对比图((a)原图；(b) GT；(c)大津双阈值法；(d)自适应阈值分割+连通域去噪；(e)高斯差分尺度空间+最大熵法；(f) UNet；(g) UNet++；(h) MSR+UNet；(i) RA-UNet；(j)改进型UNet；(k) RV-GAN；(l)本文方法)

Fig. 9 Comparison with other segmentation algorithms ((a) Origin; (b) GT; (c) OTSU double threshold; (d) Adaptive threshold+denoising; (e) Gaussian difference scale space+maximum entropy; (f) UNet; (g) UNet++; (h) MSR+UNet; (i) RA-UNet; (j) Advanced UNet; (k) RV-GAN; (l) Ours)

表5 嵌入式平台测试结果(FP16)

Table 5 Test results on embedded platform (FP16)

模型输入尺寸	mIoU	Dice	F1 Score	FPS
1280×960	0.799 3	0.886 0	0.900 4	5.67
640×480	0.801 5	0.872 0	0.891 2	21.38

参考文献 22

[1]	孙晓娜, 刘继超, 高国华. 基于视觉的乳品包装日期喷码缺陷检测技术[J]. 食品与机械, 2018, 34(10): 100-103, 108.
	SUN X N, LIU J C, GAO G H. Study on visual code-based defect detection technology for production date of dairy packaging[J]. Food & Machinery, 2018, 34(10): 100-103, 108. (in Chinese)
[2]	马玲, 罗晓曙, 蒋品群. 基于模板匹配和支持向量机的点阵字符识别研究[J]. 计算机工程与应用, 2020, 56(4): 134-139. DOI
	MA L, LUO X S, JIANG P Q. Research on dot matrix character recognition based on template matching and support vector machine[J]. Computer Engineering and Applications, 2020, 56(4): 134-139. (in Chinese) DOI
[3]	林冬婷, 程洋, 欧阳, 等. 基于喷点融合特征的点阵字符分割方法[J]. 制造业自动化, 2021, 43(8): 52-57.
	LIN D T, CHENG Y, OU Y, et al. Detection method of dot matrix character based on spray fusion feature[J]. Manufacturing Automation, 2021, 43(8): 52-57. (in Chinese)
[4]	张家财, 张良力, 曾飞. 钢坯表面点印字符图像自适应阈值分割方法[J]. 现代电子技术, 2021, 44(19): 49-54.
	ZHANG J C, ZHANG L L, ZENG F. Adaptive thresholding segmentation method for dot-printed character image on billet surface[J]. Modern Electronics Technique, 2021, 44(19): 49-54. (in Chinese)
[5]	汤勃, 孔建益, 王兴东, 等. 钢板表面低对比度微小缺陷图像增强和分割[J]. 中国图象图形学报, 2020, 25(1): 81-91.
	TANG B, KONG J Y, WANG X D, et al. Image enhancement and segmentation algorithm for low-contrast small defects on steel plate[J]. Journal of Image and Graphics, 2020, 25(1): 81-91. (in Chinese)
[6]	BARTHAKUR M, SARMA K K. Complex image Segmentation using K-means clustering aided neuro-computing[C]// The 5th International Conference on Signal Processing and Integrated Networks. New York: IEEE Press, 2018: 327-331.
[7]	李镇锋, 陈晓荣, 陈梦华, 等. 基于图像熵和傅里叶变换的复杂背景分割[J]. 软件工程, 2021, 24(11): 19-23.
	LI Z F, CHEN X R, CHEN M H, et al. Complex background segmentation based on image entropy and Fourier transform[J]. Software Engineering, 2021, 24(11): 19-23. (in Chinese)
[8]	王欣, 徐平平, 吴菲. 基于指数同态滤波耦合细节锐化规则的红外图像增强算法[J]. 电子测量与仪器学报, 2021, 35(10): 9-16.
	WANG X, XU P P, WU F. Infrared image enhancement algorithm based on exponential homomorphic filtering coupled with detail sharpening rule[J]. Journal of Electronic Measurement and Instrumentation, 2021, 35(10): 9-16. (in Chinese)
[9]	SHEN L, YUE Z H, FENG F, et al. MSR-net: low-light image enhancement using deep convolutional network[EB/OL]. (2017-11-07) [2022-06-15]. https://arxiv.org/abs/1711.02488.
[10]	QIN X B, FAN D P, HUANG C Y, et al. Boundary-aware segmentation network for mobile and web applications[EB/OL]. (2021-05-11) [2022-03-10]. https://arxiv.org/abs/2101.04704.
[11]	HE K M, ZHANG X Y, REN S Q, et al. Deep residual learning for image recognition[C]// 2016 IEEE Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2016: 770-778.
[12]	RONNEBERGER O, FISCHER P, BROX T. U-net: convolutional networks for biomedical image segmentation[M]// Lecture Notes in Computer Science. Cham: Springer International Publishing, 2015: 234-241.
[13]	JÉGOU S, DROZDZAL M, VAZQUEZ D, et al. The one hundred layers tiramisu: fully convolutional DenseNets for semantic segmentation[C]// 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops. New York: IEEE Press, 2017: 1175-1183.
[14]	UPADHYAY K, AGRAWAL M, VASHIST P. U-net based multi-level texture suppression for vessel segmentation in low contrast regions[C]// The 28th European Signal Processing Conference. New York: IEEE Press, 2021: 1304-1308.
[15]	董月, 冯华君, 徐之海, 等. Attention Res-Unet: 一种高效阴影检测算法[J]. 浙江大学学报: 工学版, 2019, 53(2): 373-381, 406.
	DONG Y, FENG H J, XU Z H, et al. Attention Res-Unet: an efficient shadow detection algorithm[J]. Journal of Zhejiang University: Engineering Science, 2019, 53(2): 373-381, 406. (in Chinese)
[16]	KAMRAN S A, HOSSAIN K F, TAVAKKOLI A, et al. RV-GAN: segmenting retinal vascular structure in fundus photographs using a novel multi-scale generative adversarial network[M]//Medical Image Computing and Computer Assisted Intervention - MICCAI 2021. Cham: Springer International Publishing, 2021: 34-44.
[17]	WOO S, PARK J, LEE J Y, et al. CBAM: convolutional block attention module[M]//Computer Vision - ECCV 2018. Cham: Springer International Publishing, 2018: 3-19.
[18]	蒋宏达, 叶西宁. 一种改进的I-Unet网络的皮肤病图像分割算法[J]. 现代电子技术, 2019, 42(12): 52-56.
	JIANG H D, YE X N. An improved skin disease image segmentation algorithm based on I-Unet network[J]. Modern Electronics Technique, 2019, 42(12): 52-56. (in Chinese)
[19]	侯向丹, 赵一浩, 刘洪普, 等. 融合残差注意力机制的UNet视盘分割[J]. 中国图象图形学报, 2020, 25(9): 1915-1929.
	HOU X D, ZHAO Y H, LIU H P, et al. Optic disk segmentation by combining UNet and residual attention mechanism[J]. Journal of Image and Graphics, 2020, 25(9): 1915-1929. (in Chinese)
[20]	SELVARAJU R R, COGSWELL M, DAS A, et al. Grad-CAM: visual explanations from deep networks via gradient-based localization[C]// 2017 IEEE International Conference on Computer Vision. New York: IEEE Press, 2017: 618-626.
[21]	ZHOU Z W, RAHMAN SIDDIQUEE M M, TAJBAKHSH N, et al. UNet++: A nested U-net architecture for medical image segmentation[M]//Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support. Cham: Springer International Publishing, 2018: 3-11.
[22]	钟文煜, 冯寿廷. 改进型Unet: 一种高效准确的视网膜血管分割方法[J]. 光学技术, 2019, 45(6): 744-748.
	ZHONG W Y, FENG S T. Improved Unet: an efficient and accurate retinal vessel segmentation method[J]. Optical Technique, 2019, 45(6): 744-748. (in Chinese)

基于残差和特征分块注意力的激光打码字符分割

Segmentation of laser coding characters based on residual and feature-grouped attention

RichHTML

PDF (PC)

可视化

摘要/Abstract

引用本文

使用本文

图/表 14

参考文献 22

相关文章 15

编辑推荐

Metrics

本文评价

[1]	吴文欢, 张淏坤. 融合空间十字注意力与通道注意力的语义分割网络[J]. 图学学报, 2023, 44(3): 531-539.
[2]	方洪波, 万广, 陈忠辉, 黄以卫, 张文勇, 谢本亮. 基于改进 YOLOv5s 的离线手写数学符号识别[J]. 图学学报, 2022, 43(3): 387-395.
[3]	张燕, 高鑫, 刘以, 张小峰, 张彩明. 基于改进像素相关性模型的图像分割算法[J]. 图学学报, 2022, 43(2): 205-213.
[4]	李翠云, 白静, 郑凉. 融合边缘增强注意力机制和 U-Net 网络的医学图像分割[J]. 图学学报, 2022, 43(2): 273-278.
[5]	陈宝玉, 张怡, 于冰冰, 刘秀平. 两阶段可调节感知蒸馏网络的虚拟试衣方法[J]. 图学学报, 2022, 43(2): 316-323.
[6]	温静, 丁友东, 于冰. 基于上下文门卷积的盲图像修复[J]. 图学学报, 2022, 43(1): 70-78.
[7]	秦宇幸, 羿旭明. 结合显著性和边缘信息的水平集图像分割方法[J]. 图学学报, 2021, 42(5): 738-743.
[8]	李彬 , 王平 , 赵思逸 . 基于双重注意力机制的图像超分辨重建算法[J]. 图学学报, 2021, 42(2): 206-215.
[9]	李晓慧, 汪西莉 . 结合目标局部和全局特征的 CV遥感图像分割模型[J]. 图学学报, 2020, 41(6): 905-916.
[10]	谷昱良，羿旭明 . 基于小波变换的权重自适应图像分割模型[J]. 图学学报, 2020, 41(5): 733-739.
[11]	黎智，徐丹 . 基于卷积神经网络的蜡染染色模拟[J]. 图学学报, 2020, 41(2): 196-203.
[12]	段杰1,2，崔志明1,2，沈艺1,2，冯威1，吴宏杰1，冯雨晴1,2. 一种改进FCN 的肝脏肿瘤CT 图像分割方法[J]. 图学学报, 2020, 41(1): 100-107.
[13]	魏晨晨，羿旭明 . 基于改进 DRLSE 水平集模型的图像分割[J]. 图学学报, 2019, 40(5): 885-891.
[14]	汪友生，王雨婷，夏章涛，叶红梅 . 序列血管内超声图像的管腔内膜自动分割算法[J]. 图学学报, 2019, 40(1): 173-180.
[15]	刘建新 1，曾嫱 1，徐可 2，王亚威 1 . 基于形态学和小波变换的烟叶病斑分割[J]. 图学学报, 2018, 39(5): 933-938.