基于改进YOLOv5的智能除草机器人蔬菜苗田杂草检测研究

doi:10.11996/JG.j.2095-302X.2023020346

图学学报 ›› 2023, Vol. 44 ›› Issue (2): 346-356.DOI: 10.11996/JG.j.2095-302X.2023020346

• 图像处理与计算机视觉 • 上一篇下一篇

基于改进YOLOv5的智能除草机器人蔬菜苗田杂草检测研究

张伟康(), 孙浩, 陈鑫凯, 李叙兵, 姚立纲, 东辉()

福州大学机械工程及自动化学院，福建福州 350108

收稿日期:2022-07-11 接受日期:2022-09-07 出版日期:2023-04-30 发布日期:2023-05-01
通讯作者: 东辉(1985-)，女，教授，博士。主要研究方向为图像处理及机器学习方法等。E-mail：hdong@fzu.edu.cn
作者简介:张伟康(1997-)，男，硕士研究生。主要研究方向为机器人技术及目标检测。E-mail：wkzhang7167@163.com
基金资助:
国家自然科学基金项目(62173093);福建省自然科学基金项目(2020J01456)

Research on weed detection in vegetable seedling fields based on the improved YOLOv5 intelligent weeding robot

ZHANG Wei-kang(), SUN Hao, CHEN Xin-kai, LI Xu-bing, YAO Li-gang, DONG Hui()

School of Mechanical Engineering and Automation, Fuzhou University, Fuzhou Fujian 350108, China

Received:2022-07-11 Accepted:2022-09-07 Online:2023-04-30 Published:2023-05-01
Contact: DONG Hui (1985-), professor, Ph.D. Her main research interests cover image processing and machine learning, etc. E-mail：hdong@fzu.edu.cn
About author:ZHANG Wei-kang (1997-), master student. His main research interests cover robotics and object detection. E-mail：wkzhang7167@163.com
Supported by:
National Natural Science Foundation of China(62173093);National Natural Science Foundation of Fujian Province(2020J01456)

摘要/Abstract

摘要：

杂草精准检测是自动化除草装备的关键技术。针对田间杂草分布复杂和种类繁多导致的检测复杂度高和鲁棒性差等问题，基于自研移动机器人平台，提出一种改进YOLOv5算法和图像处理的蔬菜苗田杂草检测方法。通过识别蔬菜间接检测杂草的方式降低杂草检测复杂度，进而提高检测精度和鲁棒性。在YOLOv5目标检测算法主干特征提取网络中引入卷积块注意力模块(CBAM)提高网络对蔬菜目标的关注度，加入Transformer模块增强模型对全局信息的捕捉能力。结果表明，改进YOLOv5算法对蔬菜目标的平均检测准确率可达95.7%，与Faster R-CNN，SSD，EfficientDet，RetinaNet，YOLOv3，YOLOv4和YOLOv5算法相比，分别提高了5.8%，6.9%，10.3%，13.1%，9.0%，5.2%和3.2%。算法单幅图像平均检测时间11 ms，具有较好的实时性。采用改进YOLOv5算法检测蔬菜，将蔬菜边框之外绿色植物定义为杂草，超绿特征(ExG)结合OTSU阈值分割法将杂草与土壤背景分割，最后标记杂草连通域输出杂草质心和检测框。本研究方法可为农业自动化精准除草提供借鉴。

关键词: 除草机器人, 杂草检测, 蔬菜识别, YOLOv5, 注意力机制

Abstract:

Accurate detection of weeds is a key technology for developing automated weeding equipment. To address the problems of high detection complexity and poor robustness resulting from the complex distribution and variety of weeds, we proposed a weed detection approach for vegetable seedling based on the improved YOLOv5 algorithm and image processing, implemented on a self-developed mobile robot platform. The weed detection complexity was reduced by indirectly detecting weeds through identifying vegetables, thus improving the detection accuracy and robustness. The convolutional block attention module (CBAM) attention module was added to the backbone feature extraction network of the YOLOv5 object detection algorithm to enhance the focus of the network on vegetable targets, and the Transformer module was added to enhance the global information capture capability. The results showed that the average detection accuracy of the improved YOLOv5 algorithm for vegetable targets could reach 95.7%, which was increased by 5.8%, 6.9%, 10.3%, 13.1%, 9.0%, 5.2%, and 3.2% compared with Faster R-CNN, SSD, EfficientDet, RetinaNet, YOLOv3, YOLOv4, and YOLOv5, respectively. The average detection time of the algorithm for a single run was 11 ms, indicating good real-time performance. The method defined green plants outside the vegetable border as weeds, and combined the extreme green (ExG) with the OTSU threshold segmentation method to segment weeds from the soil background. Finally, the weed connectivity domain was marked, followed by outputting the weed plasmids and detection frames. The proposed method could provide a technical reference for automated precision weeding in agriculture.

Key words: weeding robot, weed detection, vegetable identification, YOLOv5, attention mechanism

中图分类号:

TP391

张伟康, 孙浩, 陈鑫凯, 李叙兵, 姚立纲, 东辉. 基于改进YOLOv5的智能除草机器人蔬菜苗田杂草检测研究[J]. 图学学报, 2023, 44(2): 346-356.

ZHANG Wei-kang, SUN Hao, CHEN Xin-kai, LI Xu-bing, YAO Li-gang, DONG Hui. Research on weed detection in vegetable seedling fields based on the improved YOLOv5 intelligent weeding robot[J]. Journal of Graphics, 2023, 44(2): 346-356.

图/表 19

图1 除草机器人

Fig. 1 Weeding robot

图2 三自由度执行机构

Fig. 2 Structure diagram of robot system

图3 机器人工作流程图

Fig. 3 Robot working process chart

图4 杂草检测流程

Fig. 4 Weed detection process

图5 YOLOv5-CBTR算法网络结构

Fig. 5 YOLOv5-CBTR algorithm network structure

图6 CBAM注意力网络结构

Fig. 6 CBAM attention network structure

图7 Transformer编码器结构

Fig. 7 Transformer encoder structure

图8 杂草检测示意图

Fig. 8 Weed detection schematic

图9 数据采集环境((a)温室大棚；(b)蔬菜生长状况)

Fig. 9 Data acquisition environment ((a) Greenhouse; (b) Vegetable growing conditions)

图10 作物与杂草分布情况((a)杂草与作物伴生；(b)杂草分布密集；(c)杂草分布稀疏；(d)杂草远离作物)

Fig. 10 Crop and weed distribution ((a) Weed grow with crop; (b) Dense weed distribution; (c) Sparse weed distribution; (d) Weed away from crop)

图11 数据增强示例((a)原图；(b)翻转；(c)颜色增强；(d)高斯噪声)

Fig. 11 Data augmentation example ((a) Original; (b) Flip; (c) Colour enhancement; (d) Gaussian noise)

表1 实验平台配置

Table 1 Experiment platform configuration

配置名称	版本参数
操作系统	Ubuntu18.04
CPU	Intel(R)Core(TM)i9-10920X
内存	32 G
GPU	NVIDIA GeForce RTX3070
深度学习框架	Pytorch1.8.0

图12 模型训练曲线((a)损失函数曲线；(b) PR曲线)

Fig. 12 Model training curve ((a) Loss function curve; (b) PR curve)

图13 mAP曲线对比

Fig. 13 mAP curve comparison

表2 不同算法性能指标对比结果

Table 2 Comparison results of performance indicators of different algorithms

Model	P (%)	R (%)	mAP (%)	参数量(MB)	时间(ms·fps^-1)
Faster R-CNN	92.5	79.4	89.9	113.6	73
SSD	92.1	80.9	88.8	97.1	35
EfficientDet	93.6	77.4	85.4	20.8	44
RetinaNet	93.5	74.8	82.6	146.0	52
YOLOv3	90.4	78.9	86.7	246.5	38
YOLOv4	91.4	83.3	90.5	256.3	33
YOLOv5	90.3	87.1	92.5	14.4	9
YOLOv5-CBTR	94.5	93.1	95.7	16.1	11

表3 消融实验结果

Table 3 Ablation experimental results

Model	CBAM	Transformer	P (%)	R (%)	mAP (%)
YOLOv5	-	-	90.3	87.1	92.5
	√	-	93.8	91.3	94.2
	-	√	92.6	90.7	93.8
	√	√	94.5	93.1	95.7

图14 检测结果对比((a)原图；(b) Faster R-CNN；(c) SSD；(d) YOLOv5；(e) YOLOv5-CBTR)

Fig. 14 Comparison of test results ((a) Original; (b) Faster R-CNN; (c) SSD; (d) YOLOv5; (e) YOLOv5-CBTR)

图15 杂草检测结果((a)蔬菜检测结果；(b)杂草分割结果)

Fig. 15 Weed detection results ((a) Vegetable detection results; (b) Weed segmentation results)

图16 除草试验

Fig. 16 Weeding experiment

参考文献 28

[1]	KHAN S, TUFAIL M, KHAN M T, et al. Deep learning-based identification system of weeds and crops in strawberry and pea fields for a precision agriculture sprayer[J]. Precision Agriculture, 2021, 22(6): 1711-1727. DOI
[2]	姜红花, 张传银, 张昭, 等. 基于Mask R-CNN的玉米田间杂草检测方法[J]. 农业机械学报, 2020, 51(6): 220-228, 247.
	JIANG H H, ZHANG C Y, ZHANG Z, et al. Detection method of corn weed based on mask R-CNN[J]. Transactions of the Chinese Society for Agricultural Machinery, 2020, 51(6): 220-228, 247. (in Chinese)
[3]	COLEMAN G, SALTER W, WALSH M. OpenWeedLocator (OWL): an open-source, low-cost device for fallow weed detection[J]. Scientific Reports, 2022, 12: 170. DOI PMID
[4]	MCCOOL C, BEATTIE J, FIRN J, et al. Efficacy of mechanical weeding tools: a study into alternative weed management strategies enabled by robotics[J]. IEEE Robotics and Automation Letters, 2018, 3(2): 1184-1190.
[5]	LI Y, GUO Z Q, SHUANG F, et al. Key technologies of machine vision for weeding robots: a review and benchmark[J]. Computers and Electronics in Agriculture, 2022, 196: 106880. DOI URL
[6]	孟庆宽, 张漫, 杨晓霞, 等. 基于轻量卷积结合特征信息融合的玉米幼苗与杂草识别[J]. 农业机械学报, 2020, 51(12): 238-245, 303.
	MENG Q K, ZHANG M, YANG X X, et al. Recognition of maize seedling and weed based on light weight convolution and feature fusion[J]. Transactions of the Chinese Society for Agricultural Machinery, 2020, 51(12): 238-245, 303. (in Chinese)
[7]	NGUYEN THANH LE V, APOPEI B, ALAMEH K. Effective plant discrimination based on the combination of local binary pattern operators and multiclass support vector machine methods[J]. Information Processing in Agriculture, 2019, 6(1): 116-131. DOI URL
[8]	CHEN Y J, WU Z N, ZHAO B, et al. Weed and corn seedling detection in field based on multi feature fusion and support vector machine[J]. Sensors: Basel, Switzerland, 2020, 21(1): 212.
[9]	温德圣, 许燕, 周建平, 等. 自然光照影响下基于深度卷积神经网络和颜色迁移的杂草识别方法[J]. 中国科技论文, 2020, 15(3): 287-292.
	WEN D S, XU Y, ZHOU J P, et al. Weed identification method based on deep convolutional neural network and color migration under the influence of natural illumination[J]. China Sciencepaper, 2020, 15(3): 287-292. (in Chinese)
[10]	TOO E C, LI Y J, NJUKI S, et al. A comparative study of fine-tuning deep learning models for plant disease identification[J]. Computers and Electronics in Agriculture, 2019, 161: 272-279. DOI URL
[11]	GIRSHICK R, DONAHUE J, DARRELL T, et al. Rich feature hierarchies for accurate object detection and semantic segmentation[C]// 2014 IEEE Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2014: 580-587.
[12]	REN S Q, HE K M, GIRSHICK R, et al. Faster R-CNN: towards real-time object detection with region proposal networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(6): 1137-1149. DOI PMID
[13]	LIU W, ANGUELOV D, ERHAN D, et al. SSD: single shot multibox detector[M]//Computer Vision - ECCV 2016. Cham: Springer International Publishing, 2016: 21-37.
[14]	TAN M X, PANG R M, LE Q V. EfficientDet: scalable and efficient object detection[C]// 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2020: 10778-10787.
[15]	LIN T Y, GOYAL P, GIRSHICK R, et al. Focal loss for dense object detection[C]// 2017 IEEE International Conference on Computer Vision. New York: IEEE Press, 2017: 2999-3007.
[16]	REDMON J, FARHADI A. YOLOv3: an incremental improvement[EB/OL]. (2018-04-08) [2022-05-02]. https://arxiv.org/abs/1804.02767.
[17]	BOCHKOVSKIY A, WANG C Y, LIAO H Y M. YOLOv4: optimal speed and accuracy of object detection[EB/OL]. (2020-04-23) [2022-05-02]. https://arxiv.org/abs/2004.10934.
[18]	YING B Y, XU Y C, ZHANG S, et al. Weed detection in images of carrot fields based on improved YOLOv4[J]. Traitement Du Signal, 2021, 38(2): 341-348. DOI URL
[19]	樊湘鹏, 周建平, 许燕, 等. 基于优化Faster R-CNN的棉花苗期杂草识别与定位[J]. 农业机械学报, 2021, 52(5): 26-34.
	FAN X P, ZHOU J P, XU Y, et al. Identification and localization of weeds based on optimized faster R-CNN in cotton seedling stage[J]. Transactions of the Chinese Society for Agricultural Machinery, 2021, 52(5): 26-34. (in Chinese)
[20]	REDMON J, DIVVALA S, GIRSHICK R, et al. You only look once: unified, real-time object detection[C]// 2016 IEEE Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2016: 779-788.
[21]	HE K M, ZHANG X Y, REN S Q, et al. Spatial pyramid pooling in deep convolutional networks for visual recognition[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 37(9): 1904-1916. DOI PMID
[22]	LIN T Y, DOLLAR P, GIRSHICK R, et al. Feature pyramid networks for object detection[C]// 2017 IEEE Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2017: 2117-2125.
[23]	REZATOFIGHI H, TSOI N, GWAK J Y, et al. Generalized intersection over union: a metric and a loss for bounding box regression[C]// 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2019: 658-666.
[24]	WOO S, PARK J, LEE J Y, et al. CBAM: convolutional block attention module[M]// Computer Vision-ECCV2018. Cham: Springer International Publishing, 2018: 3-19.
[25]	奉志强, 谢志军, 包正伟, 等. 基于改进YOLOv5的无人机实时密集小目标检测算法[J/OL]. 航空学报, 2022: 1-15. [2022- 05-11]. https://kns.cnki.net/kcms/detail/11.1929.V.20220509.2316.010.html.
	FENG Z Q, XIE Z J, BAO Z W, et al. Real-time dense small object detection algorithm for UAV based on improved YOLOv5[J/OL]. Acta Aeronautica et Astronautica Sinica, 2022: 1-15. [2022-05-11]. . (in Chinese)
[26]	DOSOVITSKIY A, BEYER L, KOLESNIKOV A, et al. An image is worth 16x16 words: transformers for image recognition at scale[EB/OL]. [2022-05-02]. https://arxiv.org/abs/2010.11929.
[27]	吴兰兰, 熊利荣, 彭辉. 基于RGB植被指数的大田油菜图像分割定量评价[J]. 华中农业大学学报, 2019, 38(2): 109-113.
	WU L L, XIONG L R, PENG H. Quantitative evaluation of in-field rapeseed image segmentation based on RGB vegetation indices[J]. Journal of Huazhong Agricultural University, 2019, 38(2): 109-113. (in Chinese)
[28]	ELSTONE L, HOW K Y, BRODIE S, et al. High speed crop and weed identification in lettuce fields for precision weeding[J]. Sensors: Basel, Switzerland, 2020, 20(2): 455.

基于改进YOLOv5的智能除草机器人蔬菜苗田杂草检测研究

Research on weed detection in vegetable seedling fields based on the improved YOLOv5 intelligent weeding robot

RichHTML

PDF (PC)

可视化

摘要/Abstract

引用本文

使用本文

图/表 19

参考文献 28

相关文章 15

编辑推荐

Metrics

本文评价

[1]	李利霞, 王鑫, 王军, 张又元 . 基于特征融合与注意力机制的无人机图像小目标检测算法 [J]. 图学学报, 2023, 44(4): 658-666.
[2]	郝帅, 赵新生, 马旭, 张旭, 何田, 侯李祥. 基于 TR-YOLOv5 的输电线路多类缺陷目标检测方法 [J]. 图学学报, 2023, 44(4): 667-676.
[3]	李鑫, 普园媛, 赵征鹏, 徐丹, 钱文华 . 内容语义和风格特征匹配一致的艺术风格迁移 [J]. 图学学报, 2023, 44(4): 699-709.
[4]	余伟群, 刘佳涛, 张亚萍. 融合注意力的拉普拉斯金字塔单目深度估计 [J]. 图学学报, 2023, 44(4): 728-738.
[5]	胡欣, 周运强, 肖剑, 杨杰. 基于改进YOLOv5的螺纹钢表面缺陷检测[J]. 图学学报, 2023, 44(3): 427-437.
[6]	毛爱坤, 刘昕明, 陈文壮, 宋绍楼. 改进YOLOv5算法的变电站仪表目标检测方法[J]. 图学学报, 2023, 44(3): 448-455.
[7]	郝鹏飞, 刘立群, 顾任远. YOLO-RD-Apple果园异源图像遮挡果实检测模型[J]. 图学学报, 2023, 44(3): 456-464.
[8]	罗文宇, 傅明月. 基于YoloX-ECA模型的非法野泳野钓现场监测技术[J]. 图学学报, 2023, 44(3): 465-472.
[9]	李雨, 闫甜甜, 周东生, 魏小鹏. 基于注意力机制与深度多尺度特征融合的自然场景文本检测[J]. 图学学报, 2023, 44(3): 473-481.
[10]	吴文欢, 张淏坤. 融合空间十字注意力与通道注意力的语义分割网络[J]. 图学学报, 2023, 44(3): 531-539.
[11]	谢国波, 贺笛轩, 何宇钦, 林志毅. 基于P-CenterNet的光学遥感图像烟囱检测[J]. 图学学报, 2023, 44(2): 233-240.
[12]	熊举举, 徐杨, 范润泽, 孙少聪. 基于轻量化视觉Transformer的花卉识别[J]. 图学学报, 2023, 44(2): 271-279.
[13]	陈刚, 张培基, 龚冬冬, 于俊清. 火电厂监控视频安全服检测方法研究[J]. 图学学报, 2023, 44(2): 291-297.
[14]	成浪, 敬超. 基于改进YOLOv7的X线图像旋转目标检测[J]. 图学学报, 2023, 44(2): 324-334.
[15]	曹义亲, 伍铭林, 徐露. 基于改进YOLOv5算法的钢材表面缺陷检测[J]. 图学学报, 2023, 44(2): 335-345.