欢迎访问《图学学报》 分享到：
图学学报

图学学报

• 专论：第16届媒体智能与大数据计算会议（CIDE & DEA 2019 大连） • 上一篇    下一篇

基于 Spark 的分布式机器人强化学习训练框架

  

  1. (1. 浙江大学智能系统与控制研究所，浙江 杭州 310027； 2. 淮北职业技术学院计算机科学技术系，安徽 淮北 235000； 3. 杭州电子科技大学计算机学院，浙江 杭州 310018； 4. 国家电网浙江省电力有限公司物资分公司，浙江 杭州 310000； 5. 重庆文理学院大数据智能计算与可视化研究所，重庆 402160)
  • 出版日期:2019-10-31 发布日期:2019-11-06
  • 基金资助:
    浙江大学工业控制技术国家重点实验室开放课题项目(ICT1800413)；重庆市发改委重大产业技术研发项目(2018148208)；重庆市教委科技 项目(KJ1601129)；安徽省高校自然科学研究重点项目(KJ2018A0713)；安徽高校优秀青年骨干人才国内访问研修项目(gxgnfx2018108)；广 东省重点领域研发计划项目(2019B010120001)

Training Framework of Distributed Robot Reinforcement Learning  Based on Spark

  1. (1. Institute of Cyber Systems and Control, Zhejiang University, Hangzhou Zhejiang 310027, China;  2. Department of Computer Science and Technology, Huaibei Vocational and Technical College, Huaibei Anhui 235000, China;  3. School of Computer Science, Hangzhou Dianzi University, Hangzhou Zhejiang 310018, China;  4. Materials Branch, State Grid Zhejiang Electric Power Company, LTD, Hangzhou Zhejiang 310000, China; 5. Institute of Intelligent Computing and Visualization Based on Big Data, Chongqing University of Arts and Sciences, Chongqing 402160, China)
  • Online:2019-10-31 Published:2019-11-06

PDF (PC)

112

摘要： 强化学习能够通过自主学习的方式对机器人难以利用控制方法实现的各种任务进行 训练完成，有效避免了系统设计人员对系统建模或制定规则。然而，强化学习在机器人开发应用 领域中训练成本高昂，需要花费大量时间成本、硬件成本实现学习训练，虽然基于仿真可以一定 程度减少硬件成本，但对类似 Gazebo 这样的复杂机器人训练平台，仿真过程工作效率低，数据 采样耗时长。为了有效解决这些问题，针对机器人仿真过程的平台易用性、兼容性等方面进行优 化，提出一种基于 Spark 的分布式强化学习框架，为强化学习的训练与机器人仿真采样提供分布 式支持，具有高兼容性、健壮性的特性。通过实验数据分析对比，表明本系统框架不仅可有效提 高机器人的强化学习模型训练速度，缩短训练时间花费，且有助于节约硬件成本。

关键词: 机器人, 强化学习, Spark, 分布式, 数据管道

Abstract: Through autonomous learning, reinforcement learning can train robots to complete various tasks that are difficult for them to implement with control methods, and this can effectively avoid system designers from systemic modeling or rules making. However, the training cost of reinforcement learning in the field of robot development and application is high, and it takes a large amount of time cost and hardware cost to realize learning and training. Although the hardware cost can be reduced to some extent based on simulation, for the complicated robot training platform such as Gazebo, the working efficiency of simulation process is low, and it takes a long time for data sampling. In order to effectively solve these problems, a distributed reinforcement learning framework based on Spark is put forward, which optimizes the usability and compatibility of platform of robot simulation process, offers distributed support for the training of reinforcement learning and robot simulation sampling, and has the characteristics of high compatibility and robustness. Through analyzing and contrasting the experimental data, the system framework can not only effectively improve the training speed of reinforcement learning model of robot and shorten the training time, but also help with the saving of hardware cost.

Key words: robot, reinforcement learning, Spark, distribute, data pipeline