Robotic arm reinforcement learning control method based on autonomous visual perception

基于强化学习的机械臂自主视觉感知控制方法

¹ School of Computer, Hubei University of Arts and Science, Xiangyang 441053, China
² School of Computer, Northwestern Polytechnical University, Xi'an 710129, China

Received: 3 June 2021

Abstract

The traditional robotic arm control methods are often based on artificially preset fixed trajectories to control them to complete specific tasks, which rely on accurate environmental models, and the control process lacks the ability of self-adaptability. Aiming at the above problems, we proposed an end-to-end robotic arm intelligent control method based on the combination of machine vision and reinforcement learning. The visual perception uses the YOLO algorithm, and the strategy control module uses the DDPG reinforcement learning algorithm, which enables the robotic arm to learn autonomous control strategies in a complex environment. Otherwise, we used imitation learning and hindsight experience replay algorithm during the training process, which accelerated the learning process of the robotic arm. The experimental results show that the algorithm can converge in a shorter time, and it has excellent performance in autonomously perceiving the target position and overall strategy control in the simulation environment.

摘要

传统机械臂控制方法按照人为预设固定轨迹来对其进行控制，完成特定的任务，依赖于精确的环境模型，并且控制过程缺乏一定的自适应性。为解决该问题，提出一种自主视觉感知与强化学习相结合的端到端机械臂智能控制方法。该方法中视觉感知使用YOLO算法，策略控制模块使用DDPG强化学习算法，使机械臂能够在复杂的环境中学习到自主控制策略，并且在训练过程使用了模仿学习与后视经验重播，加速了机械臂的学习过程。实验结果表明算法能够在更短的时间内收敛，并且在仿真环境中自主感知目标位置及整体策略控制都有着出色的表现。