Path following method for AUV based on Q-Learning and RBF neural network

基于RBF网络Q学习的AUV路径跟踪控制方法

School of Marine Science and Technology, Northwestern Polytechnical University, Xi’an 710072, China

李乐(1986-), 西北工业大学助理教授, 主要从事水下机器人协同控制研究。e-mail: leli@nwpu.edu.cn

Received: 7 September 2020

Abstract

In the underwater docking process, the oscillation on AUV velocity brings extra challenge on AUV path following. A Q-learning based Sliding Mode Control (SMC) method to increase the path following performances is proposed. Firstly, AUV guidance law is designed to reduce the path following error. Heading and depth sliding mode controllers are designed to track the guidance law. Then, according to AUV velocity, tracking error and the first derivative, the control parameters of SMC are optimized via Q-learning network. RBF neural network is built to accelerate the offline learning rate. Finally, numerical simulations are made to investigate the characteristics of the present method. Comparisons are made between the trained Q-learning based SMC and the traditional SMC. The results show the effectiveness of the present method.

摘要

水下回收过程中，AUV航行速度受到多种因素影响而产生变化，艉部操纵舵效随之改变，直接影响了AUV回收路径跟踪控制性能。根据AUV航行状态，采用强化学习方法对AUV控制器进行自主学习优化，能够改善AUV航向及深度响应的性能指标，提高路径跟踪控制性能。建立AUV路径跟踪导引律，设计航向及俯仰运动滑模控制器，保证系统对外扰动的鲁棒性；采用Q学习方法，根据AUV航速、跟踪误差及其变化率，对滑模控制参数进行离线训练优化，搭建RBF网络加快训练过程，避免"维数灾"现象；将训练得到的RBF-Q学习网络应用于在线控制，与传统滑模控制器进行跟踪控制对比。仿真结果验证了算法的有效性。