Issue |
JNWPU
Volume 42, Number 2, April 2024
|
|
---|---|---|
Page(s) | 310 - 318 | |
DOI | https://doi.org/10.1051/jnwpu/20244220310 | |
Published online | 30 May 2024 |
Exploring UAV autonomous navigation algorithm based on soft actor-critic
基于SAC的无人机自主导航方法研究
School of Computer Science, Northwestern Polytechnical University, Xi'an 710072, China
Received:
27
March
2023
The existing deep reinforced learning algorithms cannot see local environments and have insufficient perceptual information on UAV autonomous navigation tasks. The paper investigates the UAV's autonomous navigation tasks in its unknown environments based on the nondeterministic policy soft actor-critic (SAC) reinforced learning model. Specifically, the paper proposes a policy network based on a memory enhancement mechanism, which integrates the historical memory information processing with current observations to extract the temporal dependency of the statements so as to enhance the state estimation ability under locally observable conditions and avoid the learning algorithm from falling into a locally optimal solution. In addition, a non-sparse reward function is designed to reduce the challenge of the reinforced learning strategy to converge under sparse reward conditions. Finally, several complex scenarios are trained and validated in the Airsim+UE4 simulation platform. The experimental results show that the proposed method has a navigation success rate 10% higher than that of the benchmark algorithm and that the average flight distance is 21% shorter, which effectively enhances the stability and convergence of the UAV autonomous navigation algorithm.
摘要
针对现有深度强化学习算法在无人机自主导航任务中面临环境局部可观且感知信息不足问题, 基于非确定性策略SAC(soft actor-critic)强化学习算法对未知环境下的端到端无人机自主导航任务展开研究。具体而言, 提出了一种基于记忆增强机制的策略网络, 通过对历史记忆信息与当前的观测整合处理, 提取观测数据的时序依赖关系, 从而增强局部可观条件下的状态估计能力, 避免算法陷入局部最优解; 设计了非稀疏奖励函数以缓解强化学习策略在稀疏奖励条件下难以收敛的问题; 在Airsim+UE4仿真平台进行了多个复杂场景的训练验证。实验结果表明, 所提方法导航成功率比基准算法提高10%, 平均飞行距离缩短21%, 有效增强了无人机自主导航算法稳定性和收敛性。
Key words: deep reinforced learning / soft actor-critic / unmanned aerial vehicle / autonomous navigation
关键字 : 强化学习 / SAC / 无人机 / 自主导航
© 2024 Journal of Northwestern Polytechnical University. All rights reserved.
This is an Open Access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
Current usage metrics show cumulative count of Article Views (full-text article views including HTML views, PDF and ePub downloads, according to the available data) and Abstracts Views on Vision4Press platform.
Data correspond to usage on the plateform after 2015. The current usage metrics is available 48-96 hours after online publication and is updated daily on week days.
Initial download of the metrics may take a while.