Generalization strategy design of UAVs pursuit evasion game based on DDPG

基于DDPG的无人机追捕任务泛化策略设计

School of Electronics and Information, Northwestern Polytechnical University, Xi′an 710129, China

Received: 11 June 2021

Abstract

UAVs pursuit evasion game is a research hotspot in the field of air combat. Traditional solutions have many limitations to this problem, such as the difficulty of the model to adapt to complex dynamic environments to quickly make decisions, and the poor generalization of different mission scenarios. Based on the DDPG(deep deterministic policy gradient) algorithm, a mathematical model of UAVs pursuit and evasion countermeasures is established in this paper. On this basis, this research designs a variety of countermaneuver strategies for escaping UAV, and uses the training method of course learning ideas. In the training process, the intelligence of the escaping UAV is gradually improved, so as to progressively train the confrontation strategy of the chasing UAV. The simulation results show that compared with direct training, the pursuit strategy of the chasing UAV trained by the research method of course learning can converge faster, and can better perform the hunting mission of enemy aircraft, and can be applied to a variety of enemy aircraft with a variety of maneuvering strategies, which effectively improved the generalization of the UAV′s pursuit and escape confrontation decision model.

摘要

无人机追逃对抗问题是当今空战领域的研究热点，传统解决方案对此问题存在诸多限制，如模型难以适应复杂动态环境从而快速做出决策、对不同任务场景泛化性较差等问题。基于DDPG(deep deterministic policy gradient)算法设计了无人机追逃对抗策略；在此基础上，设计多种逃逸无人机的对抗机动策略，利用课程学习思想，在DDPG的训练过程中逐步提高逃逸无人机的智能程度，从而递进式地训练追捕无人机的对抗策略。仿真结果表明，相较于直接进行训练，利用课程学习的方法所训练的追捕无人机的追捕策略能够更快收敛，并能更好地执行对敌机的追捕任务，且能够适用于具有多种对抗机动策略的敌机，有效地提升了无人机追逃对抗决策模型的泛化性。