Generation method for adversarial strategies in underwater incomplete information games

水下不完全信息博弈对抗策略的生成方法

¹ Shanghai Marine Electronic Equipment Research Institute, Shanghai 201108, China
² School of Marine Science and Technology, Northwestern Polytechnical University, Xi'an 710072, China
³ School of Mechanical Engineering, Shanghai Jiao Tong University, Shanghai 200240, China

Received: 16 January 2025

Abstract

The Nash differential strategy is suitable for complete information scenarios, but in the underwater environment with incomplete information, the complexity of obtaining solutions and the associated computational cost increase significantly. Therefore, the paper proposes a generation method to seek the solutions for the game adversarial strategy in the underwater environment. First, the relative motion relationship in a bilateral pursuit-evasion game was established, and the incomplete information characteristics were described using discrete and non-uniform time sequences, thus providing a foundation for constructing the expressions of the strategy. Secondly, based on the separation theorem of the secondary payment function and its minimum and maximum principles, the forms of the strategy within discrete time intervals were derived, and the strategy expressions for state estimation is presented. Finally, the Markov method was employed to construct the state transfer payment function, and key state transfer nodes were selected using threshold conditions, effectively reducing strategy update frequency and achieving efficient approximate solutions. The simulation results demonstrate that the proposed generation method exhibits good game effects in the underwater environment with incomplete information, which can quickly generate adversarial strategies.

摘要

纳什微分对策适用于完全信息博弈, 在水下环境不完全信息博弈时, 求解难度与计算成本显著增加。针对水下不完全信息博弈对抗策略的解析, 开展了策略的生成方法研究。建立双边追逃博弈的相对运动关系, 通过离散、非均匀的时间序列描述信息不完全特性, 为对策表达式构建打下基础。基于二次型支付函数的分离定理与极小极大原理, 推导离散时间间隔内的对策形式, 给出了状态估计的策略表达式。提出了基于马尔科夫方法的状态转移支付函数, 通过阈值条件确定关键状态转移节点, 减少策略更新频率, 实现高效的策略近似求解。仿真试验验证了文中方法在水下不完全信息条件下的良好博弈效果, 有效提升了博弈对抗策略的生成速度。

1. SINGH S KREDDY P V. Dynamic network analysis of a target defense differential game with limited observations[J]. IEEE Trans on Control of Network Systems, 2022, 10(1): 308–320 [Google Scholar]
2. WANG ZhongWEN ZhiwenCAI Weijun, et al. Research on game strategy of underwater attack and defense process in typical situation[J]. Journal of Northwestern Polytechnical University, 2023, 41(4): 774–783 [Article] (in Chinese) [Google Scholar]
3. WANG ZiyaoTANG ShengjingGUO Jie, et al. Adaptive 3-dimensional differential game guidance for hypersonic attack and defense[J]. Acta Armamentarii, 2023, 44(8): 2342–2353 (in Chinese) [Google Scholar]
4. WANG QiLIAO Zhizhong. Computational intelligence game guidance law based on online adaptive dynamic programming[J]. Aerospace Control, 2022, 40(6): 39–45 (in Chinese) [Google Scholar]
5. WEI NaLIU Mingyong. Target allocation decision of incomplete information game based on Bayesian Nash equilibrium[J]. Journal of Northwestern Polytechnical University, 2022, 40(4): 755–763 [Article] (in Chinese) [Google Scholar]
6. YE DSHI MSUN Z. Satellite proximate interception vector guidance based on differential games[J]. Chinese Journal of Aeronautics, 2018, 31(6): 1352–1361 [Article] [Google Scholar]
7. SUN BZENG Y RSU Z N. Task allocation in multi-AUV dynamic games based on interval ranking under uncertain information[J]. Ocean Engineering, 2023, 288(1): 116057 [Google Scholar]
8. TANG XuYE DongXIAO Yanet al. Epsilon Nash equilibrium differential game strategy for spacecraft terminal pursuit-evasion under incomplete information[J]. Journal of Astronautics, 2024, 45(1): 63–73 (in Chinese) [Google Scholar]
9. LI Z YZHU HLUO Y Z. An escape strategy in orbital pursuit-evasion games with incomplete information[J]. Science China Technological Sciences, 2021, 63(3): 559–570 [Google Scholar]
10. CAVALIERI K A, SATAK N, JURTADO J E. Incomplete information pursuit-evasion games with uncertain relative dynamics[C]//AIAA Guidance, Navigation, and Control Conference, National Harbor, USA, 2014: 13-17 [Google Scholar]
11. SATAK N. Behavior learning in differential games and reorientation maneuvers[D]. College Station: Texas A & M University, 2013 [Google Scholar]
12. CHENG Ruifeng. Interception and tracking of underwater maneuvering target based on differential games guidance[D]. Xi'an: Northwestern Polytechnical University, 2019 (in Chinese) [Google Scholar]
13. CHENG RuifengLIU WeidongGAO Li'e, et al. Underwater active defense intercept guidance method based on differential game[J]. Journal of Northwestern Polytechnical University, 2016, 34(5): 851–856 [Article] (in Chinese) [Google Scholar]
14. SUN J, YONG J. Linear-quadratic stochastic two-person nonzero-sum differential games: open-loop and closed-loop Nash equilibria[J]. Stochastic Processes and their Applications, 2019, 129(2): 381–418 [Article] [Google Scholar]
15. ISAACS R. Differential games[M]. New York: John Wiley & Sons, 1965 [Google Scholar]
16. MAITY D, BARAS J S. Optimal strategies for stochastic linear quadratic differential games with costly information[C]//2016 IEEE 55th Conference on Decision and Control, 2016: 276-282 [Google Scholar]
17. AGGARWAL S, ZAMAN M, BASTOPCU M, et al. Weighted age of information-based scheduling for large population games on networks[J]. IEEE Journal on Selected Areas in Information Theory, 2023, 4: 682–697 [Article] [Google Scholar]
18. NIU Liyuan. Research on confronting policy generation method of multi-agent system based on reinforcement learning[D]. Changsha: National University of Defense Technology, 2022 (in Chinese) [Google Scholar]