无线电工程

随着人工智能等技术的发展，多智能体如无人机群等的实际应用领域逐渐广泛。多智能体深度确定性策略(Multi-Agent Deep Deterministic Policy Gradient, MADDPG)算法旨在解决多智能体在协作环境中的协同配合问题，凭借其独特的Actor-Critic架构已成为多智能体领域主流的应用算法之一。针对指挥决策中多智能体协同任务存在的角色分工模糊、信息过载导致的算法策略收敛较慢等问题，提出了一种引入动态角色注意力(Dynamic Role Attention, DRA)机制的改进MADDPG算法——DRA-MADDPG。该算法在Actor-Critic架构中嵌入了DRA模块，通过动态调整智能体对不同角色同伴的关注权重，来实现分工协作的精准优化。具体而言，定义了指挥任务的角色集合与阶段划分，进而构建角色协同矩阵和阶段调整系数；在Critic网络中设计DRA模块，依托角色相关性与任务阶段来计算权重并筛选关键信息；改进了Actor网络，结合角色职责生成针对性的动作。仿真实验表明，与MADDPG相比，DRA-MADDPG的训练累积回报曲线下面积(Area Under the Curve, AUC)提升了2.4%,任务完成耗时降低了19.3%,且通过训练回报曲线对比分析可知，DRA-MADDPG对于短期训练拥有更好的学习效率。证明了该方法适用于复杂指挥决策场景，为多智能体协同提供了一种相对高效的解决方案。

With the development of technologies such as artificial intelligence, multi-agents(e.g., unmanned aerial vehicle swarms) have been increasingly applied in practical combat operations. The Multi-Agent Deep Deterministic Policy Gradient(MADDPG) algorithm, designed to solve the coordination problems of multi-agents in cooperative environments, has become one of the mainstream applied algorithms in the multi-agent field owing to its unique Actor-Critic framework. To address the problems in multi-agent collaborative tasks during command and decision-making—including ambiguous role division and slow convergence of the algorithm's policy caused by information overload—an improved MADDPG algorithm incorporating a Dynamic Role Attention(DRA) mechanism, namely DRA-MADDPG, is proposed. This algorithm embeds a DRA module into the Actor-Critic framework, and achieves accurate optimization of division of labor and collaboration by dynamically adjusting the attention weights of each agent towards peers with different roles. Specifically, the role set(reconnaissance, assault, command) and phase division(exploration→execution→encirclement) for command tasks are defined, and on this basis, a role coordination matrix and phase adjustment coefficients are constructed. A DRA module is designed in the Critic network to calculate weights and filter key information by leveraging role relevance and task phases. Additionally, the Actor network is improved to generate targeted actions by integrating role responsibilities. Simulation experiments show that compared with MADDPG, the Area Under the Curve(AUC) of the cumulative training reward of DRA-MADDPG increases by 2.4%, and the task completion time decreases by 19.3%. Furthermore, comparative analysis of training reward curves reveals that DRA-MADDPG exhibits better learning efficiency in short-term training. It is demonstrated that this method is suitable for complex command and decision-making scenarios and provides a relatively efficient solution for multi-agent coordination.