| 837 | 11 | 1214 |
| 下载次数 | 被引频次 | 阅读次数 |
在车联网中，合理分配频谱资源对满足不同车辆链路业务的服务质量(Quality of Service, QoS)需求具有重要意义。为解决车辆高速移动性和全局状态信息获取困难等问题，提出了一种基于完全分布式多智能体深度强化学习(Multi-Agent Deep Reinforcement Learning, MADRL)的资源分配算法。该算法在考虑车辆通信延迟和可靠性的情况下，通过优化频谱选择和功率分配策略来实现最大化网络吞吐量。引入共享经验池机制来解决多智能体并发学习导致的非平稳性问题。该算法基于深度Q网络(Deep Q Network, DQN),利用长短期记忆(Long Short Term Memory, LSTM)网络来捕捉和利用动态环境信息，以解决智能体的部分可观测性问题。将卷积神经网络(Convolutional Neural Network, CNN)和残差网络(Residual Network, ResNet)结合增强算法训练的准确性和预测能力。实验结果表明，所提出的算法能够满足车对基础设施(Vehicle-to-Infrastructure, V2I)链路的高吞吐量以及车对车(Vehicle-to-Vehicle, V2V)链路的低延迟要求，并且对环境变化表现出良好的适应性。
Abstract：In vehicular networks, the rational allocation of spectrum resources is of great importance in meeting the Quality of Service(QoS) requirements for diverse vehicular link services. To address challenges such as high vehicular mobility and difficulties in obtaining global state information, a resource allocation algorithm based on fully distributed Multi-Agent Deep Reinforcement Learning(MADRL) is proposed. With vehicle communication delays and reliability taken into account, the network throughput is maximized by optimizing spectrum selection and power allocation strategies. Firstly, a shared experience pool mechanism is introduced to tackle the non-stationarity issues caused by concurrent multi-agent learning. Secondly, dynamic environmental information is captured and utilized by a Deep Q Network(DQN) built upon Long Short Term Memory(LSTM) networks, addressing the challenge of partially observable environments for agents. Finally, he training accuracy and predictive capabilities of the algorithm are enhanced by integrating Convolutional Neural Network(CNN) and Residual Network(ResNet). Experimental results demonstrate that the proposed algorithm is capable of meeting the high throughput requirements of Vehicle-to-Infrastructure(V2I) links and low latency requirements of Vehicle-to-Vehicle(V2V) links while showing adaptability to changing environments.
[1] YADAV S,PANDEY A,DO D T,et al.Secure Cognitive Radio-enabled Vehicular Communications Under Spectrum-Sharing Constraints[J].Sensors,2021,21(21):7160.
[2] QI W J,SONG Q Y,GUO L,et al.Energy-efficient Resource Allocation for UAV-assisted Vehicular Networks with Spectrum Sharing[J].IEEE Transactions on Vehicular Technology,2022,71(7):7691-7702.
[3] CHEN L L,ZHAO Q J,FU K,et al.Multi-user Reinforcement Learning Based Multi-reward for Spectrum Access in Cognitive Vehicular Networks[J].Telecommunication Systems,2023,83(1):51-65.
[4] 方维维，王云鹏，张昊，等.基于多智能体深度强化学习的车联网通信资源分配优化[J].北京交通大学学报，2022,46(2):64-72.
[5] XIANG P,SHAN H G,WANG M,et al.Multi-agent RL Enables Decentralized Spectrum Access in Vehicular Networks[J].IEEE Transactions on Vehicular Technology,2021,70(10):10750-10762.
[6] ZHANG M L,DOU Y,CHONG P H J,et al.Fuzzy Logic-based Resource Allocation Algorithm for V2X Communications in 5G Cellular Networks[J].IEEE Journal on Selected Areas in Communications,2021,39(8):2501-2513.
[7] XIE Y C,YU K,TANG Z X,et al.An Effective Capacity Empowered Resource Allocation Approach in Low-latency C-V2X[C]//2022 14th International Conference on Wireless Communications and Signal Processing (WCSP).Nanjing:IEEE,2022:794-799.
[8] 赵莎莎.基于PSO的D2D蜂窝网络联合信道分配和功率控制[J].无线电工程，2023,53(7):1660-1669.
[9] LIANG L,YE H,YU G D,et al.Deep-learning-based Wireless Resource Allocation with Application to Vehicular Networks[J].Proceedings of the IEEE,2020,108(2):341-356.
[10] TAN J J,LIANG Y C,ZHANG L,et al.Deep Reinforcement Learning for Joint Channel Selection and Power Control in D2D Networks[J].IEEE Transactions on Wireless Communications,2020,20(2):1363-1378.
[11] YUAN Y,ZHENG G,WONG K K,et al.Meta-reinforcement Learning Based Resource Allocation for Dynamic V2X Communications[J].IEEE Transactions on Vehicular Technology,2021,70(9):8964-8977.
[12] GUAN Z,WANG Y Y,HE M.Deep Reinforcement Learning-based Spectrum Allocation Algorithm in Internet of Vehicles Discriminating Services[J].Applied Sciences,2022,12(3):1764.
[13] HAN D,SO J.Energy-efficient Resource Allocation Based on Deep Q-network in V2V Communications[J].Sensors,2023,23(3):1295.
[14] TIAN J,SHI Y,TONG X L,et al.Deep Reinforcement Learning Based Resource Allocation with Heterogeneous QoS for Cellular V2X[C]//2023 IEEE Wireless Communications and Networking Conference (WCNC).Glasgow:IEEE,2023:1-6.
[15] VU H V,FARZANULLAH M,LIU Z Y,et al.Multi-agent Reinforcement Learning for Channel Assignment and Power Allocation in Platoon-based C-V2X Systems[C]//2022 IEEE 95th Vehicular Technology Conference(VTC2022-Spring).Helsinki:IEEE,2022:1-5.
[16] 3GPP.Study LTE-based V2X Services (Release 14)[R].Valbonne:3GPP Support Office,2016.
[17] KY?STI P,MEINIL? J,HENTILA L,et al.WINNER II Channel Models[M].Hoboken:John Wiley & Sons,2008.
[18] LIANG L,YE H,LI G Y.Spectrum Sharing in Vehicular Networks Based on Multi-agent Reinforcement Learning[J].IEEE Journal on Selected Areas in Communications,2019,37(10):2282-2292.
基本信息:
中图分类号:TP18;TN929.5;U495
引用信息:
[1]孟水仙,刘艳超,王树彬.基于多智能体深度强化学习的车联网资源分配方法[J].无线电工程,2024,54(06):1388-1397.
基金信息:
国家自然科学基金(62361048)~~
2023-11-09
2023-11-09
2023-11-09