无线电工程

近年交通拥堵已成为制约城市经济发展的重要问题，利用深度强化学习(Deep Reinforcement Learning, DRL)对交通信号灯进行自适应控制是缓解交通拥堵的研究热点。针对决斗双重深度Q网络(Dueling Double Deep Q-Network, D3QN)算法在交通信号控制中存在的样本利用率低、学习速度慢，以及路网状态信息复杂且灵活性差等问题，基于非均匀划分道路的离散交通状态编码(Discrete Traffic State Encode, DTSE)方法，提出一种D3PQN2交通信号控制算法。该算法在D3QN算法基础上引入噪声网络、优先级经验回放技术来提高样本的利用效率以及学习速度，通过噪声扰动代替传统的ε-贪婪策略，使得算法能够更快更好地收敛到全局最优解。以扬州市文昌路和扬子江路交叉口为例，在Weibull分布生成的车流下进行实验，结果表明，改进后的算法相较于对抗深度Q网络(Dueling Deep Q-Network, Dueling DQN)算法和固定配时的控制方法，车辆平均排队长度分别减少了12.11%和67.44%,累计延误时间分别减少了13.89%和42.88%,具有更好的控制效果。

In recent years, traffic congestion has become an important issue restricting the development of urban economy. Using Deep Reinforcement Learning(DRL) to adaptively control traffic lights has become a research hotspot to alleviate traffic congestion. To solve the problems of low sample utilization, slow learning speed, complex road network state information and poor flexibility in the Dueling Double Deep Q-Network(D3QN) algorithm in traffic signal control, a traffic signal control algorithm D3PQN2 is proposed based on the Discrete Traffic State Encode(DTSE) method for non-uniformly divided roads. Based on the D3QN algorithm, the algorithm introduces noise network and priority experience replay technology to improve the utilization efficiency and learning speed of samples, and replaces the traditional ε-greedy strategy with noise perturbation, so that the algorithm can converge to the global optimal solution faster and better. Taking the intersection of Wenchang Road and Yangtze River Road in Yangzhou as an example, experiments are carried out based on the traffic flow generated by Weibull distribution. The results show that compared with the Dueling Deep Q-Network(Dueling DQN) algorithm and the fixed timing control method, the average queue length of vehicles is reduced by 12.11% and 67.44%, and the cumulative delay time is reduced by 13.89% and 42.88%, respectively, proving that the method has better control effect.