| 1,462 | 4 | 118 |
| 下载次数 | 被引频次 | 阅读次数 |
随着无人机技术的深入发展，基于深度学习且以视觉输入的四旋翼无人机(四旋翼)自主飞行感知和避障研究备受该领域学者的关注，以无人机飞行模式朝着完全自主的终极目标不断迈进。无人机的自主感知和避障正是实现无人机自主飞行的关键所在。简要地阐述了目前无人机自治水平等级和相关深度学习方法；对四旋翼的仿真平台和公开数据集进行了较为全面的介绍；从无人机自主飞行感知和避障2个方面，对当前基于深度视觉的四旋翼自主飞行感知和避障相关国内外文献报道，进行了较为全面的分析和总结；结合深度学习方法和以视觉输入的四旋翼自主飞行感知和避障在一些关键的开放性问题上的不足，对其未来挑战和发展趋势进行了总结和展望，旨在为后续研究提供参考。
Abstract：With the in-depth development of UAV technology, the research on autonomous flight perception and obstacle avoidance of quadrotor UAVs(quadrotors) based on deep learning and with visual input has attracted much attention from scholars in this field, and has been moving toward the ultimate goal of full autonomy UAV flight mode. Among them, the autonomous perception and obstacle avoidance of UAVs is the key to realize the autonomous flight of UAVs. To this end, firstly the current level of UAV autonomy and related deep learning methods are briefly described; Secondly, a more comprehensive introduction to the simulation platform and public data set of quadrotors is provided; Then, a more comprehensive analysis and summary of the current domestic and foreign literature reports on quadrotors autonomous flight perception and obstacle avoidance based on deep learning and with visual input is provided from two aspects of UAV autonomous flight perception and obstacle avoidance. Finally, combining some problems of deep learning methods and quadrotors with visual input for autonomous flight perception and obstacle avoidance in some key open issues, the future challenges and development trends are summarized and prospected, aiming to provide reference for follow-up research.
[1] HILDMANN H,KOVACS E.Review:Using Unmanned Aerial Vehicles(UAVs) as Mobile Sensing Platforms(MSPs) for Disaster Response,Civil Security and Public Safety[J].Drones,2019,3(3):59.
[2] CHAMBERLAIN B,SHEIKH W.Design and Implementation of a Quadcopter Drone Control System for Photography Applications[C]//2022 Intermountain Engineering,Technology and Computing(IETC).Orem:IEEE,2022:1-7.
[3] DENG L,HE Y H,LIU Q Y.Research on Application of Fire Uumanned Aerial Vehicles in Emergency Rescue[C]∥2019 9th International Conference on Fire Science and Fire Protection Engineering(ICFSFPE).Chengdu:IEEE,2019:1-5.
[4] PSIROFONIA P,SAMARITAKIS V,ELIOPOULOS P,et al.Use of Unmanned Aerial Vehicles for Agricultural Application with Emphasis on Crop Protection:Three Novel Case-studies[J].Journal of Agricultural Science and Technology,2017,5(1):30-39.
[5] ZHU X N.Analysis of Military Application of UAV Swarm Technology[C]//2020 3rd International Conference on Unmanned Systems(ICUS).Harbin:IEEE,2020:1200-1204.
[6] SCHERER S,REHDER J,ACHAR S,et al.River Mapping from a Flying Robot:State Estimation,River Detection,and Obstacle Mapping[J].Autonomous Robots,2012,33(1-2):189-214.
[7] SHEN S J,MULGAONKAR Y,MICHAEL N,et al.Multisensor Fusion for Robust Autonomous Flight in Indoor and Outdoor Environments with a Rotorcraft MAV[C]//2014 IEEE International Conference on Robotics and Automation(ICRA).Hong Kong:IEEE,2014:4974-4981.
[8] GALVAN J,RAJA A,LI Y Y,et al.Sensor Data-driven UAV Anomaly Detection Using Deep Learning Approach[C]//MILCOM 2021-2021 IEEE Military Communications Conference (MILCOM).San Diego:IEEE,2021:589-594.
[9] KAUFMANN E,LOQUERCIO A,RANFTL R,et al.Deep Drone Racing:Learning Agile Flight in Dynamic Environments[J/OL].(2018-06-22)[2023-03-10].https://arxiv.org/abs/1806.08548.
[10] WANG S,CLARK R,WEN H K,et al.DeepVO:Towards End-to-End Visual Odometry with Deep Recurrent Convolutional Neural Networks[C]//2017 IEEE International Conference on Robotics and Automation (ICRA).Singapore:IEEE,2017:2043-2050.
[11] KENDOUL F.Survey of Advances in Guidance,Navigation,and Control of Unmanned Rotorcraft Systems[J].Journal of Field Robotics,2012,29(2):315-378.
[12] PRAMOD A,NAICKER H S,TYAGI A K.Machine Learning and Deep Learning:Open Issues and Future Research Directions for the Next 10 Years[M/OL].[S.l.]:John Wiley & Sons,Ltd.,2021:463-490[2023-03-04].https://onlinelibrary.wiley.com/doi/abs/10.1002/9781119785750.ch18.
[13] NGUYEN G,DLUGOLINSKY S,BOBáK M,et al.Machine Learning and Deep Learning frameworks and Libraries for Large-scale Data Mining:A Survey[J].Artificial Intelligence Review,2019,52(1):77-124.
[14] HASSABALLAH M,AWAD A I.Deep Learning in Computer Vision:Principles and Applications[M].Boca Raton:CRC Press,2020.
[15] OTTER D W,MEDINA J R,KALITA J K.A Survey of the Usages of Deep Learning for Natural Language Processing[J].IEEE Transactions on Neural Networks and Learning Systems,2021,32(2):604-624.
[16] MAIORANA E.Deep Learning for EEG-based Biometric Recognition[J].Neurocomputing,2020,410:374-386.
[17] AGGARWAL R,SOUNDERAJAH V,MARTIN G,et al.Diagnostic Accuracy of Deep Learning in Medical imaging:A Systematic Review and Meta-analysis[J].NPJ Digital Medicine,2021,4(1):65.
[18] O’SHEA K,NASH R.An Introduction to Convolutional Neural Networks[J/OL].(2015-12-02)[2023-03-10].https://arxiv.org/abs/1511.08458.
[19] ZAREMBA W,SUTSKEVER I,VINYALS O.Recurrent Neural Network Regularization[J/OL].(2015-02-19)[2023-03-10].https://arxiv.org/abs/1409.2329.
[20] KRIZHEVSKY A,SUTSKEVER I,HINTON G E.ImageNet Classification with Deep Convolutional Neural Networks[J].Communications of the ACM,2017,60(6):84-90.
[21] ZHAO L Q,LI S Y.Object Detection Algorithm Based on Improved YOLOv3[J].Electronics,2020,9(3):537.
[22] HE K M,ZHANG X Y,REN S Q,et al.Deep Residual Learning for Image Recognition[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition.Las Vegas:IEEE,2016:770-778.
[23] LOQUERCIO A,MAQUEDA A I,DEL-BLANCO C R,et al.DroNet:Learning to Fly by Driving[J].IEEE Robotics and Automation Letters,2018,3(2):1088-1095.
[24] SIMONYAN K,ZISSERMAN A.Very Deep Convolutional Networks for Large-scale Image Recognition[J].(2015-04-10)[2023-03-10].https://arxiv.org/abs/1409.1556.
[25] KENDALL A,GRIMES M,CIPOLLA R.PoseNet:A Convolutional Network for Real-time 6-DOF Camera Reloca-lization[J/OL].(2015-05-27)[2023-03-10].https://arxiv.org/abs/1505.07427.
[26] HUANG G,LIU Z,VAN DER MAATEN L,et al.Densely Connected Convolutional Networks[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition.Honolulu:IEEE,2017:4700-4708.
[27] KOENIG N,HOWARD A.Design and Use Paradigms for Gazebo,an Open-source Multi-robot Simulator[C]//2004 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).Sendai:IEEE,2004:2149-2154.
[28] SONG Y L,NAJI S,KAUFMANN E,et al.Flightmare:A Flexible Quadrotor Simulator[J/OL].(2020-09-01)[2023-03-10].https://arxiv.org/abs/2009.00563.
[29] SHAH S,DEY D,LOVETT C,et al.Airsim:High-fidelity Visual and Physical Simulation for Autonomous Vehicles[J/OL].(2017-05-15)[2023-03-10].https://arxiv.org/abs/1705.05065.
[30] GEIGER A,LENZ P,STILLER C,et al.Vision Meets Robotics:The Kitti Dataset[J].International Journal of Robotics Research,2013,32(11):1231-1237.
[31] GIUSTI A,GUZZI J,CIRE?AN D C,et al.A Machine Learning Approach to Visual Perception of Forest Trails for Mobile Robots[J].IEEE Robotics and Automation Letters,2016,1(2):661-667.
[32] GANDHI D,PINTO L,GUPTA A.Learning to Fly by Crashing[C]//2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).Vancouver:ACM,2017:3948-3955.
[33] Udacity.Self Driving Car Engineer [EB/OL].[2023-03-10].https://www.udacity.com/self-driving-car.
[34] ROS G,SELLART L,MATERZYNSKA J,et al.The Synthia Dataset:A Large Collection of Synthetic Images for Semantic Segmentation of Urban Scenes[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition.Las Vegas:IEEE,2016:3234-3243.
[35] SMOLYANSKIY N,KAMENEV A,SMITH J,et al.Toward Low-flying Autonomous MAV Trail Navigation Using Deep Neural Networks for Environmental Awareness[C]//2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).Vancouver:IEEE,2017:4241-4247.
[36] OLEYNIKOVA H,HONEGGER D,POLLEFEYS M.Reactive Avoidance Using Embedded Stereo Vision for MAV Flight[C]//2015 IEEE International Conference on Robotics and Automation (ICRA).Seattle:IEEE,2015:50-56.
[37] RICHTER C,BRY A,ROY N.Polynomial Trajectory Planning for Aggressive Quadrotor Flight in Dense Indoor Environments[M]//MASAYUKI I,PETER C.Robotics Research.[S.l.]:Springer,2016:649-666.
[38] MANCINI M,COSTANTE G,VALIGI P,et al.Toward Domain Independence for Learning-based Monocular Depth Estimation[J].IEEE Robotics and Automation Letters,2017,2(3):1778-1785.
[39] MANCINI M,COSTANTE G,VALIGI P,et al.J-MOD2:Joint Monocular Obstacle Detection and Depth Estimation[J].IEEE Robotics and Automation Letters,2018,3(3):1490-1497.
[40] WANG D S,LI W,LIU X G,et al.UAV Environmental Perception and Autonomous Obstacle Avoidance:A Deep Learning and Depth Camera Combined Solution[J].Computers and Electronics in Agriculture,2020,175:105523.
[41] SHE X B,HUANG D Q,SONG C J,et al.Multi-obstacle Detection Based on Monocular Vision for UAV[C]//2021 IEEE 16th Conference on Industrial Electronics and Applications (ICIEA).Chengdu:IEEE,2021:1067-1072.
[42] MOON H,SUN Y,BALTES J,et al.The IROS 2016 Competitions [Competitions][J].IEEE Robotics and Automation Magazine,2017,24(1):20-29.
[43] JUNG S,HWANG S,SHIN H,et al.Perception,Guidance,and Navigation for Indoor Autonomous Drone Racing Using Deep Learning[J].IEEE Robotics and Automation Letters,2018,3(3):2539-2544.
[44] LOQUERCIO A,KAUFMANN E,RANFTL R,et al.Deep Drone Racing:From Simulation to Reality with Domain Randomization[J].IEEE Transactions on Robotics,2020,36(1):1-14.
[45] MOON H,MARTINEZ-CARRANZA J,CIESLEWSKI T,et al.Challenges and Implemented Technologies Used in Autonomous Drone Racing[J].Intelligent Service Robotics,2019,12(2):137-148.
[46] KAUFMANN E,GEHRIG M,FOEHN P,et al.Beauty and the Beast:Optimal Methods Meet Learning for Drone Racing[C]//2019 International Conference on Robotics and Automation (ICRA).Montreal:IEEE,2019:690-696.
[47] COCOMA-ORTEGA J A,MARTINEZ-CARRANZA J.A CNN-based Drone Localisation Approach for Autonomous Drone Racing[C]//11th International Micro Air Vehicle Competition and Conference.Madrid:[s.n.],2019:89-94.
[48] ROJAS-PEREZ L O,MARTINEZ-CARRANZA J.A Temporal CNN-based Approach for Autonomous Drone Racing[C]//2019 Workshop on Research,Education and Development of Unmanned Aerial Systems (RED UAS).Cranfield:IEEE,2019:70-77.
[49] ROJAS-PEREZ L O,MARTINEZ-CARRANZA J.DeepPilot:A CNN for Autonomous Drone Racing[J].Sensors,2020,20(16):4524.
[50] FOEHN P,BRESCIANINI D,KAUFMANN E,et al.AlphaPilot:Autonomous Drone Racing[J].Autonomous Robots,2022,46(1):307-320.
[51] COCOMA-ORTEGA J A,MARTINEZ-CARRANZA J.A Compact CNN Approach for Drone Localisation in Autonomous Drone Racing[J].Journal of Real-time Image Processing,2022,19(1):73-86.
[52] KOURIS A,BOUGANIS C S.Learning to Fly by MySelf:A Self-supervised CNN-based Approach for Autonomous Navigation[C]//2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).Madrid:IEEE,2018:1-9.
[53] PADHY R P,VERMA S,AHMAD S,et al.Deep Neural Network for Autonomous UAV Navigation in Indoor Corridor Environments[J].Procedia Computer Science,2018,133:643-650.
[54] ROSS S,MELIK-BARKHUDAROV N,SHANKAR K S,et al.Learning Monocular Reactive UAV Control in Cluttered Natural Environments[C]//2013 IEEE International Conference on Robotics and Automation.Karlsruhe:IEEE,2013:1765-1772.
[55] SADEGHI F,LEVINE S.CAD2RL:Real Single-image Flight Without a Single Real Image[J/OL].(2017-06-08)[2023-03-10].https://arxiv.org/abs/1611.04201.
[56] PALOSSI D,CONTI F,BENINI L.An Open Source and Open Hardware Deep Learning-powered Visual Navigation Engine for Autonomous Nano-UAVs[C]//2019 15th International Conference on Distributed Computing in Sensor Systems (DCOSS).Santorini:IEEE,2019:604-611.
[57] CHAKRAVARTY P,KELCHTERMANS K,ROUSSEL T,et al.CNN-based Single Image Obstacle Avoidance on a Quadrotor[C]//2017 IEEE International Conference on Robotics and Automation (ICRA).Singapore:IEEE,2017:6369-6374.
[58] CARRIO A,VEMPRALA S,RIPOLL A,et al.Drone Detection Using Depth Maps[C]//2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).Madrid:IEEE,2018:1034-1037.
[59] ZHANG Z X,XIONG M K,XIONG H L.Monocular Depth Estimation for UAV Obstacle Avoidance[C]//2019 4th International Conference on Cloud Computing and Internet of Things (CCIOT).Changchun:IEEE,2019:43-47.
[60] DAI X,MAO Y X,HUANG T P,et al.Automatic Obstacle Avoidance of Quadrotor UAV via CNN-based Learning[J].Neurocomputing,2020,402:346-358.
[61] YANG S C,KONAM S,MA C,et al.Obstacle Avoidance Through Deep Networks Based Intermediate Perception[J/OL].(2017-04-27)[2023-02-22].https://arxiv.org/abs/1704.08759v1.
[62] EIGEN D,FERGUS R.Predicting Depth,Surface Normals and Semantic Labels with a Common Multi-scale Convolutional Architecture[C]//Proceedings of the 2015 IEEE International Conference on Computer Vision.Washington D.C.:ACM,2015:2650-2658.
基本信息:
DOI：
中图分类号:V279;V249;TP391.41
引用信息:
[1]王从宝,张安思,杨磊等.基于深度视觉的四旋翼无人机自主飞行感知和避障综述[J].无线电工程,2023,53(10):2233-2243.
基金信息:
国家重点研发计划(2020YFB1713302); 贵州省高等学校集成攻关大平台项目(黔教合KY字[2020]005); 贵州省教育厅青年科技人才成长项目(黔教合KY字[2022]142号); 贵州大学引进人才科研项目(贵大人基合字(2021)74号)~~