Development of Real-Time Control System based on Deep Learning for UAVʹs Object Detection, Tracking and Safe-Landing

Abstract

In the current work, an approach to implement AI-based techniques in real-time focusing mainly on the detection, tracking, and landing on the target object is presented. For an object detection, CNN algorithm is utilized. For object tracking and stabilizing, a novel algorithm is developed that can execute along with object detection via sequential stream data. For landing, a vision-based algorithm is used to estimate the distance between the UAV and the detected object. For UAV control, a Fuzzy-PID controller is designed to steer the UAV by a continuously manipulation of the actuators based on the stream data from the tracking unit and dynamics of the UAV. For UAV landing, a new type of fuzzy logic controller is developed to compensate for the nonlinear ground effect that affects the safe landing of the UAV. All the developed algorithms are executed on an NVIDIA Jetson TX2 embedded artificial intelligence device, and an ARM Cortex M4. Experimental results show that the tracking algorithm responds faster than conventionally used approaches, and the safe landing algorithm minimized the landing time of the UAV and provided the safety assurance as compared to conventional controllers. Furthermore, a farming monitoring and automated wireless charging is considered as an application example.