Sense and avoid using hybrid convolutional and recurrent neural networks

Abstract

This work develops a Sense and Avoid strategy based on a deep learning approach to be used by UAVs using only one electro-optical camera to sense the environment. Hybrid Convolutional and Recurrent Neural Networks (CRNN) are used for object detection, classification and tracking whereas an Extended Kalman Filter (EKF) is considered for relative range estimation. Probabilistic conflict detection and geometric avoidance trajectory are considered for the last stage of this technique. The results show that the considered deep learning approach can work faster than other state-of-the-art computer vision methods. They also show that the collision can be successfully avoided considering design parameters that can be adjusted to adapt to different scenarios.