ML-Enhanced Visual Inertial Navigation System for UAVs

Abstract

Urban Air Mobility (UAM) applications, including passenger taxis, cargo transport and aerial surveys, are significantly impacted by GNSS vulnerabilities in urban environments highlighting the requirement of accurate and robust alternative navigation solutions like Visual-Inertial Navigation Systems (VINS) to ensure reliable and safe operation. However, ensuring robustness and high accuracy in the VINS system requires the development of advanced methodologies to address the limitations of current state-of-the-art solutions, which struggle to perform effectively in low-light environments, adverse weather conditions, low -texture areas, and during rapid motion variation. Therefore, the primary aim of this thesis is to develop a robust and resilient vision-based alternative solution to GNSS outage by mitigating visual faults, enabling safe and reliable navigation under challenging urban environments for UAM systems.

This thesis designs an advanced methodology to address visual faults by exploring hybrid architectures in VINS systems to improve robustness in complex visually degraded urban environments, ensuring operational safety during GNSS outages. To enhance robustness and reliability, a hybrid VINS system incorporating multiple error compensation (Multi-ML Hybrid VIO) is proposed that simultaneously addresses and mitigates visual failure mode effects arising from weather conditions, lighting effects, low-texture environments and flight dynamics to ensure integrity. Furthermore, uncertainty prediction for visual failure modes is proposed to address aleatoric and epistemic uncertainty, providing compensation for accumulated errors that improve accuracy and robustness. The evaluation of different hybrid architectures proposed during this study indicates that Multi-ML Hybrid Visual Inertial Navigation Systems (VIO) solution outperforms in mitigating visual failure modes and addressing various sources of visual uncertainty, ensuring navigation integrity. Training and testing results obtained from simulated complex urban prototypes demonstrate the remarkable performance of the proposed solution under diverse visually degraded scenarios and its seamless integration within the multi-sensor navigation system during GNSS outage.

Overall, the findings in this thesis demonstrate a robust and reliable vision-based alternative navigation solution tailored for UAM applications.