Novel deep learning approaches for flight delays prediction

Abstract

This thesis focuses on the development of predictive models for flight delay using the United States (US) Bureau of Transportation Statistics (BTS) dataset. The research aims to assess the potential of the BTS data for creating data-driven flight delay predictive models. Exploratory Data Analysis (EDA) reveals complex relationships, irregularities, outliers, missing values and invalid values within the dataset, posing challenges for traditional Machine Learning algorithms and resulting in low-performance models. To address these limitations, six advanced prediction algorithms are developed. The approaches include binary classification using a filter-based procedure and deep-stacked auto-encoder, a hybrid ensemble learner, a fully connected Bi- Long-Short Term Memory (LSTM), a social ski driver conditional autoregressive value at risk optimisation-aware deep neural network, and a novel gradient mayfly-based optimisation DeepONet approach. The US BTS dataset, along with expert-defined records, is used to validate the proposed algorithms. The research findings demonstrate that an optimisation- aware data-driven predictive approach yields reliable and robust results for flight delay prediction. Among the developed approaches, the gradient mayfly optimisation DeepONet model, particularly the proposed data-driven optimisation-aware DeepONet, outperforms other strategies in terms of prediction error on benchmark evaluation metrics. The validation results highlight the effectiveness of the data-driven optimisation- aware DeepONet model, achieving a minimum prediction error of 0.0043. This suggests that more than 90% of flights can be accurately predicted based on the findings of this research.