Transition flight control system design for fixed-wing VTOL UAV: a reinforcement learning approach

Tilt-rotor vertical takeoff and landing aerial vehicles have been gaining popularity in urban air mobility applications because of their ability in performing both hover and forward flight regimes. This hybrid concept leads energy efficiency which is quite important to obtain a profitable and sustainable operation. However, inherent dynamical nonlinearities of the aerial platform requires adaptation capability of the control systems. In addition, transition flight phase should be planned carefully not only for a profitable operation but also for a safe transition between flight regimes in the urban airspace. In this paper, transition flight phase of a tilt-rotor vertical-takeoff-and-landing unmanned aerial vehicle (UAV) is studied. Low-level flight control systems are designed based on adaptive dynamic inversion methodology to compensate aerodynamic effects during the transition phase. Reinforcement learning method is utilized to provide safety and energy efficiency during the transition flight phase. An actor-critic agent is utilized and trained by using deep deterministic policy gradient algorithm to augment the collective channel of the UAV. This augmentation on the collective input is used to adjust flight path angle of the UAV which results in adjusting the angle of attack when pitch angle is zero. By using this relationship, it is proposed to generate aerodynamic lift force and perform transition flight with minimum altitude change and energy usage. Simulation results show that the agent reduces the collective signal level as the aerodynamic lift force is created in the descent flight phase. This affects overall system efficiency, reduces operational costs and makes the enterprise more profitable.