Generalising rescue operations in disaster scenarios using drones: a lifelong reinforcement learning approach

Abstract

Search and rescue (SAR) operations in post-earthquake environments are hindered by unseen environment conditions and uncertain victim locations. While reinforcement learning (RL) has been used to enhance unmanned aerial vehicle (UAV) navigation in such scenarios, its limited generalisation to novel environments, such as post-disaster environments, remains a challenge. To deal with this issue, this paper proposes an RL-based framework that combines the principles of lifelong learning and eligibility traces. Here, the approach uses a shaping reward heuristic based on pre-training experiences obtained from similar environments to improve generalisation, and simultaneously, eligibility traces are used to accelerate convergence of the overall approach. The combined contributions allows the RL algorithm to adapt to new environments, whilst ensuring fast convergence, critical for rescue missions. Extensive simulation studies show that the proposed framework can improve the average reward return by 46% compared to baseline RL algorithms. Ablation studies are also conducted, which demonstrate a 23% improvement in the overall reward score in environments with different complexities and a 56% improvement in scenarios with varying numbers of trapped individuals.