Drone pollution tracking in cities using recurrent proximal policy optimization learning

Abstract

Future smart cities will need to monitor polluters that periodically or burst emit illegal gases that is harmful to the environment. Tracking these sources in cities that have building obstacles and variation wind vector fields is challenging. Traditional methods using gradient kernels and partial-swarm-optimisation may not be suitable when the emissions are intermittent and pollution concentrations maybe trapped in local pockets. As such, step size tuning becomes difficult to generalise in these variational dynamic pollution environments. Here, in this paper, we have developed a simulated urban pollution propagation environment, whereby a drone is scanning the environment for gradients to search and localise the source. We consider both proximal policy optimisation (PPO)-based reinforcement learning and its recurrent PPO (R-PPO) alternative to achieve stable and reliable improvement of policy without the need to fine tune step sizes. We show localisation results across a range of wind, obstacle, and emission scenarios with success rate of 76-79% and high path efficiency of 95-96% in ideal conditions. When we examine alternative city structures and burst emissions, we can achieve success rate of 34% and path efficiency of 52%, showing that there is some generalisation in capability.