Intuitive gesture control interface for swarm of aerial robots

Abstract

In recent years, research in Unmanned Aerial Vehicles (UAV) swarm technology has seen accelerated progress across various fields, including disaster response, environmental monitoring, precision agriculture, surveillance, military, last-mile delivery, and search-and-rescue operations, due to their promising potential in solving challenging problems. The ability to rapidly access locations that are difficult to reach for humans and other robots has made UAVs unique. While certain tasks can be accomplished by a single drone, a fleet of drones can perform the same tasks more efficiently, reducing task completion time and with enhanced capabilities. Human Swarm Interaction (HSI) is an important consideration in the system deployment of drone swarms, although it is needed primarily for information exchange and control for some applications, the controllability and intuitiveness of drone swarms command and control are crucial for any applications that involve human intervention. This paper explores human swarm interactions through the development of a novel gesture-based control framework, and its demonstration for efficient aerial swarm management. A custom Py-thon-based framework is designed and developed, leveraging Tensor-Flow Lite and MediaPipe for real-time hand gesture recognition, enabling precise and efficient drone swarm control.