Personalizing driver agent using large language models for driving safety and smarter human–machine interactions

Abstract

Driver assistance systems have been shown to reduce crashes by providing real-time warnings or assistance, with their effectiveness depending on communication with driver. Due to their unique characteristics, human drivers possess varying hazard perception skills and interaction preferences, making personalized assistance crucial to improving the user experience and system acceptance. However, how to leverage multimodal interfaces that dynamically adapt to warning contents and driver characteristics remains an open question. At the same time, large language models (LLMs) have demonstrated advanced capabilities in knowledge acquisition, planning, and human–machine collaboration, offering potential solutions for existing warning systems. Thus, we develop an LLM-based personalized driver agent (PDA), which provides personalized warnings through multimodal interactions (visual, voice, and tactile). The agent’s architecture mimics human cognitive processes via four core modules: memory, perception, control, and action. Results from our experiments indicate that the LLM-PDA effectively customizes warning contents for different drivers in various situations, providing enhanced safety and driver support. This article pioneers the integration of LLMs into automotive human–vehicle interaction and offers novel insights into personalized human–machine interaction in intelligent vehicles.