Large language models powered system safety assessment: applying STPA and FRAM

Abstract

The advancement of large language models (LLMs) shows immense promise in many domains. However, their reliability is still questionable. This study aims to comparatively examine the performance of ChatGPT and Gemini in conducting a stand-alone systems-based risk assessment using System-Theoretic Process Analysis (STPA) and the Functional Resonance Analysis Method (FRAM). Our findings revealed that both LLMs demonstrated weaknesses in their analyses, with ChatGPT generally outperforming Gemini regarding response comprehensiveness and adhering to the prompted format. Specifically, LLMs failed to use systems thinking in their stand-alone applications and failed to follow up on previous prompt outputs. While LLMs can provide substantial amounts of information quickly, the effectiveness of LLMs in system safety assessment is contingent on addressing their limitations and implementing strategies to improve their capabilities.