GROOT: GPT-based human-RObOT interface - Maniar, Shobhit; Tang, Gilbert; Chacin, Marco (Institute of Electrical and Electronics Engineers (IEEE), 2025)
Large Language Models (LLMs) have revolutionized the realm of Natural Language Processing (NLP). Their proficiency in planning and reasoning, combined with code generation capabilities, presents a novel avenue for robotics applications. This work introduces GROOT, a novel speech-based language-agnostic middleware that uses instructions and code examples as grounding principle to leverage Generative Pre-Trained Transformer’s ability to produce code for new and unseen tasks. Unlike methods based on language-conditioned robot policies, GROOT capitalises on auto-regressive code generation inspired by Code-as-Policy (CaP) and ProgPrompt. The aim is to create a human-robot interface using GROOT that enables the embodiment of an LLM to take user instructions like ’move in a square’, ’move 20 cm in front’, ’go to position ’X’ on the grid’ and return policy code based on robot API. In this paper, GROOT was used in a number of experiments to assess its performance to few-shot learning against spatial reasoning, logical reasoning and compound simulated tasks. This work reflects the potential of prompting code-based examples and API-based instructions as a grounding method to integrate large-language models with robotic platforms, envisioning seamless and intuitive human-robot interactions.
Keywords: 46 Information and Computing Sciences; 4608 Human-Centred Computing; 4602 Artificial Intelligence; Machine Learning and Artificial Intelligence; Networking and Information Technology R&D (NITRD); 4 Quality Education; Natural Language Processing; Human-Robot Interaction; Embodied Intelligent Agent; Few-shot Planners; Policy Code Generation; Prompt Engineering