Modular integration of LLM-managed tools for the TIAGo++ robot

Abstract

This article presents a modular adaptation of a system based on Large Language models (LLM) to the TIAGo++ robot, with the goal of reusing the same agent across different robotic platforms without modifying its reasoning logic. The agent interprets user commands and employs semantic tools organized into four categories: query, diagnostic, expression and action. These tools enable the system to obtain information from the environment, generate verbal responses and execute physical tasks, all througha common interface that facilitates usability and portability. In particular, the action tools were implemented using a manipulation server compatible with MoveIt, allowing the execution of tasks such as handing over objects, placing them near a person, pushing them, or pouring contents between containers. Each tool provides a structured output allowing the agent to assess the success of the action and decide how to proceed. This modular architecture supports reuse, portability, and more effective interaction, while remaining open to the integration of new tools.