Generation and Dynamic Adjustment of Behavior Trees using LLMs for the Control of a Social Robot
DOI:
https://doi.org/10.17979/ja-cea.2025.46.12071Keywords:
Mission planning and decision making, Cognitive aspects of automation systems and humans, Mobile robots, Large language models, Behavior TreesAbstract
Large Language Models (LLMs) have become key tools for generating flexible and context-aware robotic behaviors. However, adapting to unforeseen events and ensuring robust task completion remain significant challenges. This work presents a system that uses LLMs and Behavior Trees (BTs) to enable a social robot to generate, execute, and adapt task plans based on natural language instructions. By combining a BT planner with a failure interpretation module, the system dynamically adjusts BTs in response to execution errors or environmental changes. Unlike static BT-based methods, our approach detects problems and proposes alternatives or requests clarifications from the user in real time, improving human-robot interaction. We validate the system across various real-world scenarios, demonstrating its effectiveness in enhancing flexibility and resilience in dynamic environments.
References
Ao, J., Ren, Z., Su, H., Zhu, J., 2024. Llm-as-bt-planner: Large language models for behavior tree based robotic task planning. arXiv preprint arXiv:2402.08013.
Ben-Ari, M., Mondada, F., 2018. Finite state machines. Elements of Robotics, 55–61. DOI: 10.1007/978-3-319-62533-1 4
Biggar, O., Zamani, M., Shames, I., 2020. A principled analysis of behavior trees and their generalisations. arXiv preprint arXiv:2008.11906.
Brand, D., Zafiropulo, P., 1983. On communicating finite-state machines. Journal of the ACM (JACM) 30 (2), 323–342.
Cao, Y., Rajeswaran, A., Grover, A., Feinberg, V., 2023. Robot task planning with large language models via behavior tree generation. arXiv preprint arXiv:2310.06796.
Chang, Y., Wang, X., Wang, J., Wu, Y., Yang, L., Zhu, K., Chen, H., Yi, X., Wang, C., Wang, Y., Ye, W., Zhang, Y., Chang, Y., Yu, P. S., Yang, Q., Xie, X., 3 2024. A survey on evaluation of large language models. ACM Transactions on Intelligent Systems and Technology 15, 39. DOI: 10.1145/3641289
Colledanchise, M., Ögren, P., 8 2017. Behavior trees in robotics and ai: An introduction. Behavior Trees in Robotics and AI. DOI: 10.1201/9780429489105
De la Cruz, P., Piater, J., Saveriano, M., 2020. Reconfigurable behavior trees: Towards an executive framework meeting high-level decision making and control layer features. arXiv preprint arXiv:2007.10663.
Deng, J., Lin, Y., 1 2022. The benefits and challenges of chatgpt: An overview. Frontiers in Computing and Intelligent Systems 2, 81–83. DOI: 10.54097/FCIS.V2I2.4465
Ghzouli, R., Berger, T., Johnsen, E. B., Dragule, S., Wasowski, A., 2020. Behavior trees in action: A study of robotics applications. In: Proceedings of the 13th ACM SIGPLAN International Conference on Software Language Engineering. ACM, pp. 127–140. DOI: 10.1145/3426425.3426942
Ghzouli, R., Berger, T., Johnsen, E. B., Dragule, S., Wasowski, A., 2023. Behavior trees and state machines in robotics applications. IEEE Transactions on Software Engineering 49 (1), 1–14. DOI: 10.1109/TSE.2021.3115347
Iovino, M., Scukins, E., Styrud, J., Ögren, P., Smith, C., 2021. A survey of behavior trees in robotics and ai. Robotics and Autonomous Systems 136, 103710. DOI: 10.1016/j.robot.2020.103710
Isla, D., 2005. Handling complexity in the halo2 ai. Game Developers Conference.
Izzo, S., Ardulov, V., Scioni, E., Mohan, R., Cheng, X., He, Y., Hager, G. D., 2024. Btgenbot: Behavior tree generation using compact language models. arXiv preprint arXiv:2402.00560.
Li, M., Lin, L., Lin, Z., Li, X., Du, Y., 2024. A study of behavior tree generation from natural language with large language models. arXiv preprint arXiv:2403.12658.
Lykov, A., Ravichandar, H., Klee, S., Park, D., Nagarajan, P., Troniak, D., Liu, Z., Jenkins, R., Khante, A., Chai, J. Y., et al., 2023. Llm-mars: Leveraging large language models for multi-turn, multi-modal, multi-scene human-robot interaction. arXiv preprint arXiv:2310.02291.
Mahadevan, R., Wang, S., Thomaz, A. L., 2024. Generative language models for expressive robot social behaviors. arXiv preprint arXiv:2402.09013.
Merino-Fidalgo, S., Casanova, E. Z., García-Bermejo, J. G., Domingo, J. D., 6 2025. Generación de behavior trees mediante llm para el control de un robot social. Simposios del Comité Español de Automática (CEA) 1. URL: https://ingmec.ual.es/ojs/index.php/RBVM25/article/view/10
Mishra, B., Kertesz, A., 2020. The use of mqtt in m2m and iot systems: A survey. IEEE Access 8, 201071–201086. DOI: 10.1109/ACCESS.2020.3035849
Ögren, P., Sprague, C. I., 5 2022. Behavior trees in robot control systems. Annual Review of Control, Robotics, and Autonomous Systems 5, 81–107. DOI: 10.1146/ANNUREV-CONTROL-042920-095314
OpenAI, 2024. Introducing chatgpt — openai. Accessed: 26-12-2024. URL: https://openai.com/index/chatgpt/
Pezzato, C., Hernandez Corbato, C., Bonhof, S., Wisse, M., 2020. Active inference and behavior trees for reactive action planning and execution in robotics. arXiv preprint arXiv:2011.09756.
Sánchez-Girón, C., Gómez, M. G., Domingo, J. D., García-Bermejo, J. G., Casanova, E. Z., 2024. Integración convnext-yolo mediante cvv para detectar ca´ıdas en robot social. Jornadas de Automática. DOI: https://doi.org/10.17979/jacea.2024.45.10788
Temi, 2024. Introducing temi robot v3. Accessed: 03-01-2025. URL: https://www.robotemi.com/product/temi-sales-contact/
Wang, T., Patel, R., Qian, Y., Mataric, M. J., 2024. Srlm: Social robot language modeling for goal-conditioned navigation. arXiv preprint arXiv:2401.11317.
Downloads
Published
Issue
Section
License
Copyright (c) 2025 Sergio Merino Fidalgo, Eduardo Zalama Casanova, Jaime Gómez García-Bermejo, Jaime Duque Domingo
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
