Real-time perception and control systemfor flexible handling with industrial robots

Abstract

This work presents the development of a flexible robotic manipulation system based on computer vision for industrial applications. The main goal is to identify and locate objects through instance segmentation and automatically compute their pose (position and orientation) to enable robotic-arm manipulation. The system follows a modular architecture built on the ROS 2 ecosystem, integrating visual perception components, manipulator control, and a graphical interface. A ZED Mini stereo camera captures RGB-D images, and a YOLO-based instance segmentation model identifies objects and generates corresponding masks. These, together with depth information, allow a robust grasping point to be computed. Finally, the proposed system incorporates a verification post-grasping mechanism. The results demonstrate robust and real-time performance, validating the application of advanced vision techniques in unstructured industrial environments. The code is publicly available in the repository (https://github.com/PedroMM03/semantic-segmentation-adaptive-manipulation).