Reinforcement learning of pick and place tasks
DOI:
https://doi.org/10.17979/ja-cea.2025.46.12226Keywords:
Reinforcement learning, Intelligent robotics, Robots manipulators, Robotics technologyAbstract
Pick and place is one of the most common and widely implemented tasks in robotic environments. Any complex robotic manipulation task inherently involves the need to grasp an object from one location in order to perform a specific action with it and, once completed, return it to the same or another location. In this work, we trained a robotic agent with deep reinforcement learning to perform pick and place tasks in which our agent learns to grasp objects and place them in locations of varying difficulty, such as inside a basket, insertion into a hole or slot, and stacking on top of another small object. We have defined and adjusted policies and evaluated them in 50 experiments with arbitrary gripping poses. The results obtained show that our trained policies successfully perform the task in 98 %, 78% y 80% of cases, respectively, depending on the type of location.
References
Billard, A., Kragic, D., 2019. Trends and challenges in robot manipulation. Science 364 (6446). DOI: 10.1126/science.aat8414
Calli, B., Singh, A., Bruce, J., Walsman, A., Konolige, K., Srinivasa, S., Abbeel, P., Dollar, A., 2017. Yale-cmu-berkeley dataset for robotic manipulation research. The Int. J. of Robotics Research 36 (3), 261–268. DOI: 10.1177/0278364917700714
Coumans, E., Bai, Y., 2016-2021. Pybullet, a python module for physics simulation for games, robotics and machine learning. URL: http://pybullet.org
Cui, J., Trinkle, J., 2021. Toward next-generation learned robot manipulation. Science Robotics 6 (54). DOI: 10.1126/scirobotics.abd9461
Fang, B., Jia, S., Guo, D., et al., 2019. Survey of imitation learning for robotic manipulation. Int. J. of Intelligent Robotics Application 3, 362–369. DOI: 10.1007/s41315-019-00103-5
Gomes, N., Martins, N., Lima, J., W¨ortche, H., 2021. Deep reinforcement learning applied to a robotic pick and place application. In: Pereira, A., Fernandes, F., Coelho, J., Pacheco, M., Alves, P., Lopes, R. (Eds.), Optimization, Learning Algorithms and Applications. Springer International Publishing, pp. 251–265. DOI: 10.1007/978-3-030-91885-918
Haarnoja, T., Zhou, A., Abbeel, P., Levine, S., 2018. Soft actor-critic: Offpolicy maximum entropy deep reinforcement learning with a stochastic actor. In: Dy, J. G., Krause, A. (Eds.), Proc. of the 35th Int. Conf. on Machine Learning (ICML). Vol. 80. PMLR, pp. 1856–1865.
Han, D., Mulyana, B., Stankovic, V., Cheng, S., 2023. A survey on deep reinforcement learning algorithms for robotic manipulation. Sensors 23 (7). DOI: 10.3390/s23073762
Kinova, 2024. Kinova. URL: https://www.kinovarobotics.com/product/gen3-robots
Kroemer, O., Niekum, S., Konidaris, G., 2021. A review of robot learning for manipulation: Challenges, representations, and algorithms. J. of Machine Learning Research 22 (30), 1–82. URL: http://jmlr.org/papers/v22/19-804.html
Lobbezoo, A., Qian, Y., Yanjun, K., Kwon, H.-J., 2021. Reinforcement learning for pick and place operations in robotics: A survey. Robotics 10 (3). DOI: 10.3390/robotics10030105
Raffin, A., Hill, A., Gleave, A., Kanervisto, A., Ernestus, M., Dormann, N., 2021. Stable-baselines3: Reliable reinforcement learning implementations. J. of Machine Learning Research 22 (268), 1–8. URL: http://jmlr.org/papers/v22/20-1364.html
Robotiq, 2024. Robotiq. URL: https://robotiq.com/
Si, W., Wang, N., Yang, C., 2021. A review on manipulation skill acquisition through teleoperation-based learning from demonstration. Cognitive Computation and Systems 3 (1), 1–16. DOI: https://doi.org/10.1049/ccs2.12005
Sutton, R. S., Barto, A., 2018. Reinforcement learning: An introduction. MIT Press, Cambridge, Massachusetts.
Towers, M., Kwiatkowski, A., Terry, J. K., Balis, J. U., Cola, G. D., Deleu, T., Goul˜ao, M., Kallinteris, A., Krimmel, M., KG, A., Perez-Vicente, R., Pierré, A., Schulhoff, S., Tai, J. J., Tan, H., Younis, O. G., 2024. Gymnasium: A standard interface for reinforcement learning environments. CoRR abs/2407.17032. DOI: 10.48550/ARXIV.2407.17032
van Otterlo, M., Wiering, M., 2012. Reinforcement learning and markov decision processes. In: Wiering, M., van Otterlo, M. (Eds.), Reinforcement Learning: State of the Art. Springer Berlin Heidelberg, pp. 3–42. DOI: 10.1007/978-3-642-27645-31
Zhu, Y., Joshi, A., Stone, P., Zhu, Y., 2023. Viola: Imitation learning for vision-based manipulation with object proposal priors. In: Liu, K., Kulic, D., Ichnowski, J. (Eds.), Proc. of The 6th Conference on Robot Learning (CoRL). Vol. 205. PMLR, pp. 1199–1210.
Downloads
Published
Issue
Section
License
Copyright (c) 2025 José María Hernández Hernández, Julio Castaño Amorós, Pablo Gil
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
