Reinforcement Learning for pH Control in Microalgae Photobioreactors
DOI:
https://doi.org/10.17979/ja-cea.2025.46.12099Keywords:
Adaptive control, Offline learning, Biological process controlAbstract
This work proposes a reinforcement learning control system to regulate the pH in microalgae photobioreactors, using an agent based on the Deep Deterministic Policy Gradient (DDPG) algorithm. This approach learns from historical data generated by conventional controllers, such as the PID, without requiring direct interaction with the real system. After its implementation, the agent can continue training periodically with new experiences, allowing it to adapt to the changing dynamics of the biological process. Simulation results show that the proposed algorithm improves traditional control metrics, such as the integral of absolute error, by 12% compared to a PID controller. Additionally, periodic retraining supports the adaptation and robustness of the system. These results position reinforcement learning as a promising alternative for automating this type of bioprocess.
References
Caparroz, M., Guzmán, J. L., Berenguel, M., Acién, F., 2024. A novel datadriven model for prediction and adaptive control of pH in raceway reactor for microalgae cultivation. New Biotechnology 82, 1–13.
Caparroz, M., Guzmán, J. L., Berenguel, M., Gil, J. D., Acién, F. G., 2023. Model reference adaptive control for pH regulation. Revista Iberoamericana de Automática e Informática industrial 22 (2), 126–134.
Caparroz, M., Guzmán, J. L., Gil, J. D., Berenguel, M., Acién, F. G., 2025. A hybrid MRAC-PI approach to regulate pH in raceway reactors for microalgae production. Control Engineering Practice 156, 106191.
Castilla, M. M., Campoy-Iniesta, C., Álvarez, J. D., 2025. Control del confort térmico mediante aprendizaje por refuerzo en edificios. Revista Iberoamericana de Automática e Informática Industrial (RIAI) 22 (2), 146–155.
Guzmán, J. L., Acién, F., Berenguel, M., 2021. Modelling and control of microalgae production in industrial photobioreactors. Revista Iberoamericana de Autom´atica e Inform´atica Industrial 18 (1), 1–18.
Juneja, A., Ceballos, R. M., Murthy, G. S., 2013. Effects of environmental factors and nutrient availability on the biochemical composition of algae for biofuels production: a review. Energies 6 (9), 4607–4638.
Lillicrap, T. P., Hunt, J. J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., Silver, D., Wierstra, D., 2015. Continuous control with deep reinforcement learning. arXiv preprint arXiv:1509.02971.
Monteiro, M., Kontoravdi, C., 2024. Bioprocess control: A shift in methodology towards reinforcement learning. In: Computer Aided Chemical Engineering. Vol. 53. Elsevier, pp. 2851–2856.
Pataro, I. M., Gil, J. D., Guzmán, J. L., Berenguel, M., Lemos, J. M., 2023. A learning-based model predictive strategy for pH control in raceway photobioreactors with freshwater and wastewater cultivation media. Control Engineering Practice 138, 105619.
Petsagkourakis, P., Sandoval, I. O., Bradford, E., Zhang, D., del Rio-Chanona, E. A., 2020. Reinforcement learning for batch bioprocess optimization. Computers & Chemical Engineering 133, 106649.
Rajasekhar, N., Radhakrishnan, T., Samsudeen, N., 2025. Exploring reinforcement learning in process control: a comprehensive survey. International Journal of Systems Science, 1–30.
Sachio, S., del Rio-Chanona, E. A., Petsagkourakis, P., 2021. Simultaneous process design and control optimization using reinforcement learning. IFAC-PapersOnLine 54 (3), 510–515.
The MathWorks, Inc., 2024. MATLAB R2024b. http://es.mathworks.com/products/matlab/, accessed on 15/05/2025.
Wang, H., Kontoravdi, C., Del Rio Chanona, A., 2025. Offline reinforcement learning for bioprocess optimization with historical data. In: 14th IFAC Symposium on Dynamics and Control of Process Systems, including Biosystems (DYCOPS 2025): Slovakia, Bratislava, June 16-19, 2025.
Downloads
Published
Issue
Section
License
Copyright (c) 2025 Juan Diego Gil Vergel, Antonio Del Rio Chanona, José L. Guzmán, Manuel Berenguel
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
