Multimodal architecture for interpretable image analysis.

Abstract

Artificial intelligence models have demonstrated remarkable performance, although their limited interpretability poses a challenge in fields where understanding the decisions made is crucial. This challenge is particularly significant in the clinical setting, such as in the analysis of breast cancer, a disease characterized by high variability in its manifestations. To address this limitation, we propose a multimodal approach that uses a simplified synthetic dataset combining images with a vector of additional data to guide the model’s learning. For interpretable analysis, the SHAP tool is applied to evaluate the contributions of both inputs: on the one hand, the image, and on the other, the vector variables. The results demonstrate the viability of the proposed approach, suggesting as future lines of work the optimization of the model to reduce classification errors and improve the detection of complex cases, as well as its validation using real-world data.