Alzheimer’s Disease Diagnosis Using a Multimodal Approach with 3D MRI and PET

Abstract

Alzheimer’s disease is an irreversible brain disease that severely damages human thinking and is the seventh leading cause of death worldwide. Early diagnosis plays an important part es- pecially at the Mild Cognitive Impairment stage, where timely intervention can help slow its progression before it advances to AD. Neuroimaging data, like MRI and PET scans, can help detect brain changes early by providing structural and functional brain changes related to the disease. However, despite the availability of various imaging modalities for the same patient, the development of multi-modal models leveraging these modalities remains underexplored. This thesis aims to address this gap by proposing and evaluating classification models using 3D MRI and amyloid PET scans in a multimodal framework. We first employ a 3D Convo- lutional Neural Network, followed by three fusion techniques: feature concatenation, Gated Multimodal Unit, and Gated Self-Attention. To further improve classification performance and computational efficiency, we integrate a Mixture of Experts model, which dynamically selects the most relevant subnetworks for each prediction. Finally, we utilize Grad-CAM to visualize disease-related regions, ensuring model interpretability. The results show that the GMU-based model achieves 95.47% accuracy and specificity of 96.73% in the NC vs. AD classification task, outperforming state-of-the-art approaches. Ad- ditionally, the model successfully locates disease-related regions in both MRI and PET scans, with different activation patterns in each modality, according to Grad-CAM analysis. This re- sult supports the effectiveness of a multimodal strategy in the diagnosis of AD by confirming the complementary nature of MRI and PET.