A Novel Hybrid 3D CNN And Spatio-Temporal Transformer Model For Multi-Stage Dementia Detection And Progression Prediction Using ADNI And OASIS Datasets

Abstract

Dementia, encompassing Alzheimer’s dis- ease (AD) and related disorders, presents a growing global health challenge, necessitating advanced tools for early detection and progression prediction. This paper proposes a novel hybrid deep learning model integrating a 3D Convolutional Neural Network (CNN) for spa- tial feature extraction from structural MRI scans with a Spatio-Temporal Transformer (ST-Transformer) for joint modeling of spatial brain regions and temporal dependencies in longitudinal data. The model classifies dementia into five stages: Normal Control (NC), Early Mild Cognitive Impairment (EMCI), Significant Mem- ory Concern (SMC), Late Mild Cognitive Impairment (LMCI), and Alzheimer’s Disease (AD), while also pre- dicting future progression. Trained on combined ADNI and OASIS datasets, the proposed model achieves 98.2% accuracy in classification and a mean absolute error (MAE) of 0.15 in progression prediction, outperforming state-of-the-art hybrid CNN-Transformer models. Exten- sive evaluations, including precision, recall, F1-scores, confusion matrices, ROC curves, precision-recall curves, class-wise F1-score visualizations, training curves, and ablation studies, demonstrate its robustness and novelty in integrating interleaved spatio-temporal attention.