SpecTralUNetFormer: Advancing Hyperspectral Medical Image Segmentation through Spectral-Spatial-Attentive Learning

Abstract

Surgical precision is greatly improved after brain tumors are accurately diagnosed and traced in the operating room. Hyperspectral Imaging (HSI) is a new method that discriminates between healthy tissue and suspicious areas in real time according to their spectral signatures. This paper compares the performance of five deep learning models, i.e., CNN, 3D CNN, Vision Transformer (ViT), U-Net, and the SpecTralUNetFormer, introduced in this paper, on the Hyperspectral Imaging Benchmark for Intraoperative Brain Tumor Detection dataset. The dataset has 62 hyperspectral images captured from 34 subjects, 128 spectral bands from 400 nm to1000nm. The SpecTralUNetFormer proposed here combines 3D CNNs for learning spectral-spatial features, a U-Net encoder-decoder for spatial localization, and a Transformer bottleneck for learning long-range dependencies. Data preprocessing includes normalization, PCA-based spectral band reduction, and data augmentation. The models are tested for classification accuracy, AUC, and computational efficiency, and a comparative analysis of the various architectures is shown. The experiments show that SpecTralUNetFormer performs better than conventional architectures with improved segmentation accuracy and improved generalization in hyperspectral brain tumor detection. The objective of this work is to improve intraoperative decision-making during surgery by using deep learning methods for real-time tumor detection, ultimately resulting in improved surgical accuracy and patient outcomes.