Attention-based Fusion for Stroke Lesion Segmentation on Computed Tomography Perfusion Data

<jats:p>In recent times, stroke has emerged as a significant threat to humans, transforming affected brain tissue into core and penumbra regions. As the penumbra becomes irreversible over time, early core region segmentation is crucial. Automatic segmentation systems offer an efficient alternative to manual segmentation and aid radiologists in stroke lesion segmentation using computed tomography and Computed Tomography Perfusion (CTP) maps that comprise four parameter maps. This automatic segmentation is increasingly used in interactive, multimedia-based systems for diagnostic tools and AI-driven health applications. Top-performing models that follow the patch-processing approach suffer from high inference times. To incorporate effective feature extraction at image-level inferences, which reduces the inference time, we present a hybrid fusion technique that combines early and bottleneck fusion, leveraging two separate encoders for effective feature extraction. Moreover, fusing the information from various fusion methods arbitrarily may not yield optimal results. Consequently, we have introduced two attention modules, i.e., cross-modal attention and cross-fusion attention modules, designed for the effective integration of features derived from diverse modalities and multiple fusion strategies, respectively. The findings highlight a considerable reduction in computational time alongside achieving a comparable Dice score. Additionally, the incorporation of hybrid fusion and attention modules in the baseline notably increased the Dice score from 0.482 to 0.521 in the validation dataset and achieved 0.48 in the test dataset of ISLES 2018. It also demonstrates competitive performance compared to existing models while maintaining efficient prediction times.</jats:p>