Chartsias, Agisilaos and Joyce, Thomas and Papanastasiou, Giorgos and Semple, Scott and Williams, Michelle and Newby, David E and Dharmakumar, Rohan and Tsaftaris, Sotirios A (2019) Disentangled representation learning in cardiac image analysis. Medical Image Analysis, 58. p. 101535. DOI https://doi.org/10.1016/j.media.2019.101535
Chartsias, Agisilaos and Joyce, Thomas and Papanastasiou, Giorgos and Semple, Scott and Williams, Michelle and Newby, David E and Dharmakumar, Rohan and Tsaftaris, Sotirios A (2019) Disentangled representation learning in cardiac image analysis. Medical Image Analysis, 58. p. 101535. DOI https://doi.org/10.1016/j.media.2019.101535
Chartsias, Agisilaos and Joyce, Thomas and Papanastasiou, Giorgos and Semple, Scott and Williams, Michelle and Newby, David E and Dharmakumar, Rohan and Tsaftaris, Sotirios A (2019) Disentangled representation learning in cardiac image analysis. Medical Image Analysis, 58. p. 101535. DOI https://doi.org/10.1016/j.media.2019.101535
Abstract
Typically, a medical image offers spatial information on the anatomy (and pathology) modulated by imaging specific characteristics. Many imaging modalities including Magnetic Resonance Imaging (MRI) and Computed Tomography (CT) can be interpreted in this way. We can venture further and consider that a medical image naturally factors into some spatial factors depicting anatomy and factors that denote the imaging characteristics. Here, we explicitly learn this decomposed (disentangled) representation of imaging data, focusing in particular on cardiac images. We propose Spatial Decomposition Network (SDNet), which factorises 2D medical images into spatial anatomical factors and non-spatial modality factors. We demonstrate that this high-level representation is ideally suited for several medical image analysis tasks, such as semi-supervised segmentation, multi-task segmentation and regression, and image-to-image synthesis. Specifically, we show that our model can match the performance of fully supervised segmentation models, using only a fraction of the labelled images. Critically, we show that our factorised representation also benefits from supervision obtained either when we use auxiliary tasks to train the model in a multi-task setting (e.g. regressing to known cardiac indices), or when aggregating multimodal data from different sources (e.g. pooling together MRI and CT data). To explore the properties of the learned factorisation, we perform latent-space arithmetic and show that we can synthesise CT from MR and vice versa, by swapping the modality factors. We also demonstrate that the factor holding image specific information can be used to predict the input modality with high accuracy. Code will be made available at https://github.com/agis85/anatomy_modality_decomposition.
Item Type: | Article |
---|---|
Uncontrolled Keywords: | Disentangled representation learning; Cardiac magnetic resonance imaging; Semi-supervised segmentation; Multitask learning |
Divisions: | Faculty of Science and Health Faculty of Science and Health > Computer Science and Electronic Engineering, School of |
SWORD Depositor: | Unnamed user with email elements@essex.ac.uk |
Depositing User: | Unnamed user with email elements@essex.ac.uk |
Date Deposited: | 17 Jul 2020 09:23 |
Last Modified: | 30 Oct 2024 16:28 |
URI: | http://repository.essex.ac.uk/id/eprint/28146 |
Available files
Filename: MIA 2019.pdf
Licence: Creative Commons: Attribution-Noncommercial-No Derivative Works 3.0