Advanced deep learning towards improving prediction outcomes on medical imaging data and radiology reports

This study presents two novel methodologies to predict the outcomes of medical reports and their corresponding visual data. The increasing complexity and volume of medical imaging necessitate advanced computational techniques that can enhance medical data interpretation. The primary objective of our research is to develop sophisticated tools that improve automated diagnostic accuracy and contribute to efficient clinical decision-making processes. Two robust approaches, a) Feature extraction for Content-based image retrieval with kNN and b) Multi-label Classification, are used, applying these DenseNet-121 and EfficientNet architectures. In the context of our research, the Feature extraction for Content-based image retrieval with kNN approach demonstrated significant potential, with the highest F1 scores achieved as follows: DenseNet-121 with Cosine similarity recorded an F1 score of 0.469, EfficientNet B0 with Bray-Curtis scored 0.451, Efficient-NetB0 with Cosine achieved 0.440, EfficientNetB0 with Canberra reached 0.423, and EfficientNet B3 with Canberra obtained a score of 0.355. These findings underscore the effectiveness of different distance metrics in optimising retrieval tasks within the medical imaging domain. On the other hand, the Multi-label Classification method showed its highest performance using the DenseNet-121 model, which achieved an F1 score of 0.412. This result highlights the model’s robustness in managing the complexities associated with multi-label data, which often reflects the multifaceted nature of medical diagnoses. However, our exploration identified several challenges that may have contributed to the models’ under-performance. One significant challenge is that no modality was provided in the dataset, which, as a consequence, furthers the issue with selective label assignment in Multi-label Classification, which can lead to ambiguity and inconsistency during the training phase. This inconsistency can adversely affect the model’s ability to generalise across datasets, impacting its overall predictive accuracy. Despite these challenges, the results obtained from our experiments establish a robust baseline for future research in automated medical image analysis.