Research Repository

Overview of the ImageCLEFmed 2021 Concept & Caption Prediction Task

Pelka, Obioma and Abacha, Asma Ben and Garcia Seco De Herrera, Alba and Jacutprakart, Janadhip and Friedrich, Cristoph M and Müller, Henning (2021) Overview of the ImageCLEFmed 2021 Concept & Caption Prediction Task. In: CLEF 2021 - Conference and Labs of the Evaluation Forum, 2021-09-21 - 2022-01-24, Bucharest, Romania.

ImageCLEF_paper-111.pdf - Published Version
Available under License Creative Commons Attribution.

Download (714kB) | Preview


The 2021 ImageCLEF concept detection and caption prediction task follows similar challenges that were already run from 2017-2020. The objective is to extract UMLS-concept annotations and/or captions from the image data that are then compared against the original text captions of the images. The used images are clinically relevant radiology images and the describing captions were created by medical experts. In the caption prediction task, lexical similarity with the original image captions is evaluated with the BLEU-score. In the concept detection task, UMLS (Unified Medical Language System) terms are extracted from the original text captions and compared against the predicted concepts in a multi-label way. The F1-score was used to assess the performance. The 2021 task has been conducted in collaboration with the Visual Question Answering task and used the same images. The task attracted a strong participation with 25 registered teams. In the end 10 teams submitted 75 runs for the two sub tasks. Results show that there is a variety of used techniques that can lead to good prediction results for the two tasks. In comparison to earlier competitions, more modern deep learning architectures like EfficientNets and Transformer-based architectures for text or images were used.

Item Type: Conference or Workshop Item (Paper)
Divisions: Faculty of Science and Health
Faculty of Science and Health > Computer Science and Electronic Engineering, School of
SWORD Depositor: Elements
Depositing User: Elements
Date Deposited: 27 Sep 2022 11:40
Last Modified: 27 Sep 2022 11:40

Actions (login required)

View Item View Item