Overview of the ImageCLEFmed 2021 Concept & Caption Prediction Task

The 2021 ImageCLEF concept detection and caption prediction task follows similar challenges that were already run from 2017-2020. The objective is to extract UMLS-concept annotations and/or captions from the image data that are then compared against the original text captions of the images. The used images are clinically relevant radiology images and the describing captions were created by medical experts. In the caption prediction task, lexical similarity with the original image captions is evaluated with the BLEU-score. In the concept detection task, UMLS (Unified Medical Language System) terms are extracted from the original text captions and compared against the predicted concepts in a multi-label way. The F1-score was used to assess the performance. The 2021 task has been conducted in collaboration with the Visual Question Answering task and used the same images. The task attracted a strong participation with 25 registered teams. In the end 10 teams submitted 75 runs for the two sub tasks. Results show that there is a variety of used techniques that can lead to good prediction results for the two tasks. In comparison to earlier competitions, more modern deep learning architectures like EfficientNets and Transformer-based architectures for text or images were used.