Research Repository

Using Crowdsourcing for Multi-label Biomedical Compound Figure Annotation

Garcia Seco De Herrera, Alba and Schaer, Roger and Antani, Sameer and Müller, Henning (2016) Using Crowdsourcing for Multi-label Biomedical Compound Figure Annotation. In: First International Workshop, LABELS 2016, and Second International Workshop, DLMIA 2016, Held in Conjunction with MICCAI 2016, 2016-10-21 - ?, Athens, Greece.

[img]
Preview
Text
LABELS_Alba.pdf - Accepted Version

Download (780kB) | Preview

Abstract

Information analysis or retrieval for images in the biomedical literature needs to deal with a large amount of compound figures (figures containing several subfigures), as they constitute probably more than half of all images in repositories such as PubMed Central, which was the data set used for the task. The ImageCLEFmed benchmark proposed among other tasks in 2015 and 2016 a multi-label classification task, which aims at evaluating the automatic classification of figures into 30 image types. This task was based on compound figures and thus the figures were distributed to participants as compound figures but also in a separated form. Therefore, the generation of a gold standard was required, so that algorithms of participants can be evaluated and compared. This work presents the process carried out to generate the multi-labels of ∼2650 compound figures using a crowdsourcing approach. Automatic algorithms to separate compound figures into subfigures were used and the results were then validated or corrected via crowdsourcing. The image types (MR, CT, X–ray, ...) were also annotated by crowdsourcing including detailed quality control. Quality control is necessary to insure quality of the annotated data as much as possible. ∼625 h were invested with a cost of ∼870$.

Item Type: Conference or Workshop Item (Paper)
Additional Information: Published proceedings: First International Workshop, LABELS 2016, and Second International Workshop, DLMIA 2016, Held in Conjunction with MICCAI 2016, Athens, Greece, October 21, 2016, Proceedings. Part of the Lecture Notes in Computer Science book series (LNCS, volume 10008)
Divisions: Faculty of Science and Health > Computer Science and Electronic Engineering, School of
Depositing User: Elements
Date Deposited: 22 Jan 2020 13:28
Last Modified: 22 Jan 2020 14:15
URI: http://repository.essex.ac.uk/id/eprint/22220

Actions (login required)

View Item View Item