Research Repository

Optimising crowdsourcing efficiency: Amplifying human computation with validation

Chamberlain, Jon and Kruschwitz, Udo and Poesio, Massimo (2018) 'Optimising crowdsourcing efficiency: Amplifying human computation with validation.' it - Information Technology, 60 (1). pp. 41-49. ISSN 2196-7032

Chamberlain2018Optimising_preprint.pdf - Accepted Version

Download (1MB) | Preview


Crowdsourcing has revolutionised the way tasks can be completed but the process is frequently inefficient, costing practitioners time and money. This research investigates whether crowdsourcing can be optimised with a validation process, as measured by four criteria: quality; cost; noise; and speed. A validation model is described, simulated and tested on real data from an online crowdsourcing game to collect data about human language. Results show that by adding an agreement validation (or a like/upvote) step fewer annotations are required, noise and collection time are reduced and quality may be improved.

Item Type: Article
Uncontrolled Keywords: Crowdsourcing; Empirical studies in interaction design; Interactive games; Social networks; Natural language processing
Subjects: Q Science > QA Mathematics > QA75 Electronic computers. Computer science
Divisions: Faculty of Science and Health
Faculty of Science and Health > Computer Science and Electronic Engineering, School of
SWORD Depositor: Elements
Depositing User: Elements
Date Deposited: 14 Sep 2018 16:02
Last Modified: 23 Sep 2022 19:22

Actions (login required)

View Item View Item