Collusion-proof result inference

In traditional crowdsourcing, workers are expected to provide independent answers to tasks so as to ensure the diversity of answers. However, recent studies show that the crowd is not a collection of independent workers, but instead that workers communicate and collaborate with each other. To pursue more rewards with little effort, some workers may collude to provide repeated answers, which will damage the quality of the aggregated results. Nonetheless, there are few efforts considering the negative impact of collusion on result inference in crowdsourcing. In this paper, we are specially concerned with the Collusion-Proof result inference problem for general crowdsourcing tasks in public platforms. To that end, we design a metric, the worker performance change rate, to identify the colluded answers by computing the difference of the mean worker performance before and after removing the repeated answers. Then we incorporate the collusion detection result into existing result inference methods to guarantee the quality of the aggregated results even with the occurrence of collusion behaviors. With real-world and synthetic datasets, we conducted an extensive set of evaluations of our approach. The experimental results demonstrate the superiority of our approach in comparison with the state-of-the-art methods.

Resource

Access the Research publication

Collusion-proof result inference
Chen, P., Sun, H. and Huai, J. (2018), 'Collusion-proof result inference,' Journal of Computer Science and Technology 33(2), pp. 351-365.

Metadata

  • no specific sector focus
  • Other
  • 2018
  • Research publication
  • challenges, work organisation
  • English
  • Journal of Computer Science and Technology (Publisher)
  • Quantitative research
  • Subscription
Disclaimer  —  Eurofound aims to keep the information in this database up to date and accurate. If errors are brought to our attention, we will try to correct them. However, Eurofound bears no responsibility or liability whatsoever with regard to the accuracy or content of this database or of external links over which Eurofound services have no control and for which Eurofound assumes no responsibility.