Gadiraju, U., Kawase, R., Dietze, S. et al. (1 more author) (2015) Understanding Malicious Behavior in Crowdsourcing Platforms: The Case of Online Surveys. In: Begole, B., Kim, J., Inkpen, K. and Woo, W., (eds.) CHI '15 Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems. 33rd Annual ACM Conference on Human Factors in Computing Systems, 18-23 Apr 2015, Seoul, Korea. ACM , pp. 1631-1640. ISBN 978-1-4503-3145-6
Abstract
Crowdsourcing is increasingly being used as a means to tackle problems requiring human intelligence. With the ever-growing worker base that aims to complete microtasks on crowdsourcing platforms in exchange for financial gains, there is a need for stringent mechanisms to prevent exploitation of deployed tasks. Quality control mechanisms need to accommodate a diverse pool of workers, exhibiting a wide range of behavior. A pivotal step towards fraud-proof task design is understanding the behavioral patterns of microtask workers. In this paper, we analyze the prevalent malicious activity on crowdsourcing platforms and study the behavior exhibited by trustworthy and untrustworthy workers, particularly on crowdsourced surveys. Based on our analysis of the typical malicious activity, we define and identify different types of workers in the crowd, propose a method to measure malicious activity, and finally present guidelines for the efficient design of crowdsourced surveys.
Metadata
Item Type: | Proceedings Paper |
---|---|
Authors/Creators: |
|
Editors: |
|
Copyright, Publisher and Additional Information: | © 2015 ACM. This is an author produced version of a paper subsequently published in CHI '15 Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems. Uploaded in accordance with the publisher's self-archiving policy. |
Dates: |
|
Institution: | The University of Sheffield |
Academic Units: | The University of Sheffield > Faculty of Social Sciences (Sheffield) > Information School (Sheffield) |
Depositing User: | Symplectic Sheffield |
Date Deposited: | 02 Mar 2016 13:10 |
Last Modified: | 19 Dec 2022 13:33 |
Published Version: | https://dx.doi.org/10.1145/2702123.2702443 |
Status: | Published |
Publisher: | ACM |
Refereed: | Yes |
Identification Number: | 10.1145/2702123.2702443 |
Open Archives Initiative ID (OAI ID): | oai:eprints.whiterose.ac.uk:95877 |