Risks and Rewards of Crowdsourcing Marketplaces
Chandler, J., Paolacci, G., & Mueller, P. (2013). Risks and rewards of crowdsourcing marketplaces. In Handbook of Human Computation (pp. 377-392). Springer New York.
30 Pages Posted: 5 Feb 2014
Date Written: February 4, 2014
Crowdsourcing has become an increasingly popular means of flexibly deploying large amounts of human computational power. The present chapter investigates the role of microtask labor marketplaces in managing human and hybrid human machine computing. Labor marketplaces offer many advantages that in combination allow human intelligence to be allocated across projects rapidly and efficiently and information to be transmitted effectively between market participants. Human computation comes with a set of challenges that are distinct from machine computation, including increased unsystematic error (e.g. mistakes) and systematic error (e.g. cognitive biases), both of which can be exacerbated when motivation is low, incentives are misaligned, and task requirements are poorly communicated. We provide specific guidance to how to ameliorate these issues through task design, workforce selection, data cleaning and aggregation.
Suggested Citation: Suggested Citation