We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

cs.SE

Change to browse by:

References & Citations

DBLP - CS Bibliography

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Computer Science > Software Engineering

Title: Nobody of the Crowd: An Empirical Investigation of Worker Communities in TopCoder

Abstract: Software crowdsourcing platforms employ extrinsic rewards such as rating or ranking systems to motivate workers. Such rating systems are noisy and provide limited knowledge about workers' preferences and performance. To develop better understanding of worker reliability and trustworthiness in software crowdsourcing, this paper reports an empirical study conducted on more than one year's real-world data from TopCoder, one of the leading software crowdsourcing platforms.
To do so, first, we create a bipartite network of active workers based on common task registrations. Then, we use the Clauset-Newman-Moore graph clustering algorithm to identify worker clusters in the network. Finally, we conduct an empirical evaluation to measure and analyze workers' behavior per identified community in the platform by workers' rating. More specifically, workers' behavior is analyzed based on their performances in terms of reliability, trustworthiness, and success; their preferences in terms of efficiency and elasticity; and strategies in terms of comfort, confidence, and deceitfulness. The main result of this study identified four communities of active workers: mixed-ranked, high-ranked, mid-ranked, and low-ranked. This study shows that the low-ranked community associates with the highest reliable workers with an average reliability of 25%, while the mixed-ranked community contains the most trustworthy workers with average trustworthiness of 16%. Such empirical evidence is beneficial to help exploring resourcing options while understanding the relations among unknown resources to improve task success.
Comments: 10 pages, 7 figure, 4 tables
Subjects: Software Engineering (cs.SE); Human-Computer Interaction (cs.HC); Social and Information Networks (cs.SI)
Cite as: arXiv:2107.02221 [cs.SE]
  (or arXiv:2107.02221v2 [cs.SE] for this version)

Submission history

From: Raz Saremi [view email]
[v1] Mon, 5 Jul 2021 18:41:01 GMT (664kb,D)
[v2] Mon, 18 Oct 2021 21:15:30 GMT (676kb,D)

Link back to: arXiv, form interface, contact.