References & Citations
Computer Science > Computer Science and Game Theory
Title: The Price of Strategyproofing Peer Assessment
(Submitted on 25 Jan 2022 (this version), latest version 29 Aug 2022 (v3))
Abstract: Strategic behavior is a fundamental problem in a variety of real-world applications that require some form of peer assessment, such as peer grading of assignments, grant proposal review, conference peer review, and peer assessment of employees. Since an individual's own work is in competition with the submissions they are evaluating, they may provide dishonest evaluations to increase the relative standing of their own submission. This issue is typically addressed by partitioning the individuals and assigning them to evaluate the work of only those from different subsets. Although this method ensures strategyproofness, each submission may require a different type of expertise for effective evaluation. In this paper, we focus on finding an assignment of evaluators to submissions that maximizes assigned expertise subject to the constraint of strategyproofness. We analyze the price of strategyproofness: that is, the amount of compromise on the assignment quality required in order to get strategyproofness. We establish several polynomial-time algorithms for strategyproof assignment along with assignment-quality guarantees. Finally, we evaluate the methods on a dataset from conference peer review.
Submission history
From: Steven Jecmen [view email][v1] Tue, 25 Jan 2022 21:16:33 GMT (68kb,D)
[v2] Sat, 12 Feb 2022 20:02:27 GMT (282kb,D)
[v3] Mon, 29 Aug 2022 03:14:53 GMT (297kb,D)
Link back to: arXiv, form interface, contact.