References & Citations
Computer Science > Artificial Intelligence
Title: Better Peer Grading through Bayesian Inference
(Submitted on 2 Sep 2022 (v1), last revised 2 Dec 2022 (this version, v2))
Abstract: Peer grading systems aggregate noisy reports from multiple students to approximate a true grade as closely as possible. Most current systems either take the mean or median of reported grades; others aim to estimate students' grading accuracy under a probabilistic model. This paper extends the state of the art in the latter approach in three key ways: (1) recognizing that students can behave strategically (e.g., reporting grades close to the class average without doing the work); (2) appropriately handling censored data that arises from discrete-valued grading rubrics; and (3) using mixed integer programming to improve the interpretability of the grades assigned to students. We show how to make Bayesian inference practical in this model and evaluate our approach on both synthetic and real-world data obtained by using our implemented system in four large classes. These extensive experiments show that grade aggregation using our model accurately estimates true grades, students' likelihood of submitting uninformative grades, and the variation in their inherent grading error; we also characterize our models' robustness.
Submission history
From: Hedayat Zarkoob [view email][v1] Fri, 2 Sep 2022 19:10:53 GMT (9139kb,D)
[v2] Fri, 2 Dec 2022 07:30:52 GMT (9450kb,D)
Link back to: arXiv, form interface, contact.