Title: CodEval: Improving Student Success In Programming Assignments

Abstract: CodEval is a code evaluation tool that integrates with the Canvas Learning Management System to automatically evaluates students' work within a few minutes of the submission. This early feedback allows students to catch and correct problems in their submissions before their submission is graded and gives them a clear idea of the quality of their submission. CodEval handles the tedious aspects of grading, such as compiling and running tests, leaving graders more time to spend on the qualitative aspect of grading.
Before using CodEval, instructors would not have a clear view of the student's comprehension of the concept evaluated by the assignment until after the due date. CodeEval helps instructors identify and address the gaps in students' understanding and thus helps more students successfully complete the assignment.
We implemented CodEval using Python using the public Canvas API. Any instructor or grader for a Canvas course can use CodEval to automatically evaluate submissions for programming assignments. We developed a syntax to express requirements of submissions such as compilation parameters, inputs, outputs, command-line arguments, timeouts, exit codes, functions used, files generated, output validators, and more. We have made CodEval open source.
CodEval is an easy tool for students, graders, and instructors and seamlessly integrates with Canvas. We share our experience with using CodEval in two classes with a total of 90 students and multiple coding assignments.