The Influence of Test Suite Properties on Automated Grading

by Benjamin Clegg, Phil McMinn, and Gordon Fraser

International Conference on Software Engineering Education and Training (CSEE&T 2020)



Automated grading allows for the scalable assessment of large programming courses, often using test cases to determine the correctness of students’ programs. However, test suites can vary in multiple ways, such as quality, size, and coverage. In this paper, we investigate how much test suites with varying properties can impact generated grades, and how these properties cause this impact. We conduct a study on artificial faulty programs that simulate students’ programming mistakes and test suites generated from manually written tests. We find that these test suites generate greatly varying grades, with the standard deviation of grades for each fault typically ... [more]


Reference

Benjamin Clegg, Phil McMinn, and Gordon Fraser. The Influence of Test Suite Properties on Automated Grading. International Conference on Software Engineering Education and Training (CSEE&T 2020), 2020


Bibtex Entry
@inproceedings{Clegg2020,
  author    = "Clegg, Benjamin and McMinn, Phil and Fraser, Gordon",
  title     = "The Influence of Test Suite Properties on Automated Grading",
  booktitle = "International Conference on Software Engineering Education and Training (CSEE\&T 2020)",
  year      = "2020"
}