Reducing Qualitative Human Oracle Costs associated with Automatically Generated Test Data

by Phil McMinn, Mark Stevenson, and Mark Harman

International Workshop on Software Test Output Validation (STOV 2010)



Due to the frequent non-existence of an automated oracle, test cases are often evaluated manually in practice. However, this fact is rarely taken into account by automatic test data generators, which seek to maximise a program’s structural coverage only. The test data produced tends to be of a poor fit with the program’s operational profile. As a result, each test case takes longer for a human to check, because the scenarios that arbitrary-looking data represent require time and effort to understand. This short paper proposes methods to extracting knowledge from programmers, source code and documentation and its incorporation into the ... [more]


Reference

Phil McMinn, Mark Stevenson, and Mark Harman. Reducing Qualitative Human Oracle Costs associated with Automatically Generated Test Data. International Workshop on Software Test Output Validation (STOV 2010), pp. 1–4, 2010


Bibtex Entry
@inproceedings{McMinn2010,
  author    = "McMinn, Phil and Stevenson, Mark and Harman, Mark",
  title     = "Reducing Qualitative Human Oracle Costs associated with Automatically Generated Test Data",
  booktitle = "International Workshop on Software Test Output Validation (STOV 2010)",
  pages     = "1--4",
  year      = "2010",
  publisher = "ACM"
}