Publications

Jump to:

2022 2021 2020 2019 2018 2017 2016 2015 2014 2013 2012 2011 2010 2009 2008 2007 2006 2005 2004 2003

2022

Owain Parry, Michael Hilton, Gregory M. Kapfhammer, and Phil McMinn
Surveying the Developer Experience of Flaky Tests
International Conference on Software Engineering — Software Engineering In Practice Track (ICSE-SEIP 2022).
Owain Parry, Michael Hilton, Gregory M. Kapfhammer, and Phil McMinn
A Survey of Flaky Tests
ACM Transactions on Software Engineering and Methodology.
Owain Parry, Michael Hilton, Gregory M. Kapfhammer, and Phil McMinn
Evaluating Features for Machine Learning Detection of Order- and Non-Order-Dependent Flaky Tests
International Conference on Software Testing, Verification and Validation (ICST 2022).
Ibrahim Althomali, Gregory M. Kapfhammer, and Phil McMinn
Automated Repair of Responsive Web Page Layouts
International Conference on Software Testing, Verification and Validation (ICST 2022).
Ben Clegg, Phil McMinn, and Gordon Fraser
Diagnosability, Adequacy & Size: How Test Suites Impact Autograding
International Conference on (HICSS 2022).

2021

Ben Clegg, Maria-Cruz Villa-Uriol, Phil McMinn, and Gordon Fraser
Gradeer: An Open-Source Modular Hybrid Grader
International Conference on Software Engineering (ICSE) — Joint Track on Software Engineering Education and Training (JSEET).
Ibrahim Althomali, Gregory M. Kapfhammer, and Phil McMinn
Automated Visual Classification of DOM-based Presentation Failure Reports for Responsive Web Pages
Software Testing, Verification and Reliability.
Sonal Mahajan, Abdulmajeed Alameer, Phil McMinn, and William G.J. Halfond
Effective Automated Repair of Internationalization Presentation Failures in Web Applications Using Style Similarity Clustering and Search-Based Techniques
Software Testing, Verification and Reliability.
Ben Clegg, Phil McMinn, and Gordon Fraser
An Empirical Study to Determine if Mutants Can Effectively Simulate Students’ Programming Mistakes to Increase Tutors’ Confidence in Autograding
ACM SIGCSE Technical Symposium on Computer Science Education (SIGCSE 2021).

2020

Dave Binkley, James Glenn, Abdullah Alsharif, and Phil McMinn
An Investigation into the Effect of Control and Data Dependence Chain Length on Predicate Testability
International Working Conference on Source Code Analysis & Manipulation (SCAM 2020).
Thomas A. Walsh, Gregory M. Kapfhammer, and Phil McMinn
Automatically Identifying Potential Regressions in the Layout of Responsive Web Pages
Software Testing, Verification and Reliability.
Benjamin Clegg, Phil McMinn, and Gordon Fraser
The Influence of Test Suite Properties on Automated Grading
International Conference on Software Engineering Education and Training (CSEE&T 2020).
Abdullah Alsharif, Gregory M. Kapfhammer, and Phil McMinn
Hybrid Methods for Reducing Database Schema Test Suites: Experimental Insights from Computational and Human Studies
International Conference on Automated Software Test (AST 2020).
Owain Parry, Michael Hilton, Gregory M. Kapfhammer, and Phil McMinn
Flake It ’Till You Make It: Using Automated Repair to Induce and Fix Latent Test Flakiness
International Workshop on Automated Program Repair (APR 2020).
Abdullah Alsharif, Gregory M. Kapfhammer, and Phil McMinn
STICCER: Fast and Effective Database Test Suite Reduction Through Merging of Similar Test Cases
International Conference on Software Testing, Verification and Validation (ICST 2020).

2019

Abdullah Alsharif, Gregory M. Kapfhammer, and Phil McMinn
What Factors Make SQL Test Cases Understandable For Testers? A Human Study of Automatic Test Data Generation Techniques
International Conference on Software Maintenance and Evolution (ICSME 2019).
David Paterson, José Campos, Rui Abreu, Gregory M. Kapfhammer, Gordon Fraser, and Phil McMinn
An Empirical Study on the Use of Defect Prediction for Test Case Prioritization
International Conference on Software Testing, Verification and Validation (ICST 2019).
Ibrahim Althomali, Gregory M. Kapfhammer, and Phil McMinn
Automatic Visual Verification of Layout Failures in Responsively Designed Web Pages
International Conference on Software Testing, Verification and Validation (ICST 2019).
Benjamin Clegg, Gordon Fraser, Siobhán North, and Phil McMinn
Simulating Student Mistakes to Evaluate the Fairness of Automated Grading
International Conference on Software Engineering: Software Engineering Education and Training (ICSE-SEET 2019).
Phil McMinn, Chris J. Wright, Colton J. McCurdy, and Gregory M. Kapfhammer
Automatic Detection and Removal of Ineffective Mutants for the Mutation Analysis of Relational Database Schemas
IEEE Transactions on Software Engineering.

2018

M. Mohammad Almasi, Hadi Hemmati, Gordon Fraser, Phil McMinn, and Janis Benefelds
Search-Based Detection of Deviation Failures in the Migration of Legacy Spreadsheet Applications
International Conference on Software Testing and Analysis (ISSTA 2018).
David Paterson, Gregory M. Kapfhammer, Gordon Fraser, and Phil McMinn
Using Controlled Numbers of Real Faults and Mutants to Empirically Evaluate Coverage-Based Test Case Prioritization
International Workshop on Automated Software Test (AST 2018).
Byeonghyeon You, Junhwi Kim, Minhyuk Kwon, Phil McMinn, and Shin Yoo
Evaluation of CAVM, Austin, and CodeScroll for Test Data Generation for C
Korea Conference on Software Engineering (KCSE 2018).
Abdullah Alsharif, Gregory M. Kapfhammer, and Phil McMinn
DOMINO: Fast and Effective Test Data Generation for Relational Database Schemas
International Conference on Software Testing, Validation and Verification (ICST 2018).
Sonal Mahajan, Abdulmajeed Alameer, Phil McMinn, and William G.J. Halfond
Automated Repair of Internationalization Failures Using Style Similarity Clustering and Search-Based Techniques
International Conference on Software Testing, Validation and Verification (ICST 2018).
Sina Shamshiri, José Miguel Rojas, Luca Gazzola, Gordon Fraser, Phil McMinn, Leonardo Mariani, and Andrea Arcuri
Random or Evolutionary Search for Object-Oriented Test Suite Generation?
Software Testing, Verification and Reliability.
Sonal Mahajan, Negarsadat Abolhassani, Phil McMinn, and William G.J. Halfond
Automated Repair of Mobile Friendly Problems in Web Pages
International Conference on Software Engineering (ICSE 2018).
Mathew Hall, Neil Walkinshaw, and Phil McMinn
Effectively Incorporating Expert Knowledge in Automated Software Remodularisation
IEEE Transactions on Software Engineering.

2017

Junhwi Kim, Byeonghyeon You, Minhyuk Kwon, Phil McMinn, and Shin Yoo
Evaluating CAVM: A New Search-Based Test Data Generation Tool for C
International Symposium on Search-Based Software Engineering (SSBSE 2017).
Thomas A. Walsh, Gregory M. Kapfhammer, and Phil McMinn
Automated Layout Failure Detection for Responsive Web Pages Without an Explicit Oracle
International Conference on Software Testing and Analysis (ISSTA 2017).
Thomas A. Walsh, Gregory M. Kapfhammer, and Phil McMinn
ReDeCheck: An Automatic Layout Failure Checking Tool for Responsively Designed Web Pages
International Conference on Software Testing and Analysis (ISSTA 2017).
Sonal Mahajan, Abdulmajeed Alameer, Phil McMinn, and William G.J. Halfond
Automated Repair of Layout Cross Browser Issues Using Search-Based Techniques
International Conference on Software Testing and Analysis (ISSTA 2017).
Sonal Mahajan, Abdulmajeed Alameer, Phil McMinn, and William G.J. Halfond
XFix: Automated Tool for Repair of Layout Cross Browser Issues
International Conference on Software Testing and Analysis (ISSTA 2017).
Sina Shamshiri, José Campos, Gordon Fraser, and Phil McMinn
Disposable Testing: Avoiding Maintenance of Generated Unit Tests by Throwing Them Away
International Conference on Software Engineering (ICSE 2017) Companion Volume.

2016

Phil McMinn, Chris J. Wright, Cody Kinneer, Colton J. McCurdy, Michael Camara, and Gregory M. Kapfhammer
SchemaAnalyst: Search-Based Test Data Generation for Relational Database Schemas
International Conference on Software Maintenance and Evolution (ICSME 2016).
Colton J. McCurdy, Phil McMinn, and Gregory M. Kapfhammer
mrstudyr: Retrospectively Studying the Effectiveness of Mutant Reduction Techniques
International Conference on Software Maintenance and Evolution (ICSME 2016).
Phil McMinn, and Gregory M. Kapfhammer
AVMf: An Open-Source Framework and Implementation of the Alternating Variable Method
International Symposium on Search-Based Software Engineering (SSBSE 2016).
Phil McMinn, Gregory M. Kapfhammer, and Chris J. Wright
Virtual Mutation Analysis of Relational Database Schemas
International Workshop on Automated Software Test (AST 2016).
Gregory M. Kapfhammer, Chris J. Wright, and Phil McMinn
Hitchhikers Need Free Vehicles! Shared Repositories for Statistical Analysis in SBST
International Workshop on Search-Based Software Testing (SBST 2016).
Phil McMinn, Mark Harman, Gordon Fraser, and Gregory M. Kapfhammer
Automated Search for Good Coverage Criteria: Moving from Code Coverage to Fault Coverage Through Search-Based Software Engineering
International Workshop on Search-Based Software Testing (SBST 2016).

2015

Phil McMinn, Chris J. Wright, and Gregory M. Kapfhammer
The Effectiveness of Test Coverage Criteria for Relational Database Schema Integrity Constraints
ACM Transactions on Software Engineering and Methodology.
Thomas A. Walsh, Phil McMinn, and Gregory M. Kapfhammer
Automatic Detection of Potential Layout Faults Following Changes to Responsive Web Pages
International Conference on Automated Software Engineering (ASE 2015).
Sina Shamshiri, Rene Just, José Miguel Rojas, Gordon Fraser, Phil McMinn, and Andrea Arcuri
Do Automatically Generated Unit Tests Find Real Faults? An Empirical Study of Effectiveness and Challenges
International Conference on Automated Software Engineering (ASE 2015).
Phil McMinn, Chris J. Wright, and Gregory M. Kapfhammer
An Analysis of the Effectiveness of Different Coverage Criteria for Testing Relational Database Schema Integrity Constraints
Technical Report (Department of Computer Science, University of Sheffield).
Cody Kinneer, Gregory M. Kapfhammer, Chris J. Wright, and Phil McMinn
Automatically Evaluating the Efficiency of Search-Based Test Data Generation for Relational Database Schemas
International Conference on Software Engineering and Knowledge Engineering (SEKE 2015).
Cody Kinneer, Gregory M. Kapfhammer, Chris J. Wright, and Phil McMinn
EXPOSE: Inferring Worst-case Time Complexity by Automatic Empirical Study
International Conference on Software Engineering and Knowledge Engineering (SEKE 2015).
Sina Shamshiri, José Miguel Rojas, Gordon Fraser, and Phil McMinn
Random or Genetic Algorithm Search for Object-Oriented Test Suite Generation?
Genetic and Evolutionary Computation Conference (GECCO 2015).
Joseph Kempka, Phil McMinn, and Dirk Sudholt
Design and Analysis of Different Alternating Variable Searches for Search-Based Software Testing
Theoretical Computer Science.
Earl T. Barr, Mark Harman, Phil McMinn, Muzammil Shahbaz, and Shin Yoo
The Oracle Problem in Software Testing: A Survey
IEEE Transactions on Software Engineering.
Gordon Fraser, Matt Staats, Phil McMinn, Andrea Arcuri, and Frank Padberg
Does Automated Unit Test Generation Really Help Software Testers? A Controlled Empirical Study
ACM Transactions on Software Engineering and Methodology.
Gordon Fraser, Andrea Arcuri, and Phil McMinn
A Memetic Algorithm for Whole Test Suite Generation
Journal of Systems and Software.
Muzammil Shahbaz, Phil McMinn, and Mark Stevenson
Automatic Generation of Valid and Invalid Test Data for String Validation Routines Using Web Searches and Regular Expressions
Science of Computer Programming.

2014

Chris J. Wright, Gregory M. Kapfhammer, and Phil McMinn
The Impact Of Equivalent, Redundant And Quasi Mutants On Database Schema Mutation Analysis
International Conference on Quality Software (QSIC 2014).
Mathew Hall, Muhammad Khojaye, Neil Walkinshaw, and Phil McMinn
Establishing the Source Code Disruption Caused by Automated Remodularisation Tools
International Conference on Software Maintenance and Evolution (ICSME 2014).

2013

Mark Harman, Phil McMinn, Muzammil Shahbaz, and Shin Yoo
A Comprehensive Survey of Trends in Oracles for Software Testing
Technical Report (Department of Computer Science, University of Sheffield).
Gordon Fraser, Matt Staats, Phil McMinn, Andrea Arcuri, and Frank Padberg
Does Automated White-Box Test Generation Really Help Software Testers?
International Symposium on Software Testing and Analysis (ISSTA 2013).
Joseph Kempka, Phil McMinn, and Dirk Sudholt
A Theoretical Runtime and Empirical Analysis of Different Alternating Variable Searches for Search-Based Testing
Genetic and Evolutionary Computation Conference (GECCO 2013).
Gordon Fraser, Andrea Arcuri, and Phil McMinn
Test Suite Generation with Memetic Algorithms
Genetic and Evolutionary Computation Conference (GECCO 2013).
Chris J. Wright, Phil McMinn, and Julio Gallardo
Towards the Automatic Identification of Faulty Multi-Agent Based Simulation Runs Using MASTER
Multi-Agent-Based Simulation XIII — International Workshop on Multi-Agent Simulation (MABS 2012).
Sina Shamshiri, Gordon Fraser, Phil McMinn, and Alex Orso
Search-Based Propagation of Regression Faults in Automated Regression Testing
International Workshop on Regression Testing (Regression 2013).
Saswat Anand, Edmund Burke, Tsong Yueh Chen, John Clark, Myra B. Cohen, Wolfgang Grieskamp, Mark Harman, Mary Jean Harrold, and Phil McMinn
An Orchestrated Survey on Automated Software Test Case Generation
Journal of Systems and Software.
Chris J. Wright, Gregory M. Kapfhammer, and Phil McMinn
Efficient Mutation Analysis of Relational Database Structure Using Mutant Schemata and Parallelisation
International Workshop on Mutation Analysis (Mutation 2013).
Sheeva Afshan, Phil McMinn, and Mark Stevenson
Evolving Readable String Test Inputs Using a Natural Language Model to Reduce Human Oracle Cost
International Conference on Software Testing, Verification and Validation (ICST 2013).
Gregory M. Kapfhammer, Phil McMinn, and Chris J. Wright
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
International Conference on Software Testing, Verification and Validation (ICST 2013).
Phil McMinn
An Identification of Program Factors that Impact Crossover Performance in Evolutionary Test Input Generation for the Branch Coverage of C Programs
Information and Software Technology.

2012

Mathew Hall, Neil Walkinshaw, and Phil McMinn
Supervised Software Modularisation
International Conference on Software Maintenance (ICSM 2012).
Mike Holcombe, Salem Adra, Mesude Bicak, Shawn Chin, Simon Coakley, Alison I. Graham, Jeffrey Green, Chris Greenough, Duncan Jackson, Mariam Kiran, Sheila MacNeil, Afsaneh Maleki-Dizaji, Phil McMinn, Mark Pogson, Robert Poole, Eva Qwarnstrom, Francis Ratnieks, Matthew D. Rolfe, Rod Smallwood, Tao Sun, and David Worth
Modelling complex biological systems using an agent-based approach
Integrative Biology.
Phil McMinn, Mark Harman, Youssef Hassoun, Kiran Lakhotia, and Joachim Wegener
Input Domain Reduction through Irrelevant Variable Removal and its Effect on Local, Global and Hybrid Search-Based Structural Test Data Generation
IEEE Transactions on Software Engineering.
Phil McMinn, Muzammil Shahbaz, and Mark Stevenson
Search-Based Test Input Generation for String Data Types Using the Results of Web Queries
International Conference on Software Testing, Verification and Validation (ICST 2012).
Muzammil Shahbaz, Phil McMinn, and Mark Stevenson
Automated Discovery of Valid Test Strings from the Web using Dynamic Regular Expressions Collation and Natural Language Processing
International Conference on Quality Software (QSIC 2012).

2011

Salem Adra, Mariam Kiran, Phil McMinn, and Neil Walkinshaw
A Multiobjective Optimisation Approach for Dynamic Inference and Refinement of Agent-Based Model Specifications
Congress on Evolutionary Computation (CEC 2011).
Sheeva Afshan, and Phil McMinn
An Investigation into Qualitative Human Oracle Costs
Psychology of Programming Interest Group Annual Workshop (PPIG 2011).
Arthur Baars, Mark Harman, Youssef Hassoun, Kiran Lakhotia, Phil McMinn, Paolo Tonella, and Tanja Vos
Symbolic Search-Based Testing
International Conference on Automated Software Engineering (ASE 2011).
Mark Harman, Phil McMinn, Jerffeson Teixeira Souza, and Shin Yoo
Search-Based Software Engineering: Techniques, Taxonomy, Tutorial
Empirical Software Engineering and Verification (Lecture Notes in Computer Science).
Phil McMinn
Search-Based Software Testing: Past, Present and Future
International Workshop on Search-Based Software Testing (SBST 2011).

2010

Salem Adra, and Phil McMinn
Mutation Operators for Agent-Based Models
International Workshop on Mutation Analysis (Mutation 2010).
Sheeva Afshan, Phil McMinn, and Neil Walkinshaw
Using Dictionary Compression Algorithms to Identify Phases in Program Traces
Technical Report (Department of Computer Science, University of Sheffield).
Mathew Hall, Phil McMinn, and Neil Walkinshaw
Superstate Identification for State Machines Using Search-Based Clustering
Genetic and Evolutionary Computation Conference (GECCO 2010).
Mark Harman, and Phil McMinn
A Theoretical and Empirical Study of Search Based Testing: Local, Global and Hybrid Search
IEEE Transactions on Software Engineering.
Mark Harman, Sung Gon Kim, Kiran Lakhotia, Phil McMinn, and Shin Yoo
Optimizing for the Number of Tests Generated in Search Based Test Data Generation with an Application to the Oracle Cost Problem
International Workshop on Search-Based Software Testing (SBST 2010).
Kiran Lakhotia, Phil McMinn, and Mark Harman
An Empirical Investigation Into Branch Coverage for C Programs Using CUTE and AUSTIN
Journal of Systems and Software.
Phil McMinn
How Does Program Structure Impact the Effectiveness of the Crossover Operator in Evolutionary Testing?
International Symposium on Search-Based Software Engineering (SSBSE 2010).
Phil McMinn, Mark Stevenson, and Mark Harman
Reducing Qualitative Human Oracle Costs associated with Automatically Generated Test Data
International Workshop on Software Test Output Validation (STOV 2010).
Neil Walkinshaw, Sheeva Afshan, and Phil McMinn
Using Compression Algorithms to Support the Comprehension of Program Traces
International Workshop on Dynamic Analysis (WODA 2010).

2009

Kiran Lakhotia, Phil McMinn, and Mark Harman
Automated Test Data Generation for Coverage: Haven’t We Solved This Problem Yet?
Testing: Academic and Industrial Conference — Practice And Research Techniques (TAIC PART 2009).
Phil McMinn, David Binkley, and Mark Harman
Empirical Evaluation of a Nesting Testability Transformation for Evolutionary Testing
ACM Transactions on Software Engineering and Methodology.
Phil McMinn
Search-Based Failure Discovery using Testability Transformations to Generate Pseudo-Oracles
Genetic and Evolutionary Computation Conference (GECCO 2009).

2008

Mark Harman, André Baresel, David Binkley, Rob Hierons, Lin Hu, Bogdan Korel, Phil McMinn, and Marc Roper
Testability Transformation — Program Transformation to Improve Testability
Formal Methods and Testing (Lecture Notes in Computer Science).
Mariam Kiran, Simon Coakley, Neil Walkinshaw, Phil McMinn, and Mike Holcombe
Validation and Discovery from Computational Biology Models
BioSystems.
Kiran Lakhotia, Mark Harman, and Phil McMinn
Handling Dynamic Data Structures in Search-Based Testing
Genetic and Evolutionary Computation Conference (GECCO 2008).
Tao Sun, Phil McMinn, Mike Holcombe, Rod Smallwood, and Sheila MacNeil
Agent Based Modelling Helps in Understanding the Rules by Which Fibroblasts Support Keratinocyte Colony Formation
PLoS ONE.

2007

Mark Harman, Youssef Hassoun, Kiran Lakhotia, Phil McMinn, and Joachim Wegener
The Impact of Input Domain Reduction on Search-Based Test Data Generation
Joint meeting of the European Software Engineering Conference and the ACM SIGSOFT Symposium on the Foundations of Software Engineering (ESEC/FSE 2007).
Mark Harman, and Phil McMinn
A Theoretical and Empirical Analysis of Evolutionary Testing and Hill Climbing for Structural Test Data Generation
International Symposium on Software Testing and Analysis (ISSTA 2007).
Mark Harman, Kiran Lakhotia, and Phil McMinn
A Multi-Objective Approach to Search-Based Test Data Generation
Genetic and Evolutionary Computation Conference (GECCO 2007).
Phil McMinn
IGUANA: Input Generation Using Automated Novel Algorithms. A Plug and Play Research Tool
Technical Report (Department of Computer Science, University of Sheffield).
Tao Sun, Phil McMinn, Simon Coakley, Mike Holcombe, Rod Smallwood, and Sheila MacNeil
An Integrated Systems Biology Approach to Understanding the Rules of Keratinocyte Colony Formation
Journal of the Royal Society Interface.

2006

Phil McMinn, and Mike Holcombe
Evolutionary Testing Using an Extended Chaining Approach
Evolutionary Computation.
Phil McMinn, Mark Harman, David Binkley, and Paolo Tonella
The Species per Path Approach to Search-Based Software Test Data Generation
International Symposium on Software Testing and Analysis (ISSTA 2006).

2005

Phil McMinn, David Binkley, and Mark Harman
Testability Transformation for Efficient Automated Test Data Search in the Presence of Nesting
UK Software Testing Workshop (UKTest 2005).
Phil McMinn, and Mike Holcombe
Evolutionary Testing of State-Based Programs
Genetic and Evolutionary Computation Conference (GECCO 2005).
Phil McMinn
Evolutionary Search for Test Data in the Presence of State Behaviour
PhD thesis (The University of Sheffield).

2004

Phil McMinn
Search-Based Software Test Data Generation: A Survey
Software Testing, Verification and Reliability.
Phil McMinn, and Mike Holcombe
Hybridizing Evolutionary Testing with the Chaining Approach
Genetic and Evolutionary Computation Conference (GECCO 2004).

2003

Phil McMinn, and Mike Holcombe
The State Problem for Evolutionary Testing
Genetic and Evolutionary Computation Conference (GECCO 2003).