We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

cs.SE

Change to browse by:

cs

References & Citations

DBLP - CS Bibliography

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Computer Science > Software Engineering

Title: Comparing Mutation Coverage Against Branch Coverage in an Industrial Setting

Abstract: The state-of-the-practice in software development is driven by constant change fueled by continuous integration servers. Such constant change demands for frequent and fully automated tests capable to detect faults immediately upon project build. As the fault detection capability of the test suite becomes so important, modern software development teams continuously monitor the quality of the test suite as well. However, it appears that the state-of-the-practice is reluctant to adopt strong coverage metrics (namely mutation coverage), instead relying on weaker kinds of coverage (namely branch coverage). In this paper, we investigate three reasons that prohibit the adoption of mutation coverage in a continuous integration setting: (1) the difficulty of its integration into the build system, (2) the perception that branch coverage is "good enough", and (3) the performance overhead during the build. Our investigation is based on a case study involving four open source systems and one industrial system. We demonstrate that mutation coverage reveals additional weaknesses in the test suite compared to branch coverage and that it is able to do so with an acceptable performance overhead during project build.
Subjects: Software Engineering (cs.SE)
Journal reference: International Journal of Software Tools and Technology Transfer 22, 365--388 (2020)
DOI: 10.1007/s10009-020-00567-y
Cite as: arXiv:2104.11767 [cs.SE]
  (or arXiv:2104.11767v1 [cs.SE] for this version)

Submission history

From: Ali Parsai [view email]
[v1] Fri, 23 Apr 2021 18:13:24 GMT (247kb,D)

Link back to: arXiv, form interface, contact.