We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:


Current browse context:


Change to browse by:


References & Citations

DBLP - CS Bibliography


(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Computer Science > Software Engineering

Title: An Empirical Analysis of UI-based Flaky Tests

Authors: Alan Romano (1), Zihe Song (2), Sampath Grandhi (2), Wei Yang (2), Weihang Wang (1) ((1) University at Buffalo, SUNY, (2) University of Texas at Dallas)
Abstract: Flaky tests have gained attention from the research community in recent years and with good reason. These tests lead to wasted time and resources, and they reduce the reliability of the test suites and build systems they affect. However, most of the existing work on flaky tests focus exclusively on traditional unit tests. This work ignores UI tests that have larger input spaces and more diverse running conditions than traditional unit tests. In addition, UI tests tend to be more complex and resource-heavy, making them unsuited for detection techniques involving rerunning test suites multiple times.
In this paper, we perform a study on flaky UI tests. We analyze 235 flaky UI test samples found in 62 projects from both web and Android environments. We identify the common underlying root causes of flakiness in the UI tests, the strategies used to manifest the flaky behavior, and the fixing strategies used to remedy flaky UI tests. The findings made in this work can provide a foundation for the development of detection and prevention techniques for flakiness arising in UI tests.
Comments: 13 pages, 16 figures, 6 tables
Subjects: Software Engineering (cs.SE)
Cite as: arXiv:2103.02669 [cs.SE]
  (or arXiv:2103.02669v1 [cs.SE] for this version)

Submission history

From: Alan Romano [view email]
[v1] Wed, 3 Mar 2021 20:40:41 GMT (571kb,D)

Link back to: arXiv, form interface, contact.