McMinn, P. orcid.org/0000-0001-9137-7433, Roslan, M.F. and Kapfhammer, G.M. (2025) Beyond test flakiness: a manifesto for a holistic approach to test suite health. In: 2025 IEEE/ACM International Flaky Tests Workshop (FTW). 2nd International Flaky Tests Workshop 2025 (FTW 2025), 27 Apr 2025, Ottawa, Ontario, Canada. Institute of Electrical and Electronics Engineers (IEEE) ISBN 979-8-3315-0232-4
Abstract
Large numbers of flaky tests are a sure sign of a dysfunctional, or “unhealthy”, test suite. In this paper, we identify several further indicators of unhealthy test suites, arguing that trade-offs exist among some indicators, with others complementing one another. We encourage researchers and practitioners not to stop at test flakiness-or any individual metric of “good” or “bad” tests-and instead focus on developing and adopting techniques and tools that holistically address test suite health. For more information, see https://philmcminn.com/test-suite-health.
Metadata
Item Type: | Proceedings Paper |
---|---|
Authors/Creators: |
|
Copyright, Publisher and Additional Information: | © 2025 The Author(s). Except as otherwise noted, this author-accepted version of a proceedings paper published in 2025 IEEE/ACM International Flaky Tests Workshop (FTW) is made available via the University of Sheffield Research Publications and Copyright Policy under the terms of the Creative Commons Attribution 4.0 International License (CC-BY 4.0), which permits unrestricted use, distribution and reproduction in any medium, provided the original work is properly cited. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ |
Dates: |
|
Institution: | The University of Sheffield |
Academic Units: | The University of Sheffield > Faculty of Engineering (Sheffield) > Department of Computer Science (Sheffield) |
Funding Information: | Funder Grant number ENGINEERING AND PHYSICAL SCIENCE RESEARCH COUNCIL EP/X024539/1 |
Depositing User: | Symplectic Sheffield |
Date Deposited: | 10 Feb 2025 16:40 |
Last Modified: | 08 Jul 2025 11:48 |
Status: | Published |
Publisher: | Institute of Electrical and Electronics Engineers (IEEE) |
Refereed: | Yes |
Identification Number: | 10.1109/FTW66604.2025.00007 |
Related URLs: | |
Open Archives Initiative ID (OAI ID): | oai:eprints.whiterose.ac.uk:223068 |