Elgendy, I. orcid.org/0000-0002-8416-5480, Hierons, R. orcid.org/0000-0002-4771-1446 and Mcminn, P. orcid.org/0000-0001-9137-7433 (2024) Evaluating string distance metrics for reducing automatically generated test suites. In: Lonetti, F., Guerriero, A., Saadatmand, M., Budnik, C.J. and Li, J., (eds.) AST '24: Proceedings of the 5th ACM/IEEE International Conference on Automation of Software Test (AST 2024). AST '24: 5th ACM/IEEE International Conference on Automation of Software Test (AST 2024), 15-16 Apr 2024, Lisbon, Portugal. The Association for Computing Machinery (ACM) , pp. 171-181. ISBN: 9798400705885 ISSN: 2377-8628 EISSN: 2833-9061
Abstract
Regression test suites can have a large number of test cases, especially automatically generated ones, and tend to grow in size, making it costly to run the entire test suite. Test suite reduction aims to eliminate some test cases to reduce the test suite size and therefore reduce the cost of running it. In this paper, string distances on the text of the test cases are used as measures of similarity for reduction. A practical benefit of using string distance is that there is no need to run the test cases: The test suite source code is the only requirement, making the approach fast. We reduce test suites generated from Randoop and EvoSuite; two well-known test generation tools of Java programs. We implemented a string-based similarity reduction and compared it against random reduction. In the experiments, mutation scores using reduced test suites based on maximising string dissimilarity of test cases were higher than those for random reduction in over 70% of the test suites generated. Also, the results showed that test suites generated by Randoop can be drastically reduced in one case by 99% using the string-based similarity reduction approach while maintaining the fault-finding capabilities of the original test suite. Finally, on average, the normalised compression distance was found to be the best similarity metric choice in terms of fault-detection.
Metadata
Item Type: | Proceedings Paper |
---|---|
Authors/Creators: |
|
Editors: |
|
Copyright, Publisher and Additional Information: | © 2024 The Authors. Except as otherwise noted, this author-accepted version of a paper published in AST '24: Proceedings of the 5th ACM/IEEE International Conference on Automation of Software Test (AST 2024) is made available via the University of Sheffield Research Publications and Copyright Policy under the terms of the Creative Commons Attribution 4.0 International License (CC-BY 4.0), which permits unrestricted use, distribution and reproduction in any medium, provided the original work is properly cited. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ |
Keywords: | test suite reduction; similarity-based testing; diversity-based testing; automatically generated tests |
Dates: |
|
Institution: | The University of Sheffield |
Academic Units: | The University of Sheffield > Faculty of Engineering (Sheffield) > Department of Computer Science (Sheffield) |
Depositing User: | Symplectic Sheffield |
Date Deposited: | 06 Aug 2025 13:58 |
Last Modified: | 06 Aug 2025 13:59 |
Status: | Published |
Publisher: | The Association for Computing Machinery (ACM) |
Refereed: | Yes |
Identification Number: | 10.1145/3644032.3644455 |
Related URLs: | |
Open Archives Initiative ID (OAI ID): | oai:eprints.whiterose.ac.uk:230089 |