Walkinshaw, N. orcid.org/0000-0003-2134-6548, Foster, M., Rojas, J.M. et al. (1 more author) (Accepted: 2024) Bounding random test set size with computational learning theory. In: Proceedings of the ACM on Software Engineering (PACMSE). ACM International Conference on the Foundations of Software Engineering (FSE 2024), 17-19 Jul 2024, Porto de Galinhas, Brazil. Association for Computing Machinery . (In Press)
Abstract
Random testing approaches work by generating inputs at random, or by selecting inputs randomly from some pre-defined operational profile. One long-standing question that arises in this and other testing contexts is as follows: When can we stop testing? At what point can we be certain that executing further tests in this manner will not explore previously untested (and potentially buggy) software behaviors? This is analogous to the question in Machine Learning, of how many training examples are required in order to infer an accurate model. In this paper we show how probabilistic approaches to answer this question in Machine Learning (arising from Computational Learning Theory) can be applied in our testing context, to provide an upper-bound on the number of tests required to achieve a given level of adequacy. We validate this bound on a large set of Java units, and an autonomous driving system.
Metadata
Item Type: | Proceedings Paper | ||||
---|---|---|---|---|---|
Authors/Creators: |
|
||||
Copyright, Publisher and Additional Information: | © 2024 ACM. |
||||
Dates: |
|
||||
Institution: | The University of Sheffield | ||||
Academic Units: | The University of Sheffield > Faculty of Engineering (Sheffield) > Department of Computer Science (Sheffield) | ||||
Funding Information: |
|
||||
Depositing User: | Symplectic Sheffield | ||||
Date Deposited: | 24 Apr 2024 10:22 | ||||
Last Modified: | 02 May 2024 15:54 | ||||
Status: | In Press | ||||
Publisher: | Association for Computing Machinery | ||||
Refereed: | Yes | ||||
Identification Number: | https://doi.org/10.1145/3660819 | ||||
Related URLs: |
Download
Filename: Random_Testing_Learning_Theory (10).pdf