Clark, A., Walkinshaw, N. and Hierons, R. orcid.org/0000-0002-4771-1446 (2021) Test case generation for agent-based models : a systematic literature review. Information and Software Technology, 135. 106567. ISSN 0950-5849
Abstract
Context: Agent-based models play an important role in simulating complex emergent phenomena and supporting critical decisions. In this context, a software fault may result in poorly informed decisions that lead to disastrous consequences. The ability to rigorously test these models is therefore essential.
Objective: Our objective is to summarise the state-of-the-art techniques for test case generation in agent-based models and identify future research directions.
Method: We have conducted a systematic literature review in which we pose five research questions related to the key aspects of test case generation in agent-based models: What are the information artifacts used to generate tests? How are these tests generated? How is a verdict assigned to a generated test? How is the adequacy of a generated test suite measured? What level of abstraction of an agent-based model is targeted by a generated test?
Results: Out of the 464 initial search results, we identified 24 primary publications. Based on these primary publications, we formed a taxonomy to summarise the state-of-the-art techniques for test case generation in agent-based models. Our results show that whilst the majority of techniques are effective for testing functional requirements at the agent and integration levels of abstraction, there are comparatively few techniques capable of testing society-level behaviour. Furthermore, the majority of techniques cannot test non-functional requirements or “soft goals”.
Conclusions: This paper reports insights into the key developments and open challenges concerning test case generation in agent-based models that may be of interest to both researchers and practitioners. In particular, we identify the need for test case generation techniques that focus on societal and non-functional behaviour, and a more thorough evaluation using realistic case studies that feature challenging properties associated with a typical agent-based model.
Metadata
Item Type: | Article |
---|---|
Authors/Creators: |
|
Copyright, Publisher and Additional Information: | © 2021 Elsevier B.V. This is an author produced version of a paper subsequently published in Information and Software Technology. Uploaded in accordance with the publisher's self-archiving policy. Article available under the terms of the CC-BY-NC-ND licence (https://creativecommons.org/licenses/by-nc-nd/4.0/). |
Keywords: | Agent-Based Modelling; Multi-Agent Systems; Software Testing; Test Case Generation; Systematic Literature Review |
Dates: |
|
Institution: | The University of Sheffield |
Academic Units: | The University of Sheffield > Faculty of Engineering (Sheffield) > Department of Computer Science (Sheffield) |
Depositing User: | Symplectic Sheffield |
Date Deposited: | 15 Mar 2021 08:42 |
Last Modified: | 17 Mar 2022 01:38 |
Status: | Published |
Publisher: | Elsevier |
Refereed: | Yes |
Identification Number: | 10.1016/j.infsof.2021.106567 |
Open Archives Initiative ID (OAI ID): | oai:eprints.whiterose.ac.uk:171846 |