Arcuschin, I., Ciccaroni, C., Galeotti, J.P. et al. (1 more author) (2022) On the feasibility and challenges of synthesizing executable Espresso tests. In: Garrido, A. and Wong, W.E., (eds.) AST '22: Proceedings of the 3rd ACM/IEEE International Conference on Automation of Software Test. AST '22: IEEE/ACM 3rd International Conference on Automation of Software Test, 17-18 May 2022, Pittsburgh, PA, USA. Association for Computing Machinery , pp. 92-102. ISBN 9781450392860
Abstract
Several tools have been proposed to automatically test Android applications, achieving outstanding results in terms of both code coverage and crash discovery. While useful for crash reproduction and bug-fixing, these tools usually do not present the generated interactions in a format that motivates developers to read and modify such tests later on. This hinders the ability of developers to add those tests to their existing test suites, or adapt them to new scenarios- common practices in modern software development where tests are maintained and evolve alongside production code. In this work we present an empirical study on the challenges of automatically synthesizing Espresso test suites from sequences of interactions over widgets. We build on top of the MATE testing tool and implement a prototype that enables this study. The prototype is then evaluated on 12 open-source Android apps, followed by an analysis and discussion of challenges and limitations. We also include feedback from developers of open-source projects and an industrial app. Our empirical study shows that the creation of Espresso tests is difficult, mostly due to the lack of unique properties to unambiguously identify specific widgets in the UI. This problem is aggravated in some cases by the incomplete or ambiguous definition of GUI components and layouts. It also points out that further research is needed to find ways to improve the testability of Android apps either manually or automatically. Nonetheless, the feedback received indicates that the synthesized Espresso tests are still useful for projects with few or no test cases, serving as a starting point for creating new ones.
Metadata
Item Type: | Proceedings Paper |
---|---|
Authors/Creators: |
|
Editors: |
|
Copyright, Publisher and Additional Information: | © 2022 Copyright held by the owner/author(s). |
Keywords: | Android test generation; Espresso tests; widget-based test generators |
Dates: |
|
Institution: | The University of Sheffield |
Academic Units: | The University of Sheffield > Faculty of Engineering (Sheffield) > Department of Computer Science (Sheffield) |
Depositing User: | Symplectic Sheffield |
Date Deposited: | 29 Jul 2022 13:06 |
Last Modified: | 29 Jul 2022 13:06 |
Published Version: | https://ieeexplore.ieee.org/document/9796443 |
Status: | Published |
Publisher: | Association for Computing Machinery |
Refereed: | Yes |
Identification Number: | 10.1145/3524481.3527234 |
Open Archives Initiative ID (OAI ID): | oai:eprints.whiterose.ac.uk:189258 |