Cho, E. orcid.org/0000-0002-4293-945X, Shin, D. orcid.org/0000-0002-0840-6449 and Ko, I.-Y. orcid.org/0000-0002-3843-263X (2026) Evolution-aware regression test prioritization of ML-enabled systems using gradient-based behavior vectors. In: Winter, S., Nejati, S. and Petke, J., (eds.) ASE '26: Proceedings of the 41st IEEE/ACM International Conference on Automated Software Engineering. ASE '26: 41st IEEE/ACM International Conference on Automated Software Engineering, 12-16 Oct 2026, Munich, Germany. ACM, pp. 474-485. ISBN: 9798400728822. ISSN: 1938-4300. EISSN: 9174-8585.
Abstract
The machine learning(ML) component of an ML-enabled system evolves through retraining, fine-tuning, and optimization, so previously valid test results may no longer hold. A single evolution step can worsen performance on some test cases while improving others, making regression test prioritization inherently directional. We present Gradient-based Behavior Vector-Parameter Delta(GBV-PD), the first approach to operationalize the behavior vector space for evolution-aware regression test prioritization. GBV-PD represents each test case as a gradient-based vector(GBV), a low-dimensional projection of its loss gradient under the original model. It then projects the observed parameter update of the evolved model onto the same PCA basis and uses the resulting alignment to estimate whether each test case's loss is likely to increase or decrease, without running the evolved model on test cases during prioritization. In an empirical study across classification and regression tasks, GBV-PD consistently outperformed non-directional baselines and remained competitive with a full-gradient reference, while offering better time and storage profiles for repeated updates via reusable GBV caching. These results show that behavior-space ideas can be operationalized into a practical and efficient mechanism for repeated-update regression-test prioritization of evolving ML components within ML-enabled systems.
Metadata
| Item Type: | Proceedings Paper |
|---|---|
| Authors/Creators: | |
| Editors: |
|
| Copyright, Publisher and Additional Information: | © 2026 Copyright held by the owner/author(s). This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivs International 4.0 License. (https://creativecommons.org/licenses/by-nc-nd/4.0/) |
| Keywords: | Regression Testing; Test Prioritization; ML-Enabled Systems; EvolutionAware Testing; Gradient-Based Behavior Vectors; Behavior Vector; Space Testing |
| Dates: |
|
| Institution: | The University of Sheffield |
| Academic Units: | The University of Sheffield > Faculty of Engineering (Sheffield) > Department of Computer Science (Sheffield) |
| Date Deposited: | 16 Oct 2026 12:00 |
| Last Modified: | 16 Oct 2026 12:01 |
| Status: | Published |
| Publisher: | ACM |
| Refereed: | Yes |
| Identification Number: | 10.1145/3832783.3834368 |
| Related URLs: | |
| Open Archives Initiative ID (OAI ID): | oai:eprints.whiterose.ac.uk:246771 |

CORE (COnnecting REpositories)
CORE (COnnecting REpositories)