Gu, R., Rojas, J.M. and Shin, D. orcid.org/0000-0002-0840-6449 (Accepted: 2024) Can test generation and program repair inform automated assessment of programming projects? In: 2025 IEEE Conference on Software Testing, Verification and Validation (ICST). 18th IEEE International Conference on Software Testing, Verification and Validation (ICST) 2025, 31 Mar - 04 Apr 2025, Naples, Italy. Institute of Electrical and Electronics Engineers (IEEE) (In Press)
Abstract
Computer Science educators assessing student programming assignments are typically responsible for two challenging tasks: grading and providing feedback. Producing grades that are fair and feedback that is useful to students is a goal common to most educators. In this context, automated test generation and program repair offer promising solutions for detecting bugs and suggesting corrections in students’ code which could be leveraged to inform grading and feedback generation. Previous research on the applicability of these techniques to simple programming tasks (e.g., single-method algorithms) has shown promising results, but their effectiveness for more complex programming tasks remains unexplored. To fill this gap, this paper investigates the feasibility of applying existing test generation and program repair tools for assessing complex programming assignment projects. In a case study using a real-world Java programming assignment project with 296 incorrect student submissions, we found that generated tests were insufficient in detecting bugs in over 50% of cases, while full repairs could only be automatically generated for only 2.1% of submissions. Our findings indicate significant limitations in current tools for detecting bugs and repairing student submissions, highlighting the need for more advanced techniques to support automated assessment of complex assignment projects.
Metadata
Item Type: | Proceedings Paper |
---|---|
Authors/Creators: |
|
Copyright, Publisher and Additional Information: | © 2025 The Author(s). |
Keywords: | Automated Assessment; Test Generation; Program Repair |
Dates: |
|
Institution: | The University of Sheffield |
Academic Units: | The University of Sheffield > Faculty of Engineering (Sheffield) > Department of Computer Science (Sheffield) |
Depositing User: | Symplectic Sheffield |
Date Deposited: | 21 Feb 2025 10:21 |
Last Modified: | 21 Feb 2025 10:22 |
Status: | In Press |
Publisher: | Institute of Electrical and Electronics Engineers (IEEE) |
Refereed: | Yes |
Related URLs: | |
Open Archives Initiative ID (OAI ID): | oai:eprints.whiterose.ac.uk:223625 |
Download
Filename: ICST2025-Edu.pdf
