Bin Hezam, R. and Stevenson, R.M. orcid.org/0000-0002-9483-6006 (2024) RLStop: a reinforcement learning stopping method for TAR. In: SIGIR '24: Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval. The 47th International ACM SIGIR Conference on Research and Development in Information Retrieval, 14-18 Jul 2024, Washington D.C., USA. ACM Digital Library , pp. 2604-2608. ISBN 9798400704314
Abstract
We present RLStop, a novel Technology Assisted Review (TAR) stopping rule based on reinforcement learning that helps minimise the number of documents that need to be manually reviewed within TAR applications. RLStop is trained on example rankings using a reward function to identify the optimal point to stop examining documents. Experiments at a range of target recall levels on multiple benchmark datasets (CLEF e-Health, TREC Total Recall, and Reuters RCV1) demonstrated that RLStop substantially reduces the workload required to screen a document collection for relevance. RLStop outperforms a wide range of alternative approaches, achieving performance close to the maximum possible for the task under some circumstances.
Metadata
Item Type: | Proceedings Paper |
---|---|
Authors/Creators: |
|
Copyright, Publisher and Additional Information: | © 2024 Copyright held by the owner/author(s). This work is licensed under a Creative Commons Attribution International 4.0 License. (https://creativecommons.org/licenses/by/4.0/) |
Keywords: | Reinforcement Learning; Deep Reinforcement Learning; Technology Assisted Review; TAR; Stopping Methods |
Dates: |
|
Institution: | The University of Sheffield |
Academic Units: | The University of Sheffield > Faculty of Engineering (Sheffield) > Department of Computer Science (Sheffield) |
Depositing User: | Symplectic Sheffield |
Date Deposited: | 01 May 2024 10:19 |
Last Modified: | 15 Jul 2024 10:14 |
Status: | Published |
Publisher: | ACM Digital Library |
Refereed: | Yes |
Identification Number: | 10.1145/3626772.3657911 |
Related URLs: | |
Open Archives Initiative ID (OAI ID): | oai:eprints.whiterose.ac.uk:212134 |