Piotrkowicz, A orcid.org/0000-0002-7723-699X, Johnson, O orcid.org/0000-0003-3998-541X and Hall, G orcid.org/0000-0002-8864-5932 (2019) Finding relevant free-text radiology reports at scale with IBM Watson Content Analytics: a feasibility study in the UK NHS. Journal of Biomedical Semantics, 10. 21. ISSN 2041-1480
Abstract
Background. Significant amounts of health data are stored as free-text within clinical reports, letters, discharge summaries and notes. Busy clinicians have limited time to read such large amounts of free-text and are at risk of information overload and consequently missing information vital to patient care. Automatically identifying relevant information at the point of care has the potential to reduce these risks but represents a considerable research challenge. One software solution that has been proposed in industry is the IBM Watson analytics suite which includes rule-based analytics capable of processing large document collections at scale.
Results. In this paper we present an overview of IBM Watson Content Analytics and a feasibility study using Content Analytics with a large-scale corpus of clinical free-text reports within a UK National Health Service (NHS) context. We created dictionaries and rules for identifying positive incidence of hydronephrosis and brain metastasis from 5.6m radiology reports and were able to achieve 94% precision, 95% recall and 89% precision, 94% recall respectively on a sample of manually annotated reports. With minor changes for US English we applied the same rule set to an open access corpus of 0.5m radiology reports from a US hospital and achieved 93% precision, 94% recall and 84% precision, 88% recall respectively.
Conclusions. We were able to implement IBM Watson within a UK NHS context and demonstrate effective results that could provide clinicians with an automatic safety net which highlights clinically important information within free-text documents. Our results suggest that currently available technologies such as IBM Watson Content Analytics already have the potential to address information overload and improve clinical safety and that solutions developed in one hospital and country may be transportable to different hospitals and countries. Our study was limited to exploring technical aspects of the feasibility of one industry solution and we recognise that healthcare text analytics research is a fast moving field. That said, we believe our study suggests that text analytics is sufficiently advanced to be implemented within industry solutions that can improve clinical safety.
Metadata
Item Type: | Article |
---|---|
Authors/Creators: |
|
Copyright, Publisher and Additional Information: | © The Author(s). 2019. This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated. |
Keywords: | Information retrieval; Natural language processing; Radiology; Feasibility study; Rule-based system |
Dates: |
|
Institution: | The University of Leeds |
Academic Units: | The University of Leeds > Faculty of Engineering & Physical Sciences (Leeds) > School of Computing (Leeds) |
Funding Information: | Funder Grant number Wellcome Trust 204825/Z/16/Z |
Depositing User: | Symplectic Publications |
Date Deposited: | 04 Oct 2019 09:38 |
Last Modified: | 25 Jun 2023 21:44 |
Status: | Published |
Publisher: | BioMed Central |
Identification Number: | 10.1186/s13326-019-0213-5 |
Open Archives Initiative ID (OAI ID): | oai:eprints.whiterose.ac.uk:143205 |