Clough, P.D. (2015) Evaluation: Thinking Outside the (Search) Box. In: FIRE '14 Proceedings of the Forum for Information Retrieval Evaluation. FIRE '14 Forum for Information Retrieval Evaluation, 05-07 Dec 2014, Bangalore, India. ACM , New York, NY, USA , pp. 1-9. ISBN 9781450337557
Abstract
Evaluation of IR systems has typically focused on the system and specifically assessing the quality of a ranked list of results with respect to a query. However, IR functionality is typically just one component amongst many that are used to help support users' wider information seeking activities. Many systems that include a search box also provide features, such as faceted lists, subject hierarchies, visualizations and recommendations to help users find information. In this paper I discuss experiences gained from developing a system to support exploration and discovery in digital cultural heritage. In particular I focus on the development of system components to support search and navigation and how the different components were evaluated within the development life-cycle of the project. The importance of taking a holistic approach to evaluation, as well as utilising evaluation approaches from domains other than IR, is emphasized. In short, we need to be thinking outside the (search) box when it comes to evaluation in IR.
Metadata
Item Type: | Proceedings Paper |
---|---|
Authors/Creators: |
|
Copyright, Publisher and Additional Information: | © 2015 Association for Computing Machinery (ACM). This is an author produced version of a paper subsequently published in FIRE '14 Proceedings of the Forum for Information Retrieval Evaluation. Uploaded in accordance with the publisher's self-archiving policy. |
Dates: |
|
Institution: | The University of Sheffield |
Academic Units: | The University of Sheffield > Faculty of Social Sciences (Sheffield) > Information School (Sheffield) |
Depositing User: | Symplectic Sheffield |
Date Deposited: | 03 Mar 2016 12:26 |
Last Modified: | 19 Dec 2022 13:32 |
Published Version: | http://dx.doi.org/10.1145/2824864.2824890 |
Status: | Published |
Publisher: | ACM |
Refereed: | Yes |
Open Archives Initiative ID (OAI ID): | oai:eprints.whiterose.ac.uk:93016 |