The Relationship between IR Effectiveness Measures and User Satisfaction

Al-Maskari, A., Sanderson, M. and Clough, P. (2007) The Relationship between IR Effectiveness Measures and User Satisfaction. In: Proceedings of the 30th annual international ACM SIGIR conference on Research and development in information retrieval. SIGIR '07, July 23-27, 2007, Amsterdam, The Netherlands.. ACM , New York, USA , pp. 773-774. ISBN 978-1-59593-597-7

Abstract

This paper presents an experimental study of users assessing the quality of Google web search results. In particular we look at how users' satisfaction correlates with the effectiveness of Google as quantified by IR measures such as precision and the suite of Cumulative Gain measures (CG, DCG, NDCG). Results indicate strong correlation between users' satisfaction, CG and precision, moderate correlation with DCG, with perhaps surprisingly negligible correlation with NDCG. The reasons for the low correlation with NDCG are examined.

Metadata

Authors/Creators:	Al-Maskari, A. Sanderson, M. Clough, P.
Keywords:	User satisfaction, IR effectiveness measures
Dates:	2007
Institution:	The University of Sheffield
Academic Units:	The University of Sheffield > Faculty of Social Sciences (Sheffield) > Information School (Sheffield)
Depositing User:	Repository Officer
Date Deposited:	19 Nov 2008 10:32
Last Modified:	08 Feb 2013 16:58
Published Version:	http://dx.doi.org/10.1145/1277741.1277902
Status:	Published
Publisher:	ACM
Identification Number:	https://doi.org/10.1145/1277741.1277902

CORE (COnnecting REpositories)

The Relationship between IR Effectiveness Measures and User Satisfaction

Abstract

Metadata

Download

SIGIR2005-maskari

Export

Statistics